Secure and Robust deep learning in computer vision

Abstract

In recent years, deep learning has become the foundation of modern computer vision applications, enabling machines to recognize objects, understand scenes, and make decisions based on visual data. However, deep neural networks can be vulnerable to security threats and unstable behavior, especially when exposed to adversarial inputs, poisoned training data, or complex real-world environments.

This thesis presents three research efforts to improve the security and robustness of deep learning in computer vision.

First, we propose a new defense method called Multi-Pronged Defense (MPD), which protects deep neural networks from backdoor attacks—hidden manipulations that cause models to behave incorrectly when triggered. MPD combines semi-supervised learning, balanced data sampling, and neuron suppression to effectively block various backdoor strategies across different datasets.

Second, we design a novel attention mechanism for vision transformers that incorporates position-aware operations. This structure improves the model’s sensitivity to spatial patterns, similar to convolutional neural networks (CNNs), and achieves better performance than traditional self-attention while maintaining architectural flexibility.

Third, we introduce the Balanced Object Detector (BOD), a new object detection framework that does not rely on feature pyramid networks (FPNs). By using consistent receptive fields and parameter sharing across detection branches, BOD achieves higher accuracy on small and medium objects and shows better generalization and resistance to adversarial attacks.

Together, these contributions advance the development of deep learning models that are not only accurate, but also secure, stable, and reliable in real-world visual applications.