Robust Bayesian Neural Networks by Spectral Expectation Bound Regularization

Models trained with Adversarial Samplings are more stable and perform better

Abstract

Bayesian neural networks have been widely used in many applications because of the distinctive probabilistic representation framework. Even though Bayesian neural networks have been found more robust to adversarial attacks compared with vanilla neural networks, their ability to deal with adversarial noises in practice is still limited. In this paper, we propose Spectral Expectation Bound Regularization (SEBR) to enhance the robustness of Bayesian neural networks. Our theoretical analysis reveals that training with SEBR improves the robustness to adversarial noises. We also prove that training with SEBR can reduce the epistemic uncertainty of the model and hence it can make the model more confident with the predictions, which verifies the robustness of the model from another point of view. Experiments on multiple Bayesian neural network structures and different adversarial attacks validate the correctness of the theoretical findings and the effectiveness of the proposed approach.

Publication
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Jiaru Zhang
Jiaru Zhang
Incoming postdoc researcher at Purdue University

My research interests include Bayesian Neural Networks, Adversarial Attack and Defense, and Causal Discovery.