TY - GEN
T1 - Robust learning via ensemble density propagation in deep neural networks
AU - Carannante, Giuseppina
AU - Dera, Dimah
AU - Rasool, Ghulam
AU - Bouaynaya, Nidhal C.
AU - Mihaylova, Lyudmila
N1 - Funding Information:
This work was supported by the National Science Foundation Awards NSF ECCS-1903466, NSF CCF-1527822, NSF OAC-2008690. Giuseppina Carannante is supported by the US Department of Education through a Graduate Assistance in Areas of National Need (GAANN) Program Award Number P200A180055. We are also grateful to UK EPSRC support through EP/T013265/1 project NSF-EPSRC: ShiRAS. Towards Safe and Reliable Autonomy in Sensor Driven Systems.
Publisher Copyright:
© 2020 IEEE.
PY - 2020/9
Y1 - 2020/9
N2 - Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.
AB - Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.
UR - http://www.scopus.com/inward/record.url?scp=85096486228&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85096486228&partnerID=8YFLogxK
U2 - 10.1109/MLSP49062.2020.9231635
DO - 10.1109/MLSP49062.2020.9231635
M3 - Conference contribution
AN - SCOPUS:85096486228
T3 - IEEE International Workshop on Machine Learning for Signal Processing, MLSP
BT - Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020
PB - IEEE Computer Society
T2 - 30th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2020
Y2 - 21 September 2020 through 24 September 2020
ER -