Robust learning via ensemble density propagation in deep neural networks

Giuseppina Carannante, Dimah Dera, Ghulam Rasool, Nidhal C. Bouaynaya, Lyudmila Mihaylova

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.

Original languageEnglish (US)
Title of host publicationProceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020
PublisherIEEE Computer Society
ISBN (Electronic)9781728166629
DOIs
StatePublished - Sep 2020
Event30th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2020 - Virtual, Espoo, Finland
Duration: Sep 21 2020Sep 24 2020

Publication series

NameIEEE International Workshop on Machine Learning for Signal Processing, MLSP
Volume2020-September
ISSN (Print)2161-0363
ISSN (Electronic)2161-0371

Conference

Conference30th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2020
CountryFinland
CityVirtual, Espoo
Period9/21/209/24/20

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Signal Processing

Fingerprint Dive into the research topics of 'Robust learning via ensemble density propagation in deep neural networks'. Together they form a unique fingerprint.

Cite this