Robust learning via ensemble density propagation in deep neural networks

Giuseppina Carannante; Dimah Dera; Ghulam Rasool; Nidhal C. Bouaynaya; Lyudmila Mihaylova

doi:10.1109/MLSP49062.2020.9231635

Robust learning via ensemble density propagation in deep neural networks

Giuseppina Carannante, Dimah Dera, Ghulam Rasool, Nidhal C. Bouaynaya, Lyudmila Mihaylova

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

6 Scopus citations

Abstract

Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.

Original language	English (US)
Title of host publication	Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020
Publisher	IEEE Computer Society
ISBN (Electronic)	9781728166629
DOIs	https://doi.org/10.1109/MLSP49062.2020.9231635
State	Published - Sep 2020
Event	30th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2020 - Virtual, Espoo, Finland Duration: Sep 21 2020 → Sep 24 2020

Publication series

Name	IEEE International Workshop on Machine Learning for Signal Processing, MLSP
Volume	2020-September
ISSN (Print)	2161-0363
ISSN (Electronic)	2161-0371

Conference

Conference	30th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2020
Country/Territory	Finland
City	Virtual, Espoo
Period	9/21/20 → 9/24/20

All Science Journal Classification (ASJC) codes

Human-Computer Interaction
Signal Processing

Access to Document

10.1109/MLSP49062.2020.9231635

Cite this

Carannante, G., Dera, D., Rasool, G., Bouaynaya, N. C., & Mihaylova, L. (2020). Robust learning via ensemble density propagation in deep neural networks. In Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020 Article 9231635 (IEEE International Workshop on Machine Learning for Signal Processing, MLSP; Vol. 2020-September). IEEE Computer Society. https://doi.org/10.1109/MLSP49062.2020.9231635

@inproceedings{4eae39ffe9dd451fb8bbc84255e6464c,

title = "Robust learning via ensemble density propagation in deep neural networks",

abstract = "Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.",

author = "Giuseppina Carannante and Dimah Dera and Ghulam Rasool and Bouaynaya, {Nidhal C.} and Lyudmila Mihaylova",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 30th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2020 ; Conference date: 21-09-2020 Through 24-09-2020",

year = "2020",

month = sep,

doi = "10.1109/MLSP49062.2020.9231635",

language = "English (US)",

series = "IEEE International Workshop on Machine Learning for Signal Processing, MLSP",

publisher = "IEEE Computer Society",

booktitle = "Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020",

address = "United States",

}

Carannante, G, Dera, D, Rasool, G, Bouaynaya, NC & Mihaylova, L 2020, Robust learning via ensemble density propagation in deep neural networks. in Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020., 9231635, IEEE International Workshop on Machine Learning for Signal Processing, MLSP, vol. 2020-September, IEEE Computer Society, 30th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2020, Virtual, Espoo, Finland, 9/21/20. https://doi.org/10.1109/MLSP49062.2020.9231635

Robust learning via ensemble density propagation in deep neural networks. / Carannante, Giuseppina; Dera, Dimah; Rasool, Ghulam et al.
Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020. IEEE Computer Society, 2020. 9231635 (IEEE International Workshop on Machine Learning for Signal Processing, MLSP; Vol. 2020-September).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Robust learning via ensemble density propagation in deep neural networks

AU - Carannante, Giuseppina

AU - Dera, Dimah

AU - Rasool, Ghulam

AU - Bouaynaya, Nidhal C.

AU - Mihaylova, Lyudmila

PY - 2020/9

Y1 - 2020/9

N2 - Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.

AB - Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.

UR - http://www.scopus.com/inward/record.url?scp=85096486228&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85096486228&partnerID=8YFLogxK

U2 - 10.1109/MLSP49062.2020.9231635

DO - 10.1109/MLSP49062.2020.9231635

M3 - Conference contribution

AN - SCOPUS:85096486228

T3 - IEEE International Workshop on Machine Learning for Signal Processing, MLSP

BT - Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020

PB - IEEE Computer Society

T2 - 30th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2020

Y2 - 21 September 2020 through 24 September 2020

ER -

Carannante G, Dera D, Rasool G, Bouaynaya NC, Mihaylova L. Robust learning via ensemble density propagation in deep neural networks. In Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020. IEEE Computer Society. 2020. 9231635. (IEEE International Workshop on Machine Learning for Signal Processing, MLSP). doi: 10.1109/MLSP49062.2020.9231635

Robust learning via ensemble density propagation in deep neural networks

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this