Extended Variational Inference for Propagating Uncertainty in Convolutional Neural Networks

Dimah Dera, Ghulam Rasool, Nidhal Bouaynaya

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations

Abstract

Model confidence or uncertainty is critical in autonomous systems as they directly tie to the safety and trustworthiness of the system. The quantification of uncertainty in the output decisions of deep neural networks (DNNs) is a challenging problem. The Bayesian framework enables the estimation of the predictive uncertainty by introducing probability distributions over the (unknown) network weights; however, the propagation of these high-dimensional distributions through multiple layers and non-linear transformations is mathematically intractable. In this work, we propose an extended variational inference (eVI) framework for convolutional neural network (CNN) based on tensor Normal distributions (TNDs) defined over convolutional kernels. Our proposed eVI framework propagates the first two moments (mean and covariance) of these TNDs through all layers of the CNN. We employ first-order Taylor series linearization to approximate the mean and covariances passing through the non-linear activations. The uncertainty in the output decision is given by the propagated covariance of the predictive distribution. Furthermore, we show, through extensive simulations on the MNIST and CIFAR-10 datasets, that the CNN becomes more robust to Gaussian noise and adversarial attacks.

Original languageEnglish (US)
Title of host publication2019 IEEE 29th International Workshop on Machine Learning for Signal Processing, MLSP 2019
PublisherIEEE Computer Society
ISBN (Electronic)9781728108247
DOIs
StatePublished - Oct 2019
Event29th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2019 - Pittsburgh, United States
Duration: Oct 13 2019Oct 16 2019

Publication series

NameIEEE International Workshop on Machine Learning for Signal Processing, MLSP
Volume2019-October
ISSN (Print)2161-0363
ISSN (Electronic)2161-0371

Conference

Conference29th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2019
Country/TerritoryUnited States
CityPittsburgh
Period10/13/1910/16/19

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Signal Processing

Fingerprint

Dive into the research topics of 'Extended Variational Inference for Propagating Uncertainty in Convolutional Neural Networks'. Together they form a unique fingerprint.

Cite this