Self-compression in bayesian neural networks

Giuseppina Carannante, Dimah Dera, Ghulam Rasool, Nidhal C. Bouaynaya

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Machine learning models have achieved human-level performance on various tasks. This success comes at a high cost of computation and storage overhead, which makes machine learning algorithms difficult to deploy on edge devices. Typically, one has to partially sacrifice accuracy in favor of an increased performance quantified in terms of reduced memory usage and energy consumption. Current methods compress the networks by reducing the precision of the parameters or by eliminating redundant ones. In this paper, we propose a new insight into network compression through the Bayesian framework. We show that Bayesian neural networks automatically discover redundancy in model parameters, thus enabling self-compression, which is linked to the propagation of uncertainty through the layers of the network. Our experimental results show that the network architecture can be successfully compressed by deleting parameters identified by the network itself while retaining the same level of accuracy.

Original languageEnglish (US)
Title of host publicationProceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing, MLSP 2020
PublisherIEEE Computer Society
ISBN (Electronic)9781728166629
DOIs
StatePublished - Sep 2020
Event30th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2020 - Virtual, Espoo, Finland
Duration: Sep 21 2020Sep 24 2020

Publication series

NameIEEE International Workshop on Machine Learning for Signal Processing, MLSP
Volume2020-September
ISSN (Print)2161-0363
ISSN (Electronic)2161-0371

Conference

Conference30th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2020
Country/TerritoryFinland
CityVirtual, Espoo
Period9/21/209/24/20

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Signal Processing

Fingerprint

Dive into the research topics of 'Self-compression in bayesian neural networks'. Together they form a unique fingerprint.

Cite this