Variance Guided Continual Learning in a Convolutional Neural Network Gaussian Process Single Classifier Approach for Multiple Tasks in Noisy Images

Mahed Javed, Lyudmila Mihaylova, Nidhal Bouaynaya

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    This work provides a continual learning solution in a single-classifier to multiple classification tasks with various data sets. A Gaussian process (GP) is combined with a Convolutional Neural Network (CNN) feature extractor architecture (CNNGP). Post softmax samples are used to estimate the variance. The variance is characterising the impact of uncertainties and is part of the update process for the learning rate parameters. Within the proposed framework two learning approaches are adopted: 1) in the first, the weights of the CNN are deterministic and only the GP learning rate is updated, 2) in the second setting, prior distributions are adopted for the CNN weights. Both the learning rates of the CNN and the GP are updated. The algorithm is trained on two variants of the MNIST dataset, split-MNIST and permuted-MNIST. Results are compared with the Uncertainty Guided Continual Bayesian Networks (UCB) multi-classifier approach [1]. The validation shows that the proposed algorithm in the Bayesian setting outperforms the UCB in tasks subject to Gaussian noise image noises and shows robustness.

    Original languageEnglish (US)
    Title of host publicationProceedings of 2021 IEEE 24th International Conference on Information Fusion, FUSION 2021
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    ISBN (Electronic)9781737749714
    StatePublished - 2021
    Event24th IEEE International Conference on Information Fusion, FUSION 2021 - Sun City, South Africa
    Duration: Nov 1 2021Nov 4 2021

    Publication series

    NameProceedings of 2021 IEEE 24th International Conference on Information Fusion, FUSION 2021

    Conference

    Conference24th IEEE International Conference on Information Fusion, FUSION 2021
    Country/TerritorySouth Africa
    CitySun City
    Period11/1/2111/4/21

    All Science Journal Classification (ASJC) codes

    • Information Systems
    • Signal Processing
    • Information Systems and Management

    Fingerprint

    Dive into the research topics of 'Variance Guided Continual Learning in a Convolutional Neural Network Gaussian Process Single Classifier Approach for Multiple Tasks in Noisy Images'. Together they form a unique fingerprint.

    Cite this