Resampling Techniques for Learning under Extreme Verification Latency with Class Imbalance

Christopher Frederickson, Robi Polikar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A common, yet rarely addressed, real-world problem in computational intelligence applications is learning from non-stationary streaming data, where the underlying distribution of the data changes over time. This problem, also referred to as concept drift, is made even more challenging if, after initially receiving a small set of labeled data, the streaming data only consists of unlabeled data, requiring the learner to adapt to changing underlying distribution without the benefit of labeled data. This particular scenario is typically referred to as learning in initially labeled nonstationary environment, or as extreme verification latency (EVL), pointing to the fact that the label verification of the test data is indefinitely delayed. In our prior work, we have noted that current EVL algorithms - including the algorithm COMPOSE that we have developed - are largely unable to track changing distributions if the data drawn from those distributions are even mildly imbalanced. In this work, we integrate COMPOSE with 13 different resampling based modified algorithms, and compare accuracy, F1 score, and execution time. The results differed from what we originally expected and provided unique insight on how to choose a data rebalancing approach for different types of drift.

Original languageEnglish (US)
Title of host publication2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509060146
DOIs
StatePublished - Oct 10 2018
Event2018 International Joint Conference on Neural Networks, IJCNN 2018 - Rio de Janeiro, Brazil
Duration: Jul 8 2018Jul 13 2018

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2018-July

Other

Other2018 International Joint Conference on Neural Networks, IJCNN 2018
CountryBrazil
CityRio de Janeiro
Period7/8/187/13/18

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Resampling Techniques for Learning under Extreme Verification Latency with Class Imbalance'. Together they form a unique fingerprint.

Cite this