Learn++: An incremental learning algorithm for supervised neural networks

Robi Polikar, Lalita Udpa, Satish S. Udpa, Vasant Honavar

Research output: Contribution to journalArticlepeer-review

735 Scopus citations


We introduce Learn++, an algorithm for incremental training of neural network (NN) pattern classifiers. The proposed algorithm enables supervised NN paradigms, such as the multilayer perceptron (MLP), to accommodate new data, including examples that correspond to previously unseen classes. Furthermore, the algorithm does not require access to previously used data during subsequent incremental learning sessions, yet at the same time, it does not forget previously acquired knowledge. Learn++ utilizes ensemble of classifiers by generating multiple hypotheses using training data sampled according to carefully tailored distributions. The outputs of the resulting classifiers are combined using a weighted majority voting procedure. We present simulation results on several benchmark datasets as well as a real-world classification task. Initial results indicate that the proposed algorithm works rather well in practice. A theoretical upper bound on the error of the classifiers constructed by Learn++ is also provided.

Original languageEnglish (US)
Pages (from-to)497-508
Number of pages12
JournalIEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews
Issue number4
StatePublished - Nov 2001
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Software
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Learn++: An incremental learning algorithm for supervised neural networks'. Together they form a unique fingerprint.

Cite this