Comparison of ensemble techniques for incremental learning of new concept classes under hostile non-stationary environments

Hussein Syed Mohammed, James Leander, Matthew Marbach, Robi Polikar

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    3 Citations (Scopus)

    Abstract

    We have recently introduced Learn++, an incremental learning algorithm, inspired by the multiple classifiers structure of AdaBoost. Both algorithms generate an ensemble of classifiers trained on bootstrapped replicates of the training data, and the classifiers are then combined through a voting process. Learn+ however, generates additional ensembles as new data become available, and uses a different distribution update rule to resample the data. While AdaBoost was originally designed to improve the performance of a weak classifier, whether it can still achieve incremental learning through its ensemble structure is still an open question. In this paper, we compare the incremental learning ability of AdaBoost.M1 and Learn under very hostile nonstationary learning environments, which may introduce new concept classes. We also compare the algorithms under several combination rules to determine which of the three key components - ensemble structure, resampling procedure, or the combination rule - has the primary impact on incremental learning in nonstationary environments.

    Original languageEnglish (US)
    Title of host publication2006 IEEE International Conference on Systems, Man and Cybernetics
    Pages4838-4844
    Number of pages7
    Volume6
    DOIs
    StatePublished - Aug 28 2007
    Event2006 IEEE International Conference on Systems, Man and Cybernetics - Taipei, Taiwan, Province of China
    Duration: Oct 8 2006Oct 11 2006

    Other

    Other2006 IEEE International Conference on Systems, Man and Cybernetics
    CountryTaiwan, Province of China
    CityTaipei
    Period10/8/0610/11/06

    Fingerprint

    Adaptive boosting
    Classifiers
    Learning algorithms

    All Science Journal Classification (ASJC) codes

    • Engineering(all)

    Cite this

    Mohammed, H. S., Leander, J., Marbach, M., & Polikar, R. (2007). Comparison of ensemble techniques for incremental learning of new concept classes under hostile non-stationary environments. In 2006 IEEE International Conference on Systems, Man and Cybernetics (Vol. 6, pp. 4838-4844). [4274680] https://doi.org/10.1109/ICSMC.2006.385071
    Mohammed, Hussein Syed ; Leander, James ; Marbach, Matthew ; Polikar, Robi. / Comparison of ensemble techniques for incremental learning of new concept classes under hostile non-stationary environments. 2006 IEEE International Conference on Systems, Man and Cybernetics. Vol. 6 2007. pp. 4838-4844
    @inproceedings{b1f76795bde14babb330989bd29830ba,
    title = "Comparison of ensemble techniques for incremental learning of new concept classes under hostile non-stationary environments",
    abstract = "We have recently introduced Learn++, an incremental learning algorithm, inspired by the multiple classifiers structure of AdaBoost. Both algorithms generate an ensemble of classifiers trained on bootstrapped replicates of the training data, and the classifiers are then combined through a voting process. Learn+ however, generates additional ensembles as new data become available, and uses a different distribution update rule to resample the data. While AdaBoost was originally designed to improve the performance of a weak classifier, whether it can still achieve incremental learning through its ensemble structure is still an open question. In this paper, we compare the incremental learning ability of AdaBoost.M1 and Learn under very hostile nonstationary learning environments, which may introduce new concept classes. We also compare the algorithms under several combination rules to determine which of the three key components - ensemble structure, resampling procedure, or the combination rule - has the primary impact on incremental learning in nonstationary environments.",
    author = "Mohammed, {Hussein Syed} and James Leander and Matthew Marbach and Robi Polikar",
    year = "2007",
    month = "8",
    day = "28",
    doi = "10.1109/ICSMC.2006.385071",
    language = "English (US)",
    isbn = "1424401003",
    volume = "6",
    pages = "4838--4844",
    booktitle = "2006 IEEE International Conference on Systems, Man and Cybernetics",

    }

    Mohammed, HS, Leander, J, Marbach, M & Polikar, R 2007, Comparison of ensemble techniques for incremental learning of new concept classes under hostile non-stationary environments. in 2006 IEEE International Conference on Systems, Man and Cybernetics. vol. 6, 4274680, pp. 4838-4844, 2006 IEEE International Conference on Systems, Man and Cybernetics, Taipei, Taiwan, Province of China, 10/8/06. https://doi.org/10.1109/ICSMC.2006.385071

    Comparison of ensemble techniques for incremental learning of new concept classes under hostile non-stationary environments. / Mohammed, Hussein Syed; Leander, James; Marbach, Matthew; Polikar, Robi.

    2006 IEEE International Conference on Systems, Man and Cybernetics. Vol. 6 2007. p. 4838-4844 4274680.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    TY - GEN

    T1 - Comparison of ensemble techniques for incremental learning of new concept classes under hostile non-stationary environments

    AU - Mohammed, Hussein Syed

    AU - Leander, James

    AU - Marbach, Matthew

    AU - Polikar, Robi

    PY - 2007/8/28

    Y1 - 2007/8/28

    N2 - We have recently introduced Learn++, an incremental learning algorithm, inspired by the multiple classifiers structure of AdaBoost. Both algorithms generate an ensemble of classifiers trained on bootstrapped replicates of the training data, and the classifiers are then combined through a voting process. Learn+ however, generates additional ensembles as new data become available, and uses a different distribution update rule to resample the data. While AdaBoost was originally designed to improve the performance of a weak classifier, whether it can still achieve incremental learning through its ensemble structure is still an open question. In this paper, we compare the incremental learning ability of AdaBoost.M1 and Learn under very hostile nonstationary learning environments, which may introduce new concept classes. We also compare the algorithms under several combination rules to determine which of the three key components - ensemble structure, resampling procedure, or the combination rule - has the primary impact on incremental learning in nonstationary environments.

    AB - We have recently introduced Learn++, an incremental learning algorithm, inspired by the multiple classifiers structure of AdaBoost. Both algorithms generate an ensemble of classifiers trained on bootstrapped replicates of the training data, and the classifiers are then combined through a voting process. Learn+ however, generates additional ensembles as new data become available, and uses a different distribution update rule to resample the data. While AdaBoost was originally designed to improve the performance of a weak classifier, whether it can still achieve incremental learning through its ensemble structure is still an open question. In this paper, we compare the incremental learning ability of AdaBoost.M1 and Learn under very hostile nonstationary learning environments, which may introduce new concept classes. We also compare the algorithms under several combination rules to determine which of the three key components - ensemble structure, resampling procedure, or the combination rule - has the primary impact on incremental learning in nonstationary environments.

    UR - http://www.scopus.com/inward/record.url?scp=34548128463&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=34548128463&partnerID=8YFLogxK

    U2 - 10.1109/ICSMC.2006.385071

    DO - 10.1109/ICSMC.2006.385071

    M3 - Conference contribution

    AN - SCOPUS:34548128463

    SN - 1424401003

    SN - 9781424401000

    VL - 6

    SP - 4838

    EP - 4844

    BT - 2006 IEEE International Conference on Systems, Man and Cybernetics

    ER -

    Mohammed HS, Leander J, Marbach M, Polikar R. Comparison of ensemble techniques for incremental learning of new concept classes under hostile non-stationary environments. In 2006 IEEE International Conference on Systems, Man and Cybernetics. Vol. 6. 2007. p. 4838-4844. 4274680 https://doi.org/10.1109/ICSMC.2006.385071