TY - JOUR
T1 - Incremental learning of concept drift in nonstationary environments
AU - Elwell, Ryan
AU - Polikar, Robi
N1 - Funding Information:
Manuscript received July 31, 2010; revised January 7, 2011, and April 6, 2011; accepted June 4, 2011. Date of publication August 4, 2011; date of current version October 5, 2011. This work was supported by the National Science Foundation under Grant ECCS 0926159.
PY - 2011/10
Y1 - 2011/10
N2 - We introduce an ensemble of classifiers-based approach for incremental learning of concept drift, characterized by nonstationary environments (NSEs), where the underlying data distributions change over time. The proposed algorithm, named Learn++.NSE, learns from consecutive batches of data without making any assumptions on the nature or rate of drift; it can learn from such environments that experience constant or variable rate of drift, addition or deletion of concept classes, as well as cyclical drift. The algorithm learns incrementally, as other members of the Learn++ family of algorithms, that is, without requiring access to previously seen data. Learn++.NSE trains one new classifier for each batch of data it receives, and combines these classifiers using a dynamically weighted majority voting. The novelty of the approach is in determining the voting weights, based on each classifier's time-adjusted accuracy on current and past environments. This approach allows the algorithm to recognize, and act accordingly, to the changes in underlying data distributions, as well as to a possible reoccurrence of an earlier distribution. We evaluate the algorithm on several synthetic datasets designed to simulate a variety of nonstationary environments, as well as a real-world weather prediction dataset. Comparisons with several other approaches are also included. Results indicate that Learn++.NSE can track the changing environments very closely, regardless of the type of concept drift. To allow future use, comparison and benchmarking by interested researchers, we also release our data used in this paper.
AB - We introduce an ensemble of classifiers-based approach for incremental learning of concept drift, characterized by nonstationary environments (NSEs), where the underlying data distributions change over time. The proposed algorithm, named Learn++.NSE, learns from consecutive batches of data without making any assumptions on the nature or rate of drift; it can learn from such environments that experience constant or variable rate of drift, addition or deletion of concept classes, as well as cyclical drift. The algorithm learns incrementally, as other members of the Learn++ family of algorithms, that is, without requiring access to previously seen data. Learn++.NSE trains one new classifier for each batch of data it receives, and combines these classifiers using a dynamically weighted majority voting. The novelty of the approach is in determining the voting weights, based on each classifier's time-adjusted accuracy on current and past environments. This approach allows the algorithm to recognize, and act accordingly, to the changes in underlying data distributions, as well as to a possible reoccurrence of an earlier distribution. We evaluate the algorithm on several synthetic datasets designed to simulate a variety of nonstationary environments, as well as a real-world weather prediction dataset. Comparisons with several other approaches are also included. Results indicate that Learn++.NSE can track the changing environments very closely, regardless of the type of concept drift. To allow future use, comparison and benchmarking by interested researchers, we also release our data used in this paper.
UR - http://www.scopus.com/inward/record.url?scp=80053634784&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=80053634784&partnerID=8YFLogxK
U2 - 10.1109/TNN.2011.2160459
DO - 10.1109/TNN.2011.2160459
M3 - Review article
C2 - 21824845
AN - SCOPUS:80053634784
VL - 22
SP - 1517
EP - 1531
JO - IEEE Transactions on Neural Networks
JF - IEEE Transactions on Neural Networks
SN - 2162-237X
IS - 10
M1 - 5975223
ER -