A Martingale-based approach for flight behavior anomaly detection

Shen Shyang Ho, Matthew Schofield, Bo Sun, Jason Snouffer, Jean Kirschner

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The timely detection of anomalous flight behavior is critical to ensure a prompt and appropriate response to mitigate any dangers to flight safety or hindrance of logistics operations. Most previous approaches focused on anomaly detection, leading them to only be able to raise an alert after an occurrence of an anomaly. A more effective approach is to predict a potential anomaly based on current observations, thus cutting down on detection time and allowing for a more expedient response. We propose a novel martingale-based approach to predict anomalous flight behavior in the near future as data points are observed one by one in real-Time. The proposed anomaly prediction method consists of two components: (i) utilization of regression to model the historical full flight behavior and (ii) monitoring of the real-Time flight behavior using a martingale (stochastic) process. The latter component consists of two prediction steps: (i) first to predict future values of multiple target variables (e.g., latitude, longitude, and altitude) using regression models, and (ii) then to decide whether the predicted values exhibit anomalies. In particular, our proposed method uses martingale tests on multiple Gaussian process regression (GPR) predictive models of target variables. The main advantages of the proposed method are: (i) the use of multiple martingale tests allows one to have a tighter false positive bound for anomaly detection/prediction, and (ii) the prediction steps reduce the delay time for anomaly detection. Experimental results on real-world data show that the performance (mean delay time, recall, and precision) of our proposed approach is competitive against other compared methods.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 20th International Conference on Mobile Data Management, MDM 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages43-52
Number of pages10
ISBN (Electronic)9781728133638
DOIs
StatePublished - Jun 1 2019
Event20th International Conference on Mobile Data Management, MDM 2019 - Hong Kong, Hong Kong
Duration: Jun 10 2019Jun 13 2019

Publication series

NameProceedings - IEEE International Conference on Mobile Data Management
Volume2019-June
ISSN (Print)1551-6245

Conference

Conference20th International Conference on Mobile Data Management, MDM 2019
CountryHong Kong
CityHong Kong
Period6/10/196/13/19

Fingerprint

Time delay
Random processes
Logistics
Monitoring

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Cite this

Ho, S. S., Schofield, M., Sun, B., Snouffer, J., & Kirschner, J. (2019). A Martingale-based approach for flight behavior anomaly detection. In Proceedings - 2019 20th International Conference on Mobile Data Management, MDM 2019 (pp. 43-52). [8788752] (Proceedings - IEEE International Conference on Mobile Data Management; Vol. 2019-June). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MDM.2019.00-75
Ho, Shen Shyang ; Schofield, Matthew ; Sun, Bo ; Snouffer, Jason ; Kirschner, Jean. / A Martingale-based approach for flight behavior anomaly detection. Proceedings - 2019 20th International Conference on Mobile Data Management, MDM 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 43-52 (Proceedings - IEEE International Conference on Mobile Data Management).
@inproceedings{99fe45a6062349b8baef11e60a1e7b67,
title = "A Martingale-based approach for flight behavior anomaly detection",
abstract = "The timely detection of anomalous flight behavior is critical to ensure a prompt and appropriate response to mitigate any dangers to flight safety or hindrance of logistics operations. Most previous approaches focused on anomaly detection, leading them to only be able to raise an alert after an occurrence of an anomaly. A more effective approach is to predict a potential anomaly based on current observations, thus cutting down on detection time and allowing for a more expedient response. We propose a novel martingale-based approach to predict anomalous flight behavior in the near future as data points are observed one by one in real-Time. The proposed anomaly prediction method consists of two components: (i) utilization of regression to model the historical full flight behavior and (ii) monitoring of the real-Time flight behavior using a martingale (stochastic) process. The latter component consists of two prediction steps: (i) first to predict future values of multiple target variables (e.g., latitude, longitude, and altitude) using regression models, and (ii) then to decide whether the predicted values exhibit anomalies. In particular, our proposed method uses martingale tests on multiple Gaussian process regression (GPR) predictive models of target variables. The main advantages of the proposed method are: (i) the use of multiple martingale tests allows one to have a tighter false positive bound for anomaly detection/prediction, and (ii) the prediction steps reduce the delay time for anomaly detection. Experimental results on real-world data show that the performance (mean delay time, recall, and precision) of our proposed approach is competitive against other compared methods.",
author = "Ho, {Shen Shyang} and Matthew Schofield and Bo Sun and Jason Snouffer and Jean Kirschner",
year = "2019",
month = "6",
day = "1",
doi = "10.1109/MDM.2019.00-75",
language = "English (US)",
series = "Proceedings - IEEE International Conference on Mobile Data Management",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "43--52",
booktitle = "Proceedings - 2019 20th International Conference on Mobile Data Management, MDM 2019",
address = "United States",

}

Ho, SS, Schofield, M, Sun, B, Snouffer, J & Kirschner, J 2019, A Martingale-based approach for flight behavior anomaly detection. in Proceedings - 2019 20th International Conference on Mobile Data Management, MDM 2019., 8788752, Proceedings - IEEE International Conference on Mobile Data Management, vol. 2019-June, Institute of Electrical and Electronics Engineers Inc., pp. 43-52, 20th International Conference on Mobile Data Management, MDM 2019, Hong Kong, Hong Kong, 6/10/19. https://doi.org/10.1109/MDM.2019.00-75

A Martingale-based approach for flight behavior anomaly detection. / Ho, Shen Shyang; Schofield, Matthew; Sun, Bo; Snouffer, Jason; Kirschner, Jean.

Proceedings - 2019 20th International Conference on Mobile Data Management, MDM 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 43-52 8788752 (Proceedings - IEEE International Conference on Mobile Data Management; Vol. 2019-June).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A Martingale-based approach for flight behavior anomaly detection

AU - Ho, Shen Shyang

AU - Schofield, Matthew

AU - Sun, Bo

AU - Snouffer, Jason

AU - Kirschner, Jean

PY - 2019/6/1

Y1 - 2019/6/1

N2 - The timely detection of anomalous flight behavior is critical to ensure a prompt and appropriate response to mitigate any dangers to flight safety or hindrance of logistics operations. Most previous approaches focused on anomaly detection, leading them to only be able to raise an alert after an occurrence of an anomaly. A more effective approach is to predict a potential anomaly based on current observations, thus cutting down on detection time and allowing for a more expedient response. We propose a novel martingale-based approach to predict anomalous flight behavior in the near future as data points are observed one by one in real-Time. The proposed anomaly prediction method consists of two components: (i) utilization of regression to model the historical full flight behavior and (ii) monitoring of the real-Time flight behavior using a martingale (stochastic) process. The latter component consists of two prediction steps: (i) first to predict future values of multiple target variables (e.g., latitude, longitude, and altitude) using regression models, and (ii) then to decide whether the predicted values exhibit anomalies. In particular, our proposed method uses martingale tests on multiple Gaussian process regression (GPR) predictive models of target variables. The main advantages of the proposed method are: (i) the use of multiple martingale tests allows one to have a tighter false positive bound for anomaly detection/prediction, and (ii) the prediction steps reduce the delay time for anomaly detection. Experimental results on real-world data show that the performance (mean delay time, recall, and precision) of our proposed approach is competitive against other compared methods.

AB - The timely detection of anomalous flight behavior is critical to ensure a prompt and appropriate response to mitigate any dangers to flight safety or hindrance of logistics operations. Most previous approaches focused on anomaly detection, leading them to only be able to raise an alert after an occurrence of an anomaly. A more effective approach is to predict a potential anomaly based on current observations, thus cutting down on detection time and allowing for a more expedient response. We propose a novel martingale-based approach to predict anomalous flight behavior in the near future as data points are observed one by one in real-Time. The proposed anomaly prediction method consists of two components: (i) utilization of regression to model the historical full flight behavior and (ii) monitoring of the real-Time flight behavior using a martingale (stochastic) process. The latter component consists of two prediction steps: (i) first to predict future values of multiple target variables (e.g., latitude, longitude, and altitude) using regression models, and (ii) then to decide whether the predicted values exhibit anomalies. In particular, our proposed method uses martingale tests on multiple Gaussian process regression (GPR) predictive models of target variables. The main advantages of the proposed method are: (i) the use of multiple martingale tests allows one to have a tighter false positive bound for anomaly detection/prediction, and (ii) the prediction steps reduce the delay time for anomaly detection. Experimental results on real-world data show that the performance (mean delay time, recall, and precision) of our proposed approach is competitive against other compared methods.

UR - http://www.scopus.com/inward/record.url?scp=85071019801&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85071019801&partnerID=8YFLogxK

U2 - 10.1109/MDM.2019.00-75

DO - 10.1109/MDM.2019.00-75

M3 - Conference contribution

AN - SCOPUS:85071019801

T3 - Proceedings - IEEE International Conference on Mobile Data Management

SP - 43

EP - 52

BT - Proceedings - 2019 20th International Conference on Mobile Data Management, MDM 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Ho SS, Schofield M, Sun B, Snouffer J, Kirschner J. A Martingale-based approach for flight behavior anomaly detection. In Proceedings - 2019 20th International Conference on Mobile Data Management, MDM 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 43-52. 8788752. (Proceedings - IEEE International Conference on Mobile Data Management). https://doi.org/10.1109/MDM.2019.00-75