TY - JOUR
T1 - Transformers in Time-Series Analysis
T2 - A Tutorial
AU - Ahmed, Sabeen
AU - Nielsen, Ian E.
AU - Tripathi, Aakash
AU - Siddiqui, Shamoon
AU - Ramachandran, Ravi P.
AU - Rasool, Ghulam
N1 - Publisher Copyright:
© 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.
PY - 2023/12
Y1 - 2023/12
N2 - Transformer architectures have widespread applications, particularly in Natural Language Processing and Computer Vision. Recently, Transformers have been employed in various aspects of time-series analysis. This tutorial provides an overview of the Transformer architecture, its applications, and a collection of examples from recent research in time-series analysis. We delve into an explanation of the core components of the Transformer, including the self-attention mechanism, positional encoding, multi-head, and encoder/decoder. Several enhancements to the initial Transformer architecture are highlighted to tackle time-series tasks. The tutorial also provides best practices and techniques to overcome the challenge of effectively training Transformers for time-series analysis.
AB - Transformer architectures have widespread applications, particularly in Natural Language Processing and Computer Vision. Recently, Transformers have been employed in various aspects of time-series analysis. This tutorial provides an overview of the Transformer architecture, its applications, and a collection of examples from recent research in time-series analysis. We delve into an explanation of the core components of the Transformer, including the self-attention mechanism, positional encoding, multi-head, and encoder/decoder. Several enhancements to the initial Transformer architecture are highlighted to tackle time-series tasks. The tutorial also provides best practices and techniques to overcome the challenge of effectively training Transformers for time-series analysis.
UR - http://www.scopus.com/inward/record.url?scp=85165691251&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85165691251&partnerID=8YFLogxK
U2 - 10.1007/s00034-023-02454-8
DO - 10.1007/s00034-023-02454-8
M3 - Article
AN - SCOPUS:85165691251
SN - 0278-081X
VL - 42
SP - 7433
EP - 7466
JO - Circuits, Systems, and Signal Processing
JF - Circuits, Systems, and Signal Processing
IS - 12
ER -