Explainable AI: Rotorcraft attitude prediction

Rotorcrafts are generally subject to a higher fatal accident rate than other segments of aviation, including commercial and general aviation. The safety improvement for rotorcrafts would directly improve the efficiency of air traffic control, since rotorcrafts operate primarily within low-level airspace; an area that is becoming increasingly complex with new entrants, such as unmanned aircraft systems and urban air mobility. The recent impact of artificial intelligence and deep learning algorithms on various aspects of our lives has led to the investigation of the application of these algorithms in the aviation domain; as it may offer a prime opportunity to enhance safety within the aviation community. In this research, we explore the efficacy, reliability, and, more importantly, the explainability of modern deep learning algorithms. We use machine learning models to predict the attitude (pitch and yaw) of rotorcrafts using video data recorded with ordinary cameras. The cameras were mounted inside the helicopter cockpit and recorded outside view through windshield continually during the flight. We train four different architectures of convolutional neural networks (CNNs), i.e., VGG16, VGG19, ResNet50, and Xception. The models achieved 90%, 91%, 88%, and 88%, respectively, average attitude prediction accuracy on the test video dataset. Furthermore, we use gradient class activation maps (grad-CAM) to ascertain the features and regions of the image that influenced the model to make a specific prediction. We show that CNNs learn to focus on similar features as human operators (pilots), i.e., the natural horizon curve. Our findings demonstrate the feasibility of using deep learning models for attitude prediction from flight videos recorded using ordinary inexpensive cameras. The proposed video analytics framework provides a cost-effective means to supplement traditional Flight Data Recorders (FDR); a technology that is often beyond the financial reach of most general aviation rotorcraft operators.