Controller-Pilot Voice Communication and Intent Monitoring for Future Aviation Systems Safety

This effort combines speech analytics, Speed to Text (STT) translation software, and intent inference algorithms in order to assess aviation system safety. Each of these components has a different perspective on spotting potential off-nominal and anomalous conditions, for instance, in the way people speak under stress and high workload, the mis-communications and mishearing of key words in a dialog, or the intent to follow a command or not follow a command. The system being designed and built analyzes pilot-controller conversations in real-time, identifies key speech features, STT translations, and intent models to identify if the situation is nominal, off-nominal, or anomalous. This innovation combines speech analytics, speech to text translations, and intent inference to continuously monitor the pilot-controller dialog for potential off-nominal and anomalous situations. These system components are designed to complement each other, and make up for any weaknesses within independent components. Based on comparisons to historical data that indicate mis-communications, off-nominal (out of conformance), and anomalous (accidents, incidents, other), current state data is compared to historical data to determine if the observed situation is potentially in a safety critical situation. Build a demonstration system that combines real-time speech analytics, speech to text translations, and intent inference Demonstrate how the system can analyze real-time audio from LiveATC, ADS-B surveillance data, and weather data to identify nominal, off-nominal, and anomaly conditions A proof of concept demonstration system will be delivered at the completion of Phase II
Benefits: This effort directly relates to NASA’s ISSA and IASMS: In-time System-wide Safety Assurance (ISSA) – Proactively mitigate risks and demonstrate innovative solutions while ultimately ensuring safety to the community on the ground and in the NAS. In-Time Aviation Safety Management System (IASMS) – A scalable and distributed system approach with a service-oriented architecture to address aviation safety needs in the NAS. FAA Air Traffic Control (ATC) system monitoring Airline pilot-controller monitoring Airline pilot-dispatcher monitoring Additional commercial applications include stock trading voice communications workflow monitoring and regulatory compliance checking. The appropriate prime contractors are typically deeply involved with both voice and data communications.