System and/or method for semantic parsing of air traffic control audio
AssigneeMerlin Labs, Inc.
InventorsMichael Pust, Joseph Bondaryk, Matthew George
Life of this patent
Estimating the term…
Abstract
The method S200 can include: at an aircraft, receiving an audio utterance from air traffic control S210, converting the audio utterance to text, determining commands from the text using a question-and-answer model S240, and optionally controlling the aircraft based on the commands S250. The method functions to automatically interpret flight commands from the air traffic control (ATC) stream.
Claims14
A system for an aircraft for semantic parsing of air traffic control (ATC) utterances, the system comprising:
- a communication system onboard the aircraft;
- and
- a computing system communicatively coupled to the communication system and configured to receive an ATC audio signal from the communication system, the computing system comprising:
- a speech-to-text module configured to determine an utterance hypothesis from the ATC audio signal, the utterance hypothesis comprising a speaker identification;
- and
- a question-and-answer (Q/A) module configured to determine aircraft commands by querying the utterance hypothesis with a pre-trained neural network model according to a sequence of natural language queries, wherein the sequence comprises a tree-based sequence with a plurality of dependencies linking one or more natural language queries to a determination, based on the speaker identification, that the aircraft is an intended recipient of an utterance corresponding to the utterance hypothesis.
A system for an aircraft for semantic parsing of air traffic control (ATC) utterances, the system comprising:
- a communication system onboard the aircraft;
- a computing system communicatively coupled to the communication system and configured to receive an ATC audio signal from the communication system, the computing system comprising:
- a speech-to-text module configured to determine an utterance hypothesis from the ATC audio signal comprising a first pre-trained neural network, wherein the first pre-trained neural network is pre-trained to identify a transition speaker associated with audio artifacts;
- and
- a question-and-answer (Q/A) module configured to determine aircraft commands based on the utterance hypothesis using a plurality of natural language queries.
A method for semantic parsing of air traffic control (ATC) utterances for an aircraft, the method comprising:
- receiving an ATC audio signal;
- based on an ATC-tuned language model, determining an utterance hypothesis from the ATC audio signal, comprising:
- with an integrated automatic speech recognition (ASR) and sentence boundary detection (SBD) module, generating a set of linguistic hypotheses based on the ATC audio signal;
- using the ATC-tuned language model, determining a respective language score for each linguistic hypothesis of the set of linguistic hypotheses;
- and
- determining the utterance hypothesis from the set of the linguistic hypotheses based on the respective language scores;
- querying the utterance hypothesis with a pre-trained neural network model according to a sequence of the natural language queries;
- and
- based on the sequence of natural language queries, determining a set of aircraft commands associated with the utterance hypothesis.
Dependencies are inferred from the claim text.