# US 12,190,861 B2 · Adaptive speech recognition methods and systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,190,861 B2 - Document ID: US12190861B2 - Status: Granted - Kind: B2 - Application number: 17/659,596 - Filed: 2022-04-18 - Published: 2022-10-27 - Granted: 2025-01-07 - Earliest claimed: 2021-04-22 - Assignees: Honeywell International Inc. - Applicants: Honeywell International Inc. - Inventors: Jitender Kumar Agarwal; Chaya Garg; Vasantha Paulraj; Ramakrishnan Raman; Mahesh Kumar Sampath; Mohan M Thippeswamy - Web page: https://patentgrep.com/patent/US12190861B2 ## CPC - G10L 15/063: {Training} - B64D 11/0015: {Arrangements for entertainment or communications, e.g. radio, television} - G08G 5/21: located onboard the aircraft - G08G 5/26: Transmission of traffic-related information between aircraft and ground stations - G08G 5/53: for cruising - G10L 15/01: Assessment or evaluation of speech recognition systems - G10L 15/183: using context dependencies, e.g. language models - G10L 15/19: Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - B64D 43/00: Arrangements or adaptations of instruments - B64D 47/00: Equipment not otherwise provided for - G08G 5/55: for a single aircraft - G10L 2015/0635: {updating or merging of old and new templates; Mean values; Weighting} - G10L 2015/223: {Execution procedure of a spoken command} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract Methods and systems are provided for assisting operation of a vehicle using speech recognition. One method involves analyzing a transcription of an audio communication with respect to the vehicle to characterize a nonstandard pattern within the transcription of the audio communication, obtaining a ground truth for the transcription of the audio communication, determining one or more performance metrics associated with the nonstandard pattern within the transcription based on a relationship between the transcription of the audio communication and the ground truth for the transcription, updating a speech recognition vocabulary for the vehicle to include the nonstandard pattern based at least in part on the one or more performance metrics and determining an updated speech recognition model for the vehicle using the updated speech recognition vocabulary and the audio communication. ## Claims ### Claim 1 A method of assisting operation of a vehicle, the method comprising: analyzing a transcription of an audio communication with respect to the vehicle to characterize a nonstandard pattern within the transcription of the audio communication; obtaining a ground truth for the transcription of the audio communication; determining one or more performance metrics associated with the nonstandard pattern within the transcription based on a relationship between the transcription of the audio communication and the ground truth for the transcription; updating a speech recognition vocabulary for the vehicle to include the nonstandard pattern based at least in part on the one or more performance metrics, resulting in an updated speech recognition vocabulary; and determining an updated speech recognition model for the vehicle using the updated speech recognition vocabulary and the audio communication. ### Claim 2 (depends on 1) The method of claim 1, further comprising pushing the updated speech recognition model to the vehicle over a communications network. ### Claim 3 (depends on 2) The method of claim 2, further comprising analyzing, at the vehicle, a second transcription of a subsequent audio communication with respect to the vehicle to detect the nonstandard pattern within the second transcription of the subsequent audio communication using at least one of the updated speech recognition vocabulary and the updated speech recognition model. ### Claim 4 (depends on 1) The method of claim 1, further comprising analyzing a second transcription of a subsequent audio communication with respect to the vehicle to detect the nonstandard pattern within the second transcription of the subsequent audio communication using at least one of the updated speech recognition vocabulary and the updated speech recognition model. ### Claim 5 (depends on 1) The method of claim 1, wherein analyzing the transcription of the audio communication with respect to the vehicle to characterize the nonstandard pattern within the transcription of the audio communication comprises: determining at least one of a phraseology pattern subject category and a phraseology pattern structure type associated with the transcription of the audio communication based at least in part on content of the transcription of the audio communication; determining whether the content of the transcription of the audio communication corresponds to one of a plurality of standard phraseology patterns based at least in part on the at least one of the phraseology pattern subject category and the phraseology pattern structure type; and determining the transcription of the audio communication comprises the nonstandard pattern when the transcription of the audio communication does not correspond to any of the plurality of standard phraseology patterns. ### Claim 6 (depends on 5) The method of claim 5, further comprising assigning a pattern identifier associated with an existing nonstandard pattern to the transcription of the audio communication when the content of the transcription of the audio communication corresponds to the existing nonstandard pattern based at least in part on the at least one of the phraseology pattern subject category and the phraseology pattern structure type. ### Claim 7 (depends on 1) The method of claim 1, wherein: analyzing the transcription of the audio communication with respect to the vehicle to characterize the nonstandard pattern comprises identifying a phraseology pattern portion associated with the nonstandard pattern within the transcription of the audio communication; and determining the one or more performance metrics associated with the nonstandard pattern within the transcription comprises determining a phraseology pattern performance metric based on a relationship between the phraseology pattern portion of the transcription of the audio communication and a second phraseology pattern portion of the ground truth. ### Claim 8 (depends on 1) The method of claim 1, wherein: determining the one or more performance metrics comprises determining a pattern-based performance metric associated with the nonstandard pattern within the transcription based on a relationship between a phraseology pattern portion of the transcription of the audio communication and the phraseology pattern portion of the ground truth for the transcription; and updating the speech recognition vocabulary comprises updating the speech recognition vocabulary based on the pattern-based performance metric. ### Claim 9 A non-transitory computer-readable medium having computer-executable instructions stored thereon that, when executed by a processing system, cause the processing system to: analyze a transcription of an audio communication with respect to a vehicle to characterize a nonstandard pattern within the transcription of the audio communication; obtain a ground truth for the transcription of the audio communication; determine one or more performance metrics associated with the nonstandard pattern within the transcription based on a relationship between the transcription of the audio communication and the ground truth for the transcription; update a speech recognition vocabulary to include the nonstandard pattern based at least in part on the one or more performance metrics, resulting in an updated speech recognition vocabulary; and determine an updated speech recognition model for the vehicle using the updated speech recognition vocabulary and the audio communication. ### Claim 10 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the computer-executable instructions cause the processing system to push the updated speech recognition model to the vehicle over a communications network. ### Claim 11 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the computer-executable instructions cause the processing system to analyze a second transcription of a subsequent audio communication with respect to the vehicle to detect the nonstandard pattern within the second transcription of the subsequent audio communication using at least one of the updated speech recognition vocabulary and the updated speech recognition model. ### Claim 12 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the computer-executable instructions cause the processing system to analyze the transcription of the audio communication with respect to the vehicle to characterize the nonstandard pattern within the transcription of the audio communication by: determining at least one of a phraseology pattern subject category and a phraseology pattern structure type associated with the transcription of the audio communication based at least in part on content of the transcription of the audio communication; determining whether the content of the transcription of the audio communication corresponds to one of a plurality of standard phraseology patterns based at least in part on the at least one of the phraseology pattern subject category and the phraseology pattern structure type; and determining the transcription of the audio communication comprises the nonstandard pattern when the transcription of the audio communication does not correspond to any of the plurality of standard phraseology patterns. ### Claim 13 (depends on 12) The non-transitory computer-readable medium of claim 12, wherein the computer-executable instructions cause the processing system to assign a pattern identifier associated with an existing nonstandard pattern to the transcription of the audio communication when the content of the transcription of the audio communication corresponds to the existing nonstandard pattern based at least in part on the at least one of the phraseology pattern subject category and the phraseology pattern structure type. ### Claim 14 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein: analyzing the transcription of the audio communication with respect to the vehicle to characterize the nonstandard pattern comprises identifying a phraseology pattern portion associated with the nonstandard pattern within the transcription of the audio communication; and determining the one or more performance metrics associated with the nonstandard pattern within the transcription comprises determining a phraseology pattern performance metric based on a relationship between the phraseology pattern portion of the transcription of the audio communication and a second phraseology pattern portion of the ground truth. ### Claim 15 (depends on 9) The non-transitory computer-readable medium of claim 9, wherein the computer-executable instructions cause the processing system to: determine a pattern-based performance metric associated with the nonstandard pattern within the transcription based on a relationship between a phraseology pattern portion of the transcription of the audio communication and the phraseology pattern portion of the ground truth for the transcription; and update the speech recognition vocabulary to include the nonstandard pattern based on the pattern-based performance metric. ### Claim 16 A computing device comprising: at least one computer-readable storage medium to store computer-executable instructions; and at least one processor, coupled to the at least one computer-readable storage medium, to execute the computer-executable instructions to: analyze a transcription of an audio communication with respect to a vehicle to characterize a nonstandard pattern within the transcription of the audio communication; obtain a ground truth for the transcription of the audio communication; determine one or more performance metrics associated with the nonstandard pattern within the transcription based on a relationship between the transcription of the audio communication and the ground truth for the transcription; update a speech recognition vocabulary to include the nonstandard pattern based at least in part on the one or more performance metrics, resulting in an updated speech recognition vocabulary; and determine an updated speech recognition model for the vehicle using the updated speech recognition vocabulary and the audio communication. ### Claim 17 (depends on 16) The computing device of claim 16, wherein the computer-executable instructions cause the at least one processor to push the updated speech recognition model to the vehicle over a communications network. ### Claim 18 (depends on 16) The computing device of claim 16, wherein the computer-executable instructions cause the at least one processor to analyze a second transcription of a subsequent audio communication with respect to the vehicle to detect the nonstandard pattern within the second transcription of the subsequent audio communication using at least one of the updated speech recognition vocabulary and the updated speech recognition model. ### Claim 19 (depends on 16) The computing device of claim 16, wherein the computer-executable instructions cause the at least one processor to: identify a phraseology pattern portion associated with the nonstandard pattern within the transcription of the audio communication; and determine a phraseology pattern performance metric based on a relationship between the phraseology pattern portion of the transcription of the audio communication and a second phraseology pattern portion of the ground truth. ### Claim 20 (depends on 16) The computing device of claim 16, wherein the computer-executable instructions cause the at least one processor to: determine a pattern-based performance metric associated with the nonstandard pattern within the transcription based on a relationship between a phraseology pattern portion of the transcription of the audio communication and the phraseology pattern portion of the ground truth for the transcription; and update the speech recognition vocabulary based on the pattern-based performance metric.