# US 2024/0201696 A1 · Autonomous detect and avoid from speech recognition and analysis > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0201696 A1 - Document ID: US20240201696A1 - Status: Published application - Kind: A1 - Application number: 18/066,220 - Filed: 2022-12-14 - Published: 2024-06-20 - Earliest claimed: 2022-12-14 - Assignees: Wing Aviation LLC - Applicants: Wing Aviation LLC - Inventors: John Mooney; Ali Shoeb - Web page: https://patentgrep.com/patent/US20240201696A1 ## CPC - G05D 1/1064: {specially adapted for avoiding collisions with other aircraft} - B64C 39/024: {of the remote controlled vehicle type, i.e. RPV} - G05D 1/104: {involving a plurality of aircrafts, e.g. formation flying (traffic control systems for aircraft G08G5/00)} - G05D 1/633: Dynamic obstacles - G08G 5/21: located onboard the aircraft - G08G 5/53: for cruising - G08G 5/54: for approach or landing - G08G 5/55: for a single aircraft - G08G 5/57: for unmanned aircraft - G08G 5/723: {from the aircraft} - G08G 5/74: for monitoring terrain - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - B64U 2101/64: for parcel delivery or retrieval - B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] - G05D 2105/285: {postal packages} - G05D 2107/17: Spaces with priority for humans, e.g. populated areas, pedestrian ways, parks or beaches - G05D 2109/24: {Convertible aircraft, e.g. tiltrotor aircraft} - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - B64C: AEROPLANES; HELICOPTERS - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR ## Abstract A technique for detecting and avoiding obstacles by an unmanned aerial vehicle (UAV) includes: querying a knowledge graph having information related to a dynamic obstacle that may be in proximity to the UAV when traveling along a planned route; comparing the location of the dynamic obstacle to the UAV to detect conflicts; and in response to detecting a conflict, performing an action to avoid conflict with the dynamic obstacle. The knowledge graph can be updated by receiving a VHF radio signal containing the information related to the dynamic obstacle in the audible speech format; translating the audible speech format to a text format using speech recognition; analyzing the text format for relevant information related to the dynamic obstacle; comparing the relevant information related to the dynamic obstacle of the text format to the knowledge graph to detect changes; and updating the knowledge graph. ## Claims ### Claim 1 A non-transitory computer-readable medium having logic stored thereon that, in response to execution by one or more processors of an unmanned aerial vehicle (UAV), causes the UAV to perform actions comprising: traveling along a planned route of the UAV; querying a knowledge graph having information related to a dynamic obstacle that may be in proximity to the UAV when traveling along the planned route, wherein the knowledge graph includes information related to the dynamic obstacle received as a wireless radio signal in an audible speech format; comparing a location of the dynamic obstacle to the UAV when traveling along the planned route to detect conflicts; and in response to detecting a conflict between the dynamic obstacle and the UAV when traveling along the planned route, performing an action by the UAV to avoid the conflict with the dynamic obstacle. ### Claim 2 (depends on 1) The non-transitory computer-readable medium of claim 1, further comprising updating the knowledge graph by: receiving, by a wireless radio interface of the UAV, the wireless radio signal containing the information related to the dynamic obstacle in the audible speech format; translating the audible speech format to a text format using speech recognition; analyzing the text format for relevant information related to the dynamic obstacle; comparing the relevant information related to the dynamic obstacle received from the text format to the information related to the dynamic obstacle within the knowledge graph to detect changes; and in response to detecting changes, updating the knowledge graph. ### Claim 3 (depends on 2) The non-transitory computer-readable medium of claim 2, wherein the wireless radio interface is a VHF radio interface, and wherein the wireless radio signal is a VHF radio signal. ### Claim 4 (depends on 2) The non-transitory computer-readable medium of claim 2, further comprising: in response to updating the knowledge graph, transmitting a notification to the UAV to perform actions comprising: querying the updated knowledge graph; comparing the location of the dynamic obstacle to the UAV when traveling along the planned route to detect conflicts; and in response to detecting a conflict between the dynamic obstacle and the UAV when traveling along the planned route, performing an action by the UAV to avoid the conflict with the dynamic obstacle. ### Claim 5 (depends on 2) The non-transitory computer-readable medium of claim 2, wherein updating the knowledge graph is performed autonomously by the UAV. ### Claim 6 (depends on 5) The non-transitory computer-readable medium of claim 5, wherein the updated knowledge graph is transmitted to a fleet management system. ### Claim 7 (depends on 2) The non-transitory computer-readable medium of claim 2, wherein updating the knowledge graph is performed by a fleet management system located remote from the UAV, and wherein the fleet management system transmits at least a portion of the updated knowledge graph to the UAV. ### Claim 8 (depends on 2) The non-transitory computer-readable medium of claim 2, wherein the relevant information related to the dynamic obstacle includes one or more of craft identification information, heading information, airspeed information, flightpath information, destination information, and pilot intent information. ### Claim 9 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the knowledge graph comprises information related to the dynamic obstacle received by the wireless radio signal and one or more of an ADS-B message and a radar signal. ### Claim 10 (depends on 2) The non-transitory computer-readable medium of claim 2, wherein the knowledge graph comprises information related to the dynamic obstacle received by an ADS-B message, and wherein the wireless radio signal includes different or additional information related to the dynamic obstacle. ### Claim 11 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the action of querying the knowledge graph is performed at a frequency selected from a group consisting of one cycle every ten seconds, one cycle every five seconds, one cycle per second, four cycles per second, ten cycles per second, and 50 cycles per second. ### Claim 12 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the action by the UAV to avoid conflict with the dynamic obstacle comprises: determining a safe landing location; and descending to land to deconflict with the dynamic obstacle. ### Claim 13 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the action by the UAV to avoid conflict with the dynamic obstacle comprises: adjusting the planned route to an adjusted planned route that deconflicts with the dynamic obstacle. ### Claim 14 A method for detecting and avoiding a dynamic obstacle by an autonomous vehicle, the method comprising: controlling propulsion devices of the autonomous vehicle along a planned route; querying a knowledge graph having information related to the dynamic obstacle that may be in proximity to the autonomous vehicle when traveling along the planned route, wherein the knowledge graph includes information related to the dynamic obstacle received by a radio interface in an audible speech format; comparing a location of the dynamic obstacle to the autonomous vehicle when traveling along the planned route to detect conflicts; and in response to detecting a conflict between the dynamic obstacle and the autonomous vehicle when traveling along the planned route, controlling the propulsion devices to avoid conflict with the dynamic obstacle in response to receiving the notification of the conflict. ### Claim 15 (depends on 14) The method of claim 14, further comprising the step of updating the knowledge graph by: receiving a wireless radio signal containing the information related to the dynamic obstacle in the audible speech format; translating the audible speech format to a text format; analyzing the text format for relevant information related to the dynamic obstacle; comparing the relevant information related to the dynamic obstacle received from the text format to the information related to the dynamic obstacle within the knowledge graph to detect differences; and in response to detecting differences, updating the knowledge graph with the differences in the information related to the dynamic obstacle received from the text format. ### Claim 16 (depends on 15) The method of claim 15, wherein the wireless radio signal is a VHF radio signal. ### Claim 17 (depends on 15) The method of claim 15, wherein updating the knowledge graph further comprises transmitting a notification to the autonomous vehicle to perform steps comprising: querying the updated knowledge graph; comparing the location of the dynamic obstacle to the autonomous vehicle when traveling along the planned route to detect conflicts; and in response to detecting a conflict between the dynamic obstacle and the autonomous vehicle when traveling along the planned route, controlling the propulsion devices to avoid conflict with the dynamic obstacle in response to receiving the notification of the conflict. ### Claim 18 (depends on 15) The method of claim 15, wherein updating the knowledge graph is performed autonomously by the autonomous vehicle. ### Claim 19 (depends on 15) The method of claim 15, wherein the updated knowledge graph is transmitted to a fleet management system. ### Claim 20 (depends on 15) The method of claim 15, wherein updating the knowledge graph is performed by a fleet management system located remote from the autonomous vehicle, and wherein the fleet management system transmits at least a portion of the updated knowledge graph to the autonomous vehicle. ### Claim 21 (depends on 15) The method of claim 15, wherein the relevant information related to the dynamic obstacle includes one or more of craft identification information, heading information, airspeed information, flightpath information, destination information, and pilot intent information. ### Claim 22 (depends on 15) The method of claim 15, wherein the knowledge graph comprises information related to the dynamic obstacle received by an ADS-B message, and wherein the wireless radio signal includes different or additional information related to the dynamic obstacle. ### Claim 23 (depends on 14) The method of claim 14, wherein controlling the propulsion devices to avoid conflict with the dynamic obstacle comprises: determining a safe landing location; and descending to land to deconflict with the dynamic obstacle. ### Claim 24 (depends on 14) The method of claim 14, wherein controlling the propulsion devices to avoid conflict with the dynamic obstacle comprises: adjusting the planned route to an adjusted planned route that deconflicts with the dynamic obstacle. ### Claim 25 (depends on 14) The method of claim 14, wherein the knowledge graph is an on-board knowledge graph stored in a knowledge graph data store on the autonomous vehicle, and wherein the on-board knowledge graph is a subset of an expanded knowledge graph of a fleet management system.