# US 10,120,377 B2 · Multiple unmanned aerial vehicle autonomous coordination > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,120,377 B2 - Document ID: US10120377B2 - Status: Granted - Kind: B2 - Application number: 15/236,409 - Filed: 2016-08-13 - Published: 2018-02-15 - Granted: 2018-11-06 - Earliest claimed: 2016-08-13 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Jui-Hsin Lai; Yu Ma; Conglei Shi; Yinglong Xia - Web page: https://patentgrep.com/patent/US10120377B2 ## CPC - G05D 1/0027: {involving a plurality of vehicles, e.g. fleet or convoy travelling (fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00)} - G05D 1/104: {involving a plurality of aircrafts, e.g. formation flying (traffic control systems for aircraft G08G5/00)} - G08G 5/26: Transmission of traffic-related information between aircraft and ground stations - G08G 5/34: for flight plan modification - G08G 5/55: for a single aircraft - G08G 5/56: for two or more aircraft - G08G 5/57: for unmanned aircraft - G08G 5/80: Anti-collision systems - H04W 4/023: {using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds} - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract At each of a plurality of unmanned aerial vehicles operating in a three-dimensional space, current relative locations of other ones of the plurality of unmanned aerial vehicles which are visible to each of the plurality of unmanned aerial vehicles are determined based on two-dimensional camera images from on-board dual-lens cameras. The current relative locations are wirelessly transmitted to a controller. At the plurality of unmanned aerial vehicles, specification of a path to be followed by each of the plurality of unmanned aerial vehicles is wirelessly received from the controller. At each of the plurality of unmanned aerial vehicles, on-board collision avoidance is carried out based on the two-dimensional camera images from the on-board dual-lens cameras, while following the path specification. ## Claims ### Claim 1 A method comprising: at each of a plurality of unmanned aerial vehicles operating in a three-dimensional space, determining, based on two-dimensional camera images from at least an on-board double lens camera of that unmanned aerial vehicle, current relative locations with reference to that same unmanned aerial vehicle of other ones of said plurality of unmanned aerial vehicles which are visible to that same unmanned aerial vehicle, wherein determining said current relative locations comprises applying multidimensional scaling to said two-dimensional camera images; wirelessly transmitting said current relative locations from at least one of said plurality of unmanned aerial vehicles to a controller; wirelessly receiving, at each of said plurality of unmanned aerial vehicles, from said controller, a specification of a path to be followed by that unmanned aerial vehicle; and at each of said plurality of unmanned aerial vehicles, carrying out on-board collision avoidance based on said two-dimensional camera images, while following said path. ### Claim 2 (depends on 1) The method of claim 1, further comprising, with said controller, determining said specifications of said paths to be followed by each of said plurality of unmanned aerial vehicles, based at least in part on said current relative locations of said plurality of unmanned aerial vehicles. ### Claim 3 (depends on 2) The method of claim 2, further comprising, with said controller, constructing enhanced-accuracy 3D coordinates and velocities for each of said plurality of unmanned aerial vehicles by aggregating current relative locations determined at each of said plurality of unmanned aerial vehicles. ### Claim 4 (depends on 2) The method of claim 2, wherein determining said specification of said path to be followed includes constructing 3D coordinates of each of said plurality of unmanned aerial vehicles from said current relative locations of said plurality of unmanned aerial vehicles and applying a spring-force model to said 3D coordinates. ### Claim 5 (depends on 1) The method of claim 1, wherein carrying out said on-board collision avoidance includes constructing 3D coordinates of said other ones of said plurality of unmanned aerial vehicles from said current relative locations and applying a spring-force model to said 3D coordinates. ### Claim 6 (depends on 1) The method of claim 1, further comprising locating said controller on a control unmanned aerial vehicle. ### Claim 7 (depends on 1) The method of claim 1, further comprising receiving, at each of said plurality of unmanned aerial vehicles, specifications of paths to be followed by other ones of said plurality of unmanned aerial vehicles. ### Claim 8 A system comprising: a plurality of unmanned aerial vehicles operating in a three-dimensional space, each of said unmanned aerial vehicles in turn comprising a memory, at least one processor coupled to said memory, an on-board double lens camera coupled to said processor, and a wireless interface to a remote controller; wherein each of said processors of said plurality of unmanned aerial vehicles is configured to: determine, based on two-dimensional camera images from at least said on-board double lens camera of the unmanned aerial vehicle carrying said processor, current relative locations of other ones of said plurality of unmanned aerial vehicles which are visible to that unmanned aerial vehicle, wherein determining said current relative locations comprises applying multidimensional scaling to said two-dimensional camera images; wirelessly transmit said current relative locations to the remote controller; wirelessly receive, from the remote controller, a specification of a path to be followed by the unmanned aerial vehicle carrying said processor; and carry out on-board collision avoidance based on said two-dimensional camera images from said on-board double lens camera, while following said path. ### Claim 9 (depends on 8) The system of claim 8, further comprising said remote controller, wherein said remote controller determines specifications of paths to be followed by each of said plurality of unmanned aerial vehicles, based at least in part on said current relative locations. ### Claim 10 (depends on 8) The system of claim 8, wherein carrying out said on-board collision avoidance constructing 3D coordinates of said other ones of said plurality of unmanned aerial vehicles from said current relative locations and applying a spring-force model to said 3D coordinates. ### Claim 11 (depends on 9) The system of claim 9, wherein said remote controller is located on a control unmanned aerial vehicle that is one of said plurality of unmanned aerial vehicles. ### Claim 12 (depends on 9) The system of claim 9, further comprising a control unmanned aerial vehicle separate and distinct from said plurality of unmanned aerial vehicles, wherein said remote controller is located on said control unmanned aerial vehicle. ### Claim 13 (depends on 9) The system of claim 9, wherein determining said specification of said path to be followed includes constructing 3D coordinates of each of said plurality of unmanned aerial vehicles from said current relative locations of said plurality of unmanned aerial vehicles and applying a spring-force model to said 3D coordinates. ### Claim 14 (depends on 9) The system of claim 9, wherein said remote controller constructs enhanced-accuracy 3D coordinates and velocities for each of said plurality of unmanned aerial vehicles by aggregating current relative locations determined at each of said plurality of unmanned aerial vehicles. ### Claim 15 A non-transitory computer readable medium comprising computer executable instructions which when executed by a computer cause the computer to perform the method of: at each of a plurality of unmanned aerial vehicles operating in a three-dimensional space, determining, based on two-dimensional camera images from at least an on-board double lens camera of that unmanned aerial vehicle, current relative locations of other ones of said plurality of unmanned aerial vehicles which are visible to that unmanned aerial vehicle, wherein determining said current relative locations comprises applying multidimensional scaling to said two-dimensional camera images; wirelessly transmitting said current relative locations from each of said plurality of unmanned aerial vehicles to a controller; wirelessly receiving, at each of said plurality of unmanned aerial vehicles, from said controller, a specification of a path to be followed by that unmanned aerial vehicle; and at each of said plurality of unmanned aerial vehicles, carrying out on-board collision avoidance based on said two-dimensional camera images, while following said path. ### Claim 16 (depends on 15) The non-transitory computer readable medium of claim 15, wherein said computer executable instructions cause said controller to determine said specifications of said paths to be followed by each of said plurality of unmanned aerial vehicles, based at least in part on said current relative locations of said plurality of unmanned aerial vehicles. ### Claim 17 (depends on 15) The non-transitory computer readable medium of claim 15, wherein said computer executable instructions cause said controller to construct enhanced-accuracy 3D coordinates and velocities for each of said plurality of unmanned aerial vehicles by aggregating current relative locations determined at each of said plurality of unmanned aerial vehicles. ### Claim 18 (depends on 16) The non-transitory computer readable medium of claim 16, wherein determining said specification of said path to be followed includes constructing 3D coordinates of each of said plurality of unmanned aerial vehicles from said current relative locations of said plurality of unmanned aerial vehicles and applying a spring-force model to said 3D coordinates. ### Claim 19 (depends on 1) The method of claim 1, wherein determining said current relative locations comprises applying multidimensional scaling to two-dimensional camera images obtained from a plurality of on-board double lens cameras each mounted on a respective one of said plurality of unmanned aerial vehicles. ### Claim 20 (depends on 8) The system of claim 8, wherein determining said current relative locations comprises applying multidimensional scaling to two-dimensional camera images obtained from a plurality of on-board double lens cameras each mounted on a respective one of said plurality of unmanned aerial vehicles.