# US 2013/0253733 A1 · Computing device and method for controlling unmanned aerial vehicle in flight space > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2013/0253733 A1 - Document ID: US20130253733A1 - Status: Published application - Kind: A1 - Application number: 13/674,959 - Filed: 2012-11-13 - Published: 2013-09-26 - Earliest claimed: 2012-03-26 - Assignees: Hon Hai Precision Industry Co., Ltd. - Applicants: Hon Hai Precision Industry Co., Ltd. - Inventors: HOU-HSIEN LEE; CHANG-JUNG LEE; CHIH-PING LO - Web page: https://patentgrep.com/patent/US20130253733A1 ## CPC - G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)} - B64C 19/00: Aircraft control not otherwise provided for - G05D 1/0016: {characterised by the operator's input device (input arrangements for computing systems in general G06F3/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - B64U 10/17: Helicopters (flying platforms B64U10/13) - B64U 2201/00: UAVs characterised by their flight controls - B64U 2201/20: Remote controls - G05D 1/0202: {specially adapted to aircraft} - G06T 2207/10016: Video; Image sequence - G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE - B64C: AEROPLANES; HELICOPTERS - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract In a method for controlling an unmanned aerial vehicle (UAV) in a flight space using a computing device, a 3D sample database is created and store in a storage device of the computing device. The computing device includes a depth-sensing camera that captures a 3D scene image of a scene in front of a user, and senses a depth distance between the user and the depth-sensing camera. A 3D person image of the user is detected from the 3D scene image, and gesture information of the user is obtained by comparing the 3D person image with human gesture data stored in the 3D sample database. The method converts the gesture information of the user into one or more flight control commands, and drives a driver of the UAV to control the UAV to fly in a flight space according to the flight control commands ## Claims ### Claim 1 A computing device, comprising: a depth-sensing camera; a storage device; at least one processor; and one or more programs stored in the storage device comprising one or more programs and executable by the at least one processor, the one or more programs comprising: an image detecting module that captures a 3D scene image of a scene in front of a user using the depth-sensing camera, obtains a depth distance between the user and the depth-sensing camera using the depth-sensing camera, detects a 3D person image of the user from the 3D scene image, and compares the 3D person image with human gesture data stored in a 3D sample database to obtain gesture information of the user from the 3D person image; and a flight control module that converts the gesture information of the user into one or more flight control commands, sends the flight control commands to the UAV through a wireless network, and driving a driver of the UAV to control the UAV to fly in a flight space or adjust a flying mode of the UAV according to the flight control commands. ### Claim 2 (depends on 1) The computing device according to claim 1, wherein the depth-sensing camera is a time of flight (TOF) camera device having a 3D image capturing functionality. ### Claim 3 (depends on 1) The computing device according to claim 1, wherein the depth-sensing camera senses the depth distance between the depth-sensing camera and the user according to a pulse time of a short light illuminated at the user and a reflected time of a reflected light of the short light. ### Claim 4 (depends on 1) The computing device according to claim 1, wherein the 3D sample database is created by performing steps of: capturing the human gesture data using the depth-sensing camera; building a data relationship between each of the gesture data and each of the flight control commands of the UAV; and storing the human gesture data in the 3D sample database to determine a gesture and action of the user when the user controls the UAV to the fly in the flight space. ### Claim 5 (depends on 1) The computing device according to claim 1, wherein the 3D person image of the user is detected from the 3D scene image by performing steps of: converting a depth distance between each point of the user and the depth-sensing camera into a pixel value, and storing all of the pixel value in a data array; and comparing the pixel values of the data array with characteristic data of the user stored in the 3D sample database to obtain the 3D person image from the 3D scene image. ### Claim 6 (depends on 1) The computing device according to claim 1, wherein the flight control commands comprise a first control command for controlling the UAV to fly upwards, downwards, leftwards or rightwards, and a second adjustment command for adjusting a flying speed or a flying direction of the UAV. ### Claim 7 A method for controlling an unmanned aerial vehicle (UAV) in a flight space using a computing device, the method comprising: capturing a 3D scene image of a scene in front of a user using a depth-sensing camera of the computing device, and obtaining a depth distance between the user and the depth-sensing camera using the depth-sensing camera; detecting a 3D person image of the user from the 3D scene image; comparing the 3D person image with gesture data stored in a 3D sample database to obtain gesture information of the user from the 3D person image; converting the gesture information of the user to one or more flight control commands, and sending the flight control commands to the UAV through a wireless network; and driving a driver of the UAV to control the UAV to fly in a flight space or adjust a flying mode of the UAV according to the flight control commands ### Claim 8 (depends on 7) The method according to claim 7, wherein the depth-sensing camera is a time of flight (TOF) camera device having a 3D image capturing function. ### Claim 9 (depends on 7) The method according to claim 7, wherein the depth-sensing camera senses the depth distance between the depth-sensing camera and the user according to a pulse time of a short light illuminated at the user and a reflected time of a reflected light of the short light. ### Claim 10 (depends on 7) The method according to claim 7, wherein the 3D sample database is created by performing steps of: capturing the human gesture data using the depth-sensing camera; building a data relationship between each of the gesture data and each of the flight control commands of the UAV; and storing the human gesture data in the 3D sample database to determine a gesture and action of the user when the user controls the UAV to the fly in the flight space. ### Claim 11 (depends on 7) The method according to claim 7, wherein the 3D person image of the user is detected from the 3D scene image by performing steps of: converting a depth distance between each point of the user and the depth-sensing camera into a pixel value, and storing all of the pixel values in a data array; and comparing the pixel values of the data array with characteristic data of the user stored in the 3D sample database to obtain the 3D person image from the 3D scene image. ### Claim 12 (depends on 7) The method according to claim 7, wherein the flight control commands comprise a first control command for controlling the UAV to fly upwards, downwards, leftwards or rightwards, and a second adjustment command for adjusting a flying speed or a flying direction of the UAV. ### Claim 13 A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor of a computing device, cause the computing device to perform a method for controlling an unmanned aerial vehicle (UAV) in a flight space, the method comprising: capturing a 3D scene image of a scene in front of a user using a depth-sensing camera of the computing device, and obtaining a depth distance between the user and the depth-sensing camera using the depth-sensing camera; detecting a 3D person image of the user from the 3D scene image; comparing the 3D person image with human gesture data stored in a 3D sample database to obtain gesture information of the user from the 3D person image; converting the gesture information of the user to one or more flight control commands, and sending the flight control commands to the UAV through a wireless network; and driving a driver of the UAV to control the UAV to fly in a flight space or adjust a flying mode of the UAV according to the flight control commands ### Claim 14 (depends on 13) The storage medium according to claim 13, wherein the depth-sensing camera is a time of flight (TOF) camera device having a 3D image capturing functionality. ### Claim 15 (depends on 13) The storage medium according to claim 13, wherein the depth-sensing camera senses the depth distance between the depth-sensing camera and the user according to a pulse time of a short light illuminated at the user and a reflected time of a reflected light of the short light. ### Claim 16 (depends on 13) The storage medium according to claim 13, wherein the 3D sample database is created by performing steps of: capturing the human gesture data using the depth-sensing camera; building a data relationship between each of the human gesture data and each of the flight control commands of the UAV; and storing the human gesture data in the 3D sample database to determine a gesture and action of the user when the user controls the UAV to the fly in the flight space. ### Claim 17 (depends on 13) The storage medium according to claim 13, wherein the 3D person image of the user is detected from the 3D scene image by performing steps of: converting a depth distance between each point of the user and the depth-sensing camera into a pixel value, and storing all of the pixel value in a data array; and comparing the pixel values of the data array with characteristic data of the user stored in the 3D sample database to obtain the 3D person image from the 3D scene image. ### Claim 18 (depends on 13) The storage medium according to claim 13, wherein the flight control commands comprise a first control command for controlling the UAV to fly upwards, downwards, leftwards or rightwards, and a second adjustment command for adjusting a flying speed or a flying direction of the UAV.