# US 10,873,702 B2 · Adaptive motion filtering in an unmanned autonomous vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,873,702 B2 - Document ID: US10873702B2 - Status: Granted - Kind: B2 - Application number: 16/324,308 - Filed: 2017-02-24 - Published: 2019-06-13 - Granted: 2020-12-22 - Earliest claimed: 2016-09-23 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated; Yin Huang; Liang Zhang; Xiaoyi Zhu; Ruowei Wang; Jiangtao Ren - Inventors: Yin Huang; Liang Zhang; Xiaoyi Zhu; Ruowei Wang; Jiangtao Ren - Web page: https://patentgrep.com/patent/US10873702B2 ## CPC - H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory} - H04N 5/144: {Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20)} - H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet - H04N 23/6811: {based on the image signal} - H04N 23/6812: {based on additional sensors, e.g. acceleration sensors} - H04N 23/6815: {by distinguishing pan or tilt from motion} - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - B64D 1/02: Dropping, ejecting, or releasing articles (jettisonable fuel reservoirs B64D37/12) - B64U 10/13: Flying platforms - B64U 20/80: Arrangement of on-board electronics, e.g. avionics systems or wiring - B64U 30/20: Rotors; Rotor supports - B64U 60/50: with landing legs - B64U 2101/30: for imaging, photography or videography - B64U 2201/10: autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] - H04N 25/531: by controlling rolling shutters in CMOS SSIS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR ## Abstract Embodiments include devices and methods for adaptive image processing in an unmanned autonomous vehicle (UAV). In various embodiments, an image sensor may capture an image, while a processor of the UAV obtains attitude information from one or more attitude sensors. Such information may include the relative attitude of the UAV and changes in attitude. The processor of the UAV may determine a UAV motion mode based, at least in part, on the obtained attitude information. The UAV motion mode may result in the modification of yaw correction parameters. The processor of the UAV may further execute yaw filtering on the image based, at least in part, on the determined motion mode. ## Claims ### Claim 1 A method of motion filtering an image in an unmanned autonomous vehicle (UAV), comprising: capturing an image by an image sensor associated with the UAV; obtaining attitude information of the UAV by an attitude sensor of the UAV; determining a UAV motion mode based, at least in part, on the obtained attitude information; and executing yaw filtering on the image based, at least in part, on the determined UAV motion mode wherein the executing yaw filtering comprises: calculating an average yaw over a sliding window; calculating a standard deviation of the calculated average yaw; determining whether a yaw threshold is exceeded based, at least in part, on the calculated average yaw and the calculated standard deviation; modifying a yaw correction parameter based, at least in part, on the determination of whether the yaw threshold is exceeded; and executing a filter function using the yaw correction parameter. ### Claim 2 (depends on 1) The method of claim 1, further comprising setting the yaw threshold to a frequency of 35 Hz in response to determining that the UAV motion mode is a fast yaw mode. ### Claim 3 (depends on 1) The method of claim 1, further comprising setting the yaw threshold to a frequency of 25 Hz in response to determining that the UAV motion mode is a hover mode. ### Claim 4 (depends on 1) The method of claim 1, further comprising in response to determining that the yaw threshold is exceeded: incrementing the yaw correction parameter; determining whether a maximum yaw correction is exceeded; and setting the yaw correction parameter to the maximum yaw correction. ### Claim 5 (depends on 1) The method of claim 1, further comprising in response to determining that the yaw threshold is not exceeded: decrementing the yaw correction parameter; determining whether a minimum yaw correction is exceeded; and setting the yaw correction parameter to the minimum yaw correction. ### Claim 6 (depends on 1) The method of claim 1, wherein the filter function includes an infinite impulse response filter. ### Claim 7 An unmanned autonomous vehicle (UAV), comprising: an image sensor; an attitude sensor; and a processor coupled to the image sensor and the attitude sensor, and configured with processor-executable instructions to: capture an image by the image sensor; obtain attitude information of the UAV by the attitude sensor; determine a UAV motion mode based, at least in part, on the obtained attitude information; and execute yaw filtering on the image based, at least in part, on the determined UAV motion mode by: calculating an average yaw over a sliding window; calculating a standard deviation of the calculated average yaw; determining whether a yaw threshold is exceeded based, at least in part, on the calculated average yaw and the calculated standard deviation; modifying a yaw correction parameter based, at least in part, on the determination of whether the yaw threshold is exceeded; and executing a filter function using the yaw correction parameter. ### Claim 8 (depends on 7) The UAV of claim 7, wherein the processor is further configured with processor-executable instructions to set the yaw threshold to a frequency of 35 Hz in response to determining that the UAV motion mode is a fast yaw mode. ### Claim 9 (depends on 7) The UAV of claim 7, wherein the processor is further configured with processor-executable instructions to set the yaw threshold to a frequency of 25 Hz in response to determining that the UAV motion mode is a hover mode. ### Claim 10 (depends on 7) The UAV of claim 7, wherein in response to determining that the yaw threshold is exceeded the processor is further configured with processor-executable instructions to: increment the yaw correction parameter; determine whether a maximum yaw correction is exceeded; and set the yaw correction parameter to the maximum yaw correction. ### Claim 11 (depends on 7) The UAV of claim 7, wherein in response to determining that the yaw threshold is not exceeded the processor is further configured with processor-executable instructions to: decrement the yaw correction parameter; determine whether a minimum yaw correction is exceeded; and setting the yaw correction parameter to the minimum yaw correction. ### Claim 12 (depends on 7) The UAV of claim 7, wherein the filter function includes an infinite impulse response filter. ### Claim 13 An unmanned autonomous vehicle (UAV), comprising: means for capturing an image; means for obtaining attitude information of the UAV; means for determining a UAV motion mode based, at least in part, on the obtained attitude information; and means for executing yaw filtering on the image based, at least in part, on the determined UAV motion mode comprising: means for calculating an average yaw over a sliding window; means for calculating a standard deviation of the calculated average yaw; means for determining whether a yaw threshold is exceeded based, at least in part, on the calculated average yaw and the calculated standard deviation; means for modifying a yaw correction parameter based, at least in part, on the determination of whether the yaw threshold is exceeded; and means for executing a filter function using the yaw correction parameter. ### Claim 14 A non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of an unmanned autonomous vehicle (UAV) to perform operations comprising: capturing an image by an image sensor associated with the UAV; obtaining attitude information of the UAV by an attitude sensor of the UAV; determining a UAV motion mode based, at least in part, on the obtained attitude information; and executing yaw filtering on the image based, at least in part, on the determined UAV motion mode by: calculating an average yaw over a sliding window; calculating a standard deviation of the calculated average yaw; determining whether a yaw threshold is exceeded based, at least in part, on the calculated average yaw and the calculated standard deviation; modifying a yaw correction parameter based, at least in part, on the determination of whether the yaw threshold is exceeded; and executing a filter function using the yaw correction parameter. ### Claim 15 (depends on 14) The non-transitory processor-readable storage medium of claim 14, wherein the stored processor-executable instructions are configured to cause the processor of the UAV to perform operations further comprising setting the yaw threshold to a frequency of 35 Hz in response to determining that the UAV motion mode is a fast yaw mode. ### Claim 16 (depends on 14) The non-transitory processor-readable storage medium of claim 14, wherein the stored processor-executable instructions are configured to cause the processor of the UAV to perform operations further comprising setting the yaw threshold to a frequency of 25 Hz in response to determining that the UAV motion mode is a hover mode. ### Claim 17 (depends on 14) The non-transitory processor-readable storage medium of claim 14, wherein the stored processor-executable instructions are configured to cause the processor of the UAV to perform operations further comprising in response to determining that the yaw threshold is exceeded: incrementing the yaw correction parameter; determining whether a maximum yaw correction is exceeded; and setting the yaw correction parameter to the maximum yaw correction. ### Claim 18 (depends on 14) The non-transitory processor-readable storage medium of claim 14, wherein the stored processor-executable instructions are configured to cause the processor of the UAV to perform operations further comprising in response to determining that the yaw threshold is not exceeded: decrementing the yaw correction parameter; determining whether a minimum yaw correction is exceeded; and setting the yaw correction parameter to the minimum yaw correction. ### Claim 19 (depends on 14) The non-transitory processor-readable storage medium of claim 14, wherein the stored processor-executable instructions are further configured to cause the processor of the UAV to perform operations such that the filter function includes an infinite impulse response filter.