# US 2026/0292338 A1 · Electronic Image Stabilization Of Unmanned Aerial Vehicle Video Streams With Intentional Motion Handling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0292338 A1
- Document ID: US20260292338A1
- Status: Published application
- Kind: A1
- Application number: 19/685,702
- Filed: 2026-05-22
- Published: 2026-09-24
- Earliest claimed: 2022-11-29
- Assignees: Skydio, Inc.
- Applicants: Skydio, Inc.
- Inventors: Stepan Moskovchenko; Dominic William Pattison; Adam Parker Bry
- Web page: https://patentgrep.com/patent/US20260292338A1

## CPC

- H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory}
- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- H04N 23/681: {Motion detection}
- H04N 23/6811: {based on the image signal}
- H04N 23/6812: {based on additional sensors, e.g. acceleration sensors}
- H04N 23/6815: {by distinguishing pan or tilt from motion}
- H04N 23/689: {Motion occurring during a rolling shutter mode}
- B64U 2101/30: for imaging, photography or videography
- B64U 2201/20: Remote controls
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR

## Abstract

Disclosed herein are systems and methods for operating an unmanned aerial vehicle (UAV). A method includes capturing a raw video stream input using one or more camera sensors of the UAV, receiving user-commanded motion data representative of motion commanded by a user of the UAV, and receiving total-motion data representative of motion of the one or more camera sensors including the motion commanded by the user. Residual motion data is generated by subtracting the user-commanded motion data from the total-motion data. A frame-based stabilization process operates using the residual motion data, while a rolling-shutter stabilization process operates using the total-motion data. Outputs of the stabilization processes are fused to generate mesh data, which is applied to the raw video stream to produce a stabilized video stream. Separately utilizing different motion data for distinct stabilization processes can enhance stabilization performance and viewing experience.

## Claims

### Claim 1

A method of operating an unmanned aerial vehicle (UAV), the method comprising:
capturing, using one or more camera sensors onboard the UAV, a raw video stream defining a field of view;
receiving, at electronics onboard the UAV, user-commanded motion data representative of motion commanded by a user of the UAV;
receiving, from one or more motion sensors onboard the UAV, total-motion data representative of motion of the one or more camera sensors, the total-motion data including the user-commanded motion;
generating residual motion data by subtracting the user-commanded motion data from the total-motion data;
providing the residual motion data to a frame-based stabilization subprocess;
providing the total-motion data to a rolling-shutter stabilization subprocess without providing the residual motion data to the rolling-shutter stabilization subprocess;
determining, by the frame-based stabilization subprocess and based at least in part on the residual motion data, viewport position data representative of a translational position of a stabilization viewport within the field of view;
determining, by the rolling-shutter stabilization subprocess and based at least in part on the total-motion data, slant data representative of a slant property of the stabilization viewport;
fusing the viewport position data and the slant data to generate mesh data representing at least a shape and slant of the stabilization viewport; and
producing, onboard the UAV, a stabilized video stream from the raw video stream by applying the mesh data to image data of the raw video stream.

### Claim 2 (depends on 1)

The method of claim 1, wherein receiving the user-commanded motion data comprises receiving, by a communications interface of the UAV, a signal transmitted from a ground station and encoding motion commanded by the user.

### Claim 3 (depends on 1)

The method of claim 1, wherein the user-commanded motion data comprises data representative of user-intended angular motion of at least one of the UAV or the one or more camera sensors.

### Claim 4 (depends on 1)

The method of claim 1, wherein generating the residual motion data is performed by a subtractor module onboard the UAV.

### Claim 5 (depends on 1)

The method of claim 1, wherein the total-motion data provided to the rolling-shutter stabilization subprocess includes both user-commanded motion and uncommanded motion.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining the viewport position data comprises determining a position of the stabilization viewport within the field of view on a frame-by-frame basis.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining the slant data comprises determining the slant property corresponding to rolling-shutter distortion of the stabilization viewport.

### Claim 8 (depends on 1)

The method of claim 1, further comprising cropping the field of view by a percentage to define a margin between a perimeter of the field of view and the stabilization viewport.

### Claim 9 (depends on 8)

The method of claim 8, further comprising moving the stabilization viewport within the margin according to an opposing vector that counteracts a vector represented by the residual motion data.

### Claim 10 (depends on 1)

The method of claim 1, wherein the mesh data comprises an array of values describing the shape and the slant of the stabilization viewport.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
generating texture-map data according to the mesh data; and
applying the texture-map data to the image data of the raw video stream to produce the stabilized video stream.

### Claim 12 (depends on 11)

The method of claim 11, wherein applying the texture-map data comprises using a graphics processing unit onboard the UAV to slant, stretch, or deform the stabilization viewport according to the mesh data and the texture-map data.

### Claim 13 (depends on 1)

The method of claim 1, further comprising shaping the stabilization viewport based at least in part on one or more physical or optical properties of a lens associated with the one or more camera sensors.

### Claim 14 (depends on 1)

The method of claim 1, further comprising wirelessly transmitting the stabilized video stream from the UAV to a ground station during flight.

### Claim 15

A system for operating an unmanned aerial vehicle (UAV), the system comprising:
one or more camera sensors onboard the UAV configured to capture a raw video stream defining a field of view;
one or more motion sensors onboard the UAV configured to generate total-motion data representative of motion of the one or more camera sensors, the total-motion data including motion commanded by a user of the UAV; and
at least one computing device onboard the UAV and operably coupled to the one or more camera sensors and the one or more motion sensors, the at least one computing device configured to:
receive user-commanded motion data representative of the motion commanded by the user;
generate residual motion data by subtracting the user-commanded motion data from the total-motion data;
provide the residual motion data to a frame-based stabilization subprocess;
provide the total-motion data to a rolling-shutter stabilization subprocess while not providing the residual motion data to the rolling-shutter stabilization subprocess;
determine, by the frame-based stabilization subprocess and based at least in part on the residual motion data, viewport position data representative of a translational position of a stabilization viewport within the field of view;
determine, by the rolling-shutter stabilization subprocess and based at least in part on the total-motion data, slant data representative of a slant property of the stabilization viewport;
fuse the viewport position data and the slant data to generate mesh data representing at least a shape and slant of the stabilization viewport; and
produce, onboard the UAV, a stabilized video stream from the raw video stream by applying the mesh data to image data of the raw video stream.

### Claim 16 (depends on 15)

The system of claim 15, wherein the at least one computing device comprises an electronic image stabilization module including a frame-based stabilization module, a rolling-shutter stabilization module, and a fusion module.

### Claim 17 (depends on 16)

The system of claim 16, further comprising a graphics processing unit configured to generate a stabilized video stream output by applying at least one of the mesh data or texture-map data generated according to the mesh data.

### Claim 18 (depends on 15)

The system of claim 15, further comprising a motion planner module configured to receive motion commands from a ground station and provide the user-commanded motion data to a subtractor module.

### Claim 19 (depends on 15)

The system of claim 15, further comprising a communications interface configured to transmit the stabilized video stream to a ground station for display during flight of the UAV.

### Claim 20

One or more non-transitory computer-readable media having stored thereon program instructions which, when executed by at least one processor onboard an unmanned aerial vehicle (UAV), cause the UAV to:
capture, using one or more camera sensors onboard the UAV, a raw video stream defining a field of view;
receive user-commanded motion data representative of motion commanded by a user of the UAV;
receive total-motion data representative of motion of the one or more camera sensors, the total-motion data including the user-commanded motion;
generate residual motion data by subtracting the user-commanded motion data from the total-motion data;
provide the residual motion data to a frame-based stabilization subprocess;
provide the total-motion data to a rolling-shutter stabilization subprocess without providing the residual motion data to the rolling-shutter stabilization subprocess;
determine, by the frame-based stabilization subprocess and based at least in part on the residual motion data, viewport position data representative of a translational position of a stabilization viewport within the field of view;
determine, by the rolling-shutter stabilization subprocess and based at least in part on the total-motion data, slant data representative of a slant property of the stabilization viewport;
fuse the viewport position data and the slant data to generate mesh data representing at least a shape and slant of the stabilization viewport; and
produce a stabilized video stream from the raw video stream by applying the mesh data to image data of the raw video stream.
