# US 2020/0404178 A1 · Video image anti-shake method and terminal

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0404178 A1
- Document ID: US20200404178A1
- Status: Published application
- Kind: A1
- Application number: 16/976,820
- Filed: 2018-03-23
- Published: 2020-12-24
- Earliest claimed: 2018-03-23
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Yuanyou LI; Wei LUO; Huanyu LIU
- Web page: https://patentgrep.com/patent/US20200404178A1

## CPC

- G03B 5/00: Adjustment of optical system relative to image or object surface other than for focusing
- H04N 23/6812: {based on additional sensors, e.g. acceleration sensors}
- G06T 5/80: Geometric correction
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
- H04N 23/671: {in combination with active ranging signals, e.g. using light or sound signals emitted toward objects}
- H04N 23/6811: {based on the image signal}
- H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory}
- H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20132: Image cropping
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

This application discloses a video image anti-shake method and a terminal, and relates to the field of image processing, to implement compensation for translational shake in a Z direction. A video image anti-shake method includes: turning on, by a terminal, a camera lens, and photographing a video image by using the camera lens; detecting, by the terminal, shake on an X-axis, a Y-axis, and a Z-axis during photographing, where the Z-axis is an optical axis of the camera lens, the X-axis is an axis perpendicular to the Z-axis on a horizontal plane, and the Y-axis is an axis perpendicular to the Z-axis on a vertical plane; and performing, by the terminal, anti-shake processing on the video image based on the shake on the X-axis, the Y-axis, and the Z-axis. Embodiments of this application are applied to video image anti-shake.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A video image anti-shake method, comprising:
turning on, by a terminal, a camera lens, and photographing a video image by using the camera lens;
detecting, by the terminal, shake on an X-axis, a Y-axis, and a Z-axis during photographing, wherein the Z-axis is an optical axis of the camera lens, the X-axis is an axis perpendicular to the Z-axis on a horizontal plane, and the Y-axis is an axis perpendicular to the Z-axis on a vertical plane; and
performing, by the terminal, anti-shake processing on the video image based on the shake on the X-axis, the Y-axis, and the Z-axis, wherein the anti-shake processing comprises video image processing.

### Claim 22 (depends on 21)

The method according to claim 21, wherein
the method further comprises: detecting, by the terminal, an object distance, wherein the object distance is a distance to a focused object or person; and
wherein detecting, by the terminal, shake on an X-axis, a Y-axis, and a Z-axis comprises:
if the object distance is greater than or equal to an object distance threshold, detecting, by the terminal, rotation shake on the X-axis, the Y-axis, and the Z-axis; or if the object distance is less than an object distance threshold, detecting, by the terminal, rotation shake on the X-axis, the Y-axis, and the Z-axis, and detecting, by the terminal, translational shake on the X-axis, the Y-axis, and the Z-axis.

### Claim 23 (depends on 22)

The method according to claim 22, wherein detecting, by the terminal, the object distance comprises:
detecting, by the terminal, the object distance by using a depth sensor, wherein the depth sensor comprises at least one of the following sensors: a laser sensor, a time of flight TOF sensor, and a structured light sensor.

### Claim 24 (depends on 22)

The method according to claim 22, wherein detecting, by the terminal, the rotation shake on the X-axis, Y-axis, and Z-axis comprises:
detecting, by the terminal, the rotation shake on the X-axis, the Y-axis, and the Z-axis by using an angle sensor, wherein the angle sensor comprises a gyroscope.

### Claim 25 (depends on 22)

The method according to claim 22, wherein detecting, by a terminal, the translational shake on the X-axis, Y-axis, and Z-axis comprises:
detecting, by the terminal, the translational shake on the X-axis, the Y-axis, and the Z-axis by using a displacement sensor, wherein the displacement sensor comprises an accelerometer.

### Claim 26 (depends on 22)

The method according to claim 22, wherein
the method further comprises: detecting, by the terminal, an image distance; and
wherein performing, by the terminal, anti-shake processing on the video image based on the shake on the X-axis, the Y-axis, and the Z-axis comprises:
performing, by the terminal, anti-shake processing on the video image based on the object distance, the image distance, and the shake on the X-axis, the Y-axis, and the Z-axis.

### Claim 27 (depends on 26)

The method according to claim 26, wherein detecting, by the terminal, the image distance comprises:
detecting, by the terminal, the image distance by using a location sensor, wherein the location sensor comprises at least one of the following sensors: a Hall effect sensor, an anisotropic magneto resistance (AMR) sensor, a giant magneto resistance (GMR) sensor, and a tunneling magneto resistance (TMR) sensor.

### Claim 28 (depends on 26)

The method according to claim 26, wherein performing, by the terminal, the anti-shake processing on the video image based on the object distance, the image distance, and the shake on the X-axis, the Y-axis, and the Z-axis comprises:
for image shake caused by rotation of the terminal around the Z-axis, performing, by the terminal, rotation compensation on the video image in an opposite direction of the rotation and at a same angle of the rotation;
for image shake caused by rotation shake of the terminal around the X-axis or rotation shake around the Y-axis, performing, by the terminal based on a formula d=v*tan(θ), compensation on the video image with a same translation distance d in an opposite direction of translation of the video image, and restoring trapezoidal distortion of the video image to a rectangle in a rotation axis direction based on a trapezoidal correction algorithm, wherein d is an image displacement distance, v is the image distance, and θ is an angle of rotation around the X-axis or the Y-axis; and
for image shake caused by the translational shake of the terminal on the X-axis or the Y-axis, performing, by the terminal based on a formula d=(v+u)*Δ/v, compensation on the video image with a same translation distance d in an opposite direction of translation of the video image, and cropping a redundant part of the video image, wherein d is an image displacement distance, v is the image distance, u is the object distance, and Δ is a translation distance of the terminal; and
for image shake caused by the translational shake of the terminal on the Z-axis, zooming, by the terminal, the video image based on a zoom ratio (u+Δ)/u, wherein u is the object distance, and Δ is a translation distance of the terminal in a direction away from an object on the Z-axis.

### Claim 29 (depends on 26)

The method according to claim 26, wherein the method further comprises:
for image shake caused by focus adjustment of the terminal, zooming, by the terminal, the video image based on a zoom ratio [(u−Δ)v]/[(v+Δ)u], wherein u is the object distance, v is the image distance, and Δ is a distance by which a lens moves towards the object.

### Claim 30

A terminal, comprising: a processor, a display, a memory, and a communications interface; wherein
the memory is configured to store a computer execution instruction, the processor is coupled to the memory, and when the terminal is run, the processor executes the computer execution instruction stored in the memory, so that the terminal performs the method comprising:
turning on, by a terminal, a camera lens, and photographing a video image by using the camera lens;
detecting, by the terminal, shake on an X-axis, a Y-axis, and a Z-axis during photographing, wherein the Z-axis is an optical axis of the camera lens, the X-axis is an axis perpendicular to the Z-axis on a horizontal plane, and the Y-axis is an axis perpendicular to the Z-axis on a vertical plane; and
performing, by the terminal, anti-shake processing on the video image based on the shake on the X-axis, the Y-axis, and the Z-axis, wherein the anti-shake processing comprises video image processing.

### Claim 31 (depends on 30)

The terminal according to claim 30, wherein
the method further comprises: detecting, by the terminal, an object distance, wherein the object distance is a distance to a focused object or person; and
detecting, by the terminal, the shake on the X-axis, the Y-axis, and the Z-axis comprises:
if the object distance is greater than or equal to an object distance threshold, detecting, by the terminal, rotation shake on the X-axis, the Y-axis, and the Z-axis; or if the object distance is less than an object distance threshold, detecting, by the terminal, rotation shake on the X-axis, the Y-axis, and the Z-axis, and detecting, by the terminal, translational shake on the X-axis, the Y-axis, and the Z-axis.

### Claim 32 (depends on 31)

The terminal according to claim 31, wherein detecting, by the terminal, the object distance comprises:
detecting, by the terminal, the object distance by using a depth sensor, wherein the depth sensor comprises at least one of the following sensors: a laser sensor, a time of flight TOF sensor, and a structured light sensor.

### Claim 33 (depends on 31)

The terminal according to claim 31, wherein detecting, by the terminal, the rotation shake on the X-axis, Y-axis, and Z-axis comprises:
detecting, by the terminal, the rotation shake on the X-axis, the Y-axis, and the Z-axis by using an angle sensor, wherein the angle sensor comprises a gyroscope.

### Claim 34 (depends on 31)

The terminal according to claim 31, wherein detecting, by the terminal, the translational shake on the X-axis, Y-axis, and Z-axis comprises:
detecting, by the terminal, the translational shake on the X-axis, the Y-axis, and the Z-axis by using a displacement sensor, wherein the displacement sensor comprises an accelerometer.

### Claim 35 (depends on 31)

The terminal according to claim 31, wherein
the method further comprises: detecting, by the terminal, an image distance; and
wherein performing, by the terminal, anti-shake processing on the video image based on the shake on the X-axis, the Y-axis, and the Z-axis comprises:
performing, by the terminal, anti-shake processing on the video image based on the object distance, the image distance, and the shake on the X-axis, the Y-axis, and the Z-axis.

### Claim 36 (depends on 35)

The terminal according to claim 35, wherein detecting, by the terminal, the image distance comprises:
detecting, by the terminal, the image distance by using a location sensor, wherein the location sensor comprises at least one of the following sensors: a Hall effect sensor, an anisotropic magneto resistance (AMR) sensor, a giant magneto resistance (GMR) sensor, and a tunneling magneto resistance (TMR) sensor.

### Claim 37 (depends on 35)

The terminal according to claim 35, wherein performing, by the terminal, the anti-shake processing on the video image based on the object distance, the image distance, and the shake on the X-axis, the Y-axis, and the Z-axis comprises:
for image shake caused by rotation of the terminal around the Z-axis, performing, by the terminal, rotation compensation on the video image in an opposite direction of the rotation and at a same angle of the rotation;
for image shake caused by rotation shake of the terminal around the X-axis or rotation shake around the Y-axis, performing, by the terminal based on a formula d=v*tan(θ), compensation on the video image with a same translation distance d in an opposite direction of translation of the video image, and restoring trapezoidal distortion of the video image to a rectangle in a rotation axis direction based on a trapezoidal correction algorithm, wherein d is an image displacement distance, v is the image distance, and θ is an angle of rotation around the X-axis or the Y-axis; and
for image shake caused by the translational shake of the terminal on the X-axis or the Y-axis, performing, by the terminal based on a formula d=(v+u)*Δ/v, compensation on the video image with a same translation distance d in an opposite direction of translation of the video image, and cropping a redundant part of the video image, wherein d is an image displacement distance, v is the image distance, u is the object distance, and Δ is a translation distance of the terminal; and
for image shake caused by the translational shake of the terminal on the Z-axis, zooming, by the terminal, the video image based on a zoom ratio (u+Δ)/u, wherein u is the object distance, and Δ is a translation distance of the terminal in a direction away from an object on the Z-axis.

### Claim 38 (depends on 35)

The terminal according to claim 35, wherein the method further comprises:
for image shake caused by focus adjustment of the terminal, zooming, by the terminal, the video image based on a zoom ratio [(u−Δ)v]/[(v+Δ)u], wherein u is the object distance, v is the image distance, and Δ is a distance by which a lens moves towards the object.

### Claim 39

A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores an instruction, and when the instruction is run on a terminal, the terminal is enabled to perform the method:
turning on, by a terminal, a camera lens, and photographing a video image by using the camera lens;
detecting, by the terminal, shake on an X-axis, a Y-axis, and a Z-axis during photographing, wherein the Z-axis is an optical axis of the camera lens, the X-axis is an axis perpendicular to the Z-axis on a horizontal plane, and the Y-axis is an axis perpendicular to the Z-axis on a vertical plane; and
performing, by the terminal, anti-shake processing on the video image based on the shake on the X-axis, the Y-axis, and the Z-axis, wherein the anti-shake processing comprises video image processing.

### Claim 40 (depends on 39)

The computer-readable storage medium according to claim 39, wherein
the method further comprises: detecting, by the terminal, an object distance, wherein the object distance is a distance to a focused object or person; and
detecting, by the terminal, the shake on the X-axis, the Y-axis, and the Z-axis comprises:
if the object distance is greater than or equal to an object distance threshold, detecting, by the terminal, rotation shake on the X-axis, the Y-axis, and the Z-axis; or if the object distance is less than an object distance threshold, detecting, by the terminal, rotation shake on the X-axis, the Y-axis, and the Z-axis, and detecting, by the terminal, translational shake on the X-axis, the Y-axis, and the Z-axis.
