# US 2020/0288060 A1 · Translation Correction For Optical Image Stabilization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0288060 A1
- Document ID: US20200288060A1
- Status: Published application
- Kind: A1
- Application number: 16/751,908
- Filed: 2020-01-24
- Published: 2020-09-10
- Earliest claimed: 2019-03-07
- Assignees: InvenSense, Inc.
- Applicants: Movea SAS
- Inventors: Pierre Grenet
- Web page: https://patentgrep.com/patent/US20200288060A1

## CPC

- H04N 23/687: {by shifting the lens or sensor position}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- H04N 23/6811: {based on the image signal}
- H04N 23/6812: {based on additional sensors, e.g. acceleration sensors}
- H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory}
- H04N 23/684: {performed by controlling the image sensor readout, e.g. by controlling the integration time}
- G06T 2207/30244: Camera pose
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An optical image stabilization system and method which also compensate for translation, in addition to rotation compensation. This translation compensation may be performed for one, two or three axes. A gyroscopic sensor may be used to detect the angular velocity of the camera and from the angular velocity, the rotation of the device can be determined, such as for example, by integration. This rotation may be determined for one or more axes. An accelerometer may be used to detect acceleration movement of the camera which may be used to determine translation movement of the camera. A two axis or two dimension accelerometer may be used to provide acceleration information along each of two different axes or dimensions. In this way, using the acceleration information for each axis or dimension, a compensation or stabilization may be achieved in each of the two axes or dimensions based on a translation movement of the camera or the camera-containing device.

## Claims

### Claim 1

A method for optically stabilizing an image obtained using a camera unit, comprising the following steps:
using a lens to obtain optical image information corresponding to an image;
providing the optical image information to an image sensor to convert the optical image information into electrical image information;
generating a rotation signal related to rotational movement of the camera unit using a first motion sensor capable of providing a signal related to rotation when the camera unit is rotated;
generating a translation signal related to translational movement of the camera unit using a second motion sensor capable of providing a signal related to translation when the camera unit undergoes translational movement;
modifying the translation signal using the rotation signal to generate a corrected translation signal, wherein the corrected translation signal compensates for movement of the lens which is different from movement of the second motion sensor; and
applying the corrected translation signal as input to control an actuator capable of adjusting a position of the lens with respect to the image sensor, based on rotation and translation experienced by the camera unit.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first motion sensor includes a first axis gyroscope capable of measuring angular velocity with respect to a first axis, and the step of generating a rotation signal includes generating a signal corresponding to angular velocity of the camera unit, the method further comprising the step of integrating the angular velocity to obtain the rotation signal.

### Claim 3 (depends on 2)

The method of claim 2, wherein the second motion sensor includes a second axis accelerometer capable of measuring acceleration with respect to a second axis and a third axis accelerometer capable of measuring acceleration with respect to a third axis, with the first axis, second axis, and third axis being orthogonal to each other, and the step of generating the translation signal includes generating a signal corresponding to acceleration of the camera unit, the method further comprising the step of integrating the acceleration with respect to the second axis and the acceleration with respect to the third axis to obtain a translation with respect to the second and third axes.

### Claim 4 (depends on 1)

The method of claim 1, wherein the step of generating the corrected translation signal comprises the further steps of:
utilizing a motion sensor/lens vector signal to indicate a distance and direction between the second motion sensor and the lens to thereby determine a location of the lens with respect to the second motion sensor;
generating a rotation correction signal by moving the motion sensor/lens vector signal by an amount corresponding to the rotation signal;
utilizing the rotation correction signal in the step of modifying the translation signal to thereby generate the corrected translation signal.

### Claim 5 (depends on 4)

The method of claim 4, wherein the step of generating the rotation correction signal further comprises the step of rotating the motion sensor/lens vector signal over an angle corresponding to the rotation signal to thereby generate the rotation correction signal.

### Claim 6 (depends on 4)

The method of claim 4, wherein the motion sensor/lens vector signal is stored in a memory device of the camera unit.

### Claim 7 (depends on 4)

The method of claim 4, wherein the motion sensor/lens vector signal is obtained using a calibration process.

### Claim 8 (depends on 7)

The method of claim 7, wherein the motion sensor/lens vector signal is obtained by comparing motion determined by the first and second motion sensors, with motion determined using image analysis.

### Claim 9 (depends on 8)

The method of claim 8, wherein the comparing further comprises:
estimating translation of the camera unit based on signals from the second motion sensor;
determining rotation of the camera unit based on signals from the first motion sensor;
determining translation of the camera unit based on image analysis of images captured by the image sensor;
determining a difference between the translation determined from the second motion sensor signals, and the translation determined from the image analysis; and
generating the motion sensor/lens vector signal based on the determined rotation of the camera unit and the difference between the translation determined from the second motion sensor signals, and the translation determined from the image analysis.

### Claim 10 (depends on 1)

The method of claim 1, wherein the step of adjusting the position of the lens with respect to the image sensor further comprises adjusting the position of one or both of the lens and the image sensor.

### Claim 11

A device for optically stabilizing an image obtained using a camera unit, comprising:
a lens positioned in the camera unit and capable of obtaining optical image information corresponding to an image;
an image sensor positioned in the camera unit and in optical communication with the lens and capable of converting the optical image information into electrical image information;
a first motion sensor positioned in the camera unit and capable of generating a signal related to rotational movement of the camera unit when the camera unit is rotated;
a second motion sensor positioned in the camera unit and capable of generating a signal related to translational movement of the camera unit when the camera unit undergoes translational movement; and
an actuator capable of adjusting a relative position between the lens and the image sensor based on a corrected translation signal obtained from the signal related to translational movement and the signal related to rotational movement to thereby adjust the relative position of the lens and image sensor based on rotation and translation experienced by the camera unit.

### Claim 12 (depends on 11)

The device of claim 11, wherein the first motion sensor includes a gyroscope with at least one axis, the second motion sensor includes an accelerometer with at least two axes, and the at least one gyroscope axis and the at least two accelerometer axes are orthogonal.

### Claim 13 (depends on 11)

The device of claim 11, wherein the first motion sensor and the second motion sensor are integrated in a single assembly.

### Claim 14 (depends on 11)

The device of claim 11, wherein the first motion sensor and the second motion sensor are MEMS based motion sensors.

### Claim 15 (depends on 11)

The device of claim 11, wherein the actuator capable of adjusting the position of the lens with respect to the image sensor is capable of adjusting the position of one or both of the lens and the image sensor.

### Claim 16 (depends on 11)

The device of claim 11, further comprising a memory for storing a motion sensor/lens vector signal to indicate a distance and direction between the second motion sensor and the lens to thereby determine a location of the lens with respect to the second motion sensor, and further comprising a processor capable of generating a rotation correction signal by moving the motion sensor/lens vector signal by an amount corresponding to the rotation signal and utilizing the rotation correction signal to modify the translation signal to thereby generate the corrected translation signal.

### Claim 17

A system for optically stabilizing an image obtained using a camera unit, comprising:
a programmed processor;
a memory containing instructions, which when executed by the processor cause the processor to perform the following steps:
use a lens to obtain optical image information corresponding to an image;
provide the optical image information to an image sensor to convert the optical image information into electrical image information;
generate a signal related to rotational movement of the camera unit using a first motion sensor capable of providing a signal related to rotation when the camera unit is rotated;
generate a signal related to translational movement of the camera unit using a second motion sensor capable of providing a signal related to translation when the camera unit undergoes translational movement;
modify the signal related to translational movement of the camera unit using the signal related to rotational movement of the camera to generate a corrected translation signal which compensates for movement of the lens which is different from movement of the second motion sensor; and
apply the corrected translation signal as input to control an actuator capable of adjusting a relative position of the lens with respect to the image sensor based on rotation and translation experienced by the camera unit.

### Claim 18 (depends on 17)

The system of claim 17, wherein the first motion sensor includes a gyroscope with at least one axis, the second motion sensor includes an accelerometer with at least two axes, and the at least one gyroscope axis and the at least two accelerometer axes are orthogonal.

### Claim 19 (depends on 17)

The system of claim 17, wherein the first motion sensor and the second motion sensor are integrated in a single assembly.

### Claim 20 (depends on 17)

The system of claim 17, wherein the first motion sensor and the second motion sensor are MEMS based motion sensors.

### Claim 21 (depends on 17)

The system of claim 17, wherein the actuator capable of adjusting the position of the lens with respect to the image sensor is capable of adjusting the position of one or both of the lens and the image sensor.

### Claim 22 (depends on 17)

The device of claim 17, further comprising a memory for storing a motion sensor/lens vector signal to indicate a distance and direction between the second motion sensor and the lens to thereby determine a location of the lens with respect to the second motion sensor, and further comprising a processor capable of generating a rotation correction signal by moving the motion sensor/lens vector signal by an amount corresponding to the rotation signal and utilizing the rotation correction signal to modify the translation signal to thereby generate the corrected translation signal.
