# US 2020/0288061 A1 · Drift Correction With Phase And Amplitude Compensation For Optical Image Stabilization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0288061 A1
- Document ID: US20200288061A1
- Status: Published application
- Kind: A1
- Application number: 16/752,009
- Filed: 2020-01-24
- Published: 2020-09-10
- Earliest claimed: 2019-03-07
- Assignees: InvenSense, Inc.
- Applicants: Movea SAS
- Inventors: Pierre Grenet; Sebastien Riccardi; Joe Youssef
- Web page: https://patentgrep.com/patent/US20200288061A1

## CPC

- H04N 23/687: {by shifting the lens or sensor position}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- H04N 23/6811: {based on the image signal}
- H04N 23/6812: {based on additional sensors, e.g. acceleration sensors}
- H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory}
- H04N 23/684: {performed by controlling the image sensor readout, e.g. by controlling the integration time}
- G06T 2207/30244: Camera pose
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An optical image stabilization system and method which also compensate for translation, in addition to rotation compensation. An accelerometer may be used to detect acceleration movement of the camera which may be used to determine translation movement of the camera. In this way, using the acceleration information, a compensation or stabilization may be achieved in each axis based on a translation movement of the camera or the camera-containing device. In an embodiment, the present invention reduces the unwanted signal drift or undesired phase shift due to integration of the accelerometer sensor signal into the resulting translational movement signal. The various contributions to phase delay from the different aspects or portions of the overall system are assessed so a net amount of phase delay may be determined, such that an equal and opposite amount of phase delay may be introduced in order to bring the net phase delay to zero or close to zero. The system takes into account a phase delay component due to the actual motion of the lens (i.e., the shaking or tremor induced motion). There is also a phase delay due to the sensors used to detect the lens motion. In addition, there is a phase delay due to the algorithm which is engaged in order to compute the compensating phase. Finally, there is a phase delay component due to the actuators used to physically implement the lens correction in order to offset the undesired shaking or tremor induced lens motion. All of these phase delay components are taken into account in order to arrive at a zero or near zero overall phase delay.

## Claims

### Claim 1

A method of correcting for signal drift in optical image stabilization of a portable device, comprising:
using an accelerometer to obtain an acceleration signal corresponding to acceleration of the portable device;
removing a gravity component from the accelerometer signal to obtain a proper acceleration signal;
performing a double integration on the proper acceleration signal to obtain a translation signal;
estimating a drift component of the translation signal;
using the estimated drift component to generate a drift correction signal;
determining a phase compensation signal based on a plurality of phase responses of a plurality of components of the optical image stabilization system; and
applying the drift correction signal and the phase compensation signal to an actuator controller which controls movement of a lens within the portable device as part of the process for optical image stabilization.

### Claim 2 (depends on 1)

The method of claim 1, wherein the step of estimating the drift component comprises curve fitting the obtained translation signal.

### Claim 3 (depends on 1)

The method of claim 1, wherein the step of applying the drift correction signal comprises subtracting the drift correction signal from the translation signal.

### Claim 4 (depends on 2)

The method of claim 2, wherein the curve fitting is performed using a predetermined time window.

### Claim 5 (depends on 2)

The method of claim 2, wherein the curve fitting is performed using a plurality of overlapping time windows.

### Claim 6 (depends on 2)

The method of claim 2, wherein the curve fitting is performed for at least two orthogonal axes of motion for the portable device.

### Claim 7 (depends on 1)

The method of claim 1, wherein the step of estimating the drift component is started after receiving a trigger signal.

### Claim 8 (depends on 7)

The method of claim 7, wherein the trigger signal indicates a user input, a start of a camera application of a mobile phone, or a start of an image capture.

### Claim 9 (depends on 1)

The method of claim 1, wherein the plurality of phase responses comprises a phase response of an actuator.

### Claim 10 (depends on 1)

The method of claim 1, wherein the plurality of phase responses comprises a phase response of the accelerometer.

### Claim 11 (depends on 1)

The method of claim 1, wherein the plurality of phase responses comprises a phase response of the step of applying the drift correction signal.

### Claim 12 (depends on 1)

The method of claim 1, wherein the phase compensation signal is determined based on a phase response of an algorithm implemented as part of the image stabilization process.

### Claim 13 (depends on 1)

The method of claim 1, wherein the step of determining the phase compensation signal comprises adjusting the phase response of the drift correction signal to obtain a desired system phase response.

### Claim 14 (depends on 1)

The method of claim 1, further comprising obtaining the accelerometer signal for each of at least two orthogonal axes with respect to the portable device.

### Claim 15 (depends on 1)

The method of claim 1, further comprising the step of using an evaluation module to determine whether or not to perform the image stabilization process.

### Claim 16 (depends on 15)

The method of claim 15, wherein the evaluation module estimates a performance of the drift correction and selectively applies the image stabilization process if a certain threshold of performance is satisfied.

### Claim 17 (depends on 16)

The method of claim 16, wherein the threshold of performance relates to motion attenuation.

### Claim 18

A portable device capable of capturing images, comprising:
a camera assembly including a lens;
an accelerometer capable of detecting motion of the portable device along at least one axis of motion;
an actuator capable of adjusting a position of the lens based on the detected motion;
a processor and a memory containing instructions which when executed by the processor cause the processor to correct for signal drift in an optical image stabilization system of the camera, the processor being programmed to:
obtain from the accelerometer an acceleration signal corresponding to acceleration of the portable device;
remove a gravity component from the accelerometer signal to obtain a proper acceleration signal;
perform a double integration on the proper acceleration signal to obtain a translation signal;
estimate a drift component of the translation signal;
use the estimated drift component to generate a drift correction signal;
determine a phase compensation signal based on a plurality of phase responses of a plurality of components of the optical image stabilization system; and
apply the drift correction signal and the phase compensation signal to an actuator controller which controls movement of the actuator to thereby move the lens within the portable device.

### Claim 19 (depends on 18)

The device of claim 18, wherein the processor is further programmed to estimate the drift component using curve fitting of the obtained translation signal, wherein the curve fitting is performed for at least two orthogonal axes and uses a predetermined time window or a plurality of overlapping time windows.

### Claim 20 (depends on 18)

The device of claim 18, wherein the processor is further programmed to subtract the drift correction signal from the translation signal.

### Claim 21 (depends on 18)

The device of claim 18, wherein the processor is further programmed to start estimating the drift component after receiving a trigger signal, wherein the trigger signal indicates a user input, a start of a camera application of a mobile phone, or a start of an image capture.

### Claim 22 (depends on 18)

The device of claim 18, wherein the plurality of phase responses comprises a phase response of an actuator, a phase response of the accelerometer, or a phase response of the step of applying the drift correction signal.

### Claim 23 (depends on 18)

The device of claim 18, wherein the phase compensation signal is determined based on a phase response of an algorithm implemented as part of the image stabilization process and wherein the processor is further programmed to adjust the phase response of the drift correction signal to obtain a desired system phase response.

### Claim 24 (depends on 18)

The device of claim 18, wherein the processor is further programmed to determine whether or not to activate the image stabilization system based on estimating a performance of the drift correction and selectively activating the image stabilization system if a certain threshold of performance relating to motion attenuation is satisfied.

### Claim 25

A motion processing unit in a portable device, the motion processing unit comprising:
an accelerometer capable of detecting motion of a portable device along at least one axis of motion;
a processor and a memory containing instructions which when executed by the processor cause the processor to correct for signal drift of the motion processing unit, the processor being programmed to:
obtain from the accelerometer an acceleration signal corresponding to acceleration of the portable device;
remove a gravity component from the accelerometer signal to obtain a proper acceleration signal;
perform a double integration on the proper acceleration signal to obtain a translation signal;
estimate a drift component of the translation signal;
use the estimated drift component to generate a drift correction signal;
subtract the drift correction signal from the translation signal to generate a corrected translation signal; and
output the corrected translation signal representing the translation of the portable device.

### Claim 26 (depends on 25)

The motion processing unit of claim 25, wherein the processor is further programmed to estimate the drift component using curve fitting of the obtained translation signal, wherein the curve fitting is performed using a predetermined time window or a plurality of overlapping time windows.

### Claim 27 (depends on 25)

The motion processing unit of claim 25 , wherein the processor is further programmed to determine a phase compensation signal based on a plurality of phase responses of a plurality of components of the portable device, and apply the phase compensation signal to the output of the corrected translation signal.

### Claim 28 (depends on 25)

The motion processing unit of claim 25, wherein the processor is further programmed to start estimating the drift component after receiving a trigger signal, wherein the trigger signal indicates a user input, or a start of an application associated with the portable device.

### Claim 29 (depends on 25)

The motion processing unit of claim 25, further comprising an actuator controller capable of controlling a position of a component of the portable device based on the corrected translation signal.
