# US 2020/0053378 A1 · Autofocusing images using motion detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0053378 A1
- Document ID: US20200053378A1
- Status: Published application
- Kind: A1
- Application number: 16/100,809
- Filed: 2018-08-10
- Published: 2020-02-13
- Earliest claimed: 2018-08-10
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Muge Wang; D. Amnon Silverstein
- Web page: https://patentgrep.com/patent/US20200053378A1

## CPC

- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- H04N 19/513: Processing of motion vectors
- G06T 3/02: Affine transformations (for image registration G06T3/147; for image mosaicing G06T3/4038)
- G06T 5/70: Denoising; Smoothing
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 23/675: {comprising setting of focusing regions}
- H04N 25/10: for transforming different wavelengths into image signals
- H04N 25/68: applied to defects
- H04N 5/144: {Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20)}
- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- H04N 23/84: for processing colour signals
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Embodiments of the present disclosure relate to autofocusing of images using motion vectors generated by an image signal processor of a device. An image being processed may include one or more motion detection windows associated with a motion vector as well as one or more autofocus windows. An autofocus window that follows a motion detection window by at least a threshold vertical distance may be selected, for example, to account for a period of time or latency for determining a motion vector of the motion detection window. The device may perform autofocusing by shifting location of the selected autofocus window.

## Claims

### Claim 1

A method comprising:
determining a current motion vector of a motion detection window in a current image;
determining a location of an autofocus window in the current image;
determining whether the motion detection window is followed by the autofocus window after at least a threshold vertical distance based on the location of the autofocus window, the threshold vertical distance accounting for at least a period of time for determining the current motion vector; and
responsive to determining that the motion detection window is followed by the autofocus window after at least the threshold vertical distance, adjusting at least one property of the autofocus window according to at least the current motion vector.

### Claim 2 (depends on 1)

The method of claim 1, wherein adjusting the at least one property of the autofocus window includes:
shifting a horizontal or vertical location of the autofocus window by a horizontal element or vertical element of the current motion vector, respectively.

### Claim 3 (depends on 1)

The method of claim 1, wherein the at least one property of the autofocus window includes the location, a shape, a size, or an orientation of the autofocus window.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
responsive to determining that the motion detection window is followed by the autofocus window within the threshold vertical distance, generating a signal indicating that the current motion vector is not to be used for performing autofocusing on the autofocus window.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
responsive to determining that the motion detection window is followed by the autofocus window within the threshold vertical distance, performing autofocusing on a different autofocus window using another motion vector corresponding to the different autofocus window.

### Claim 6 (depends on 1)

The method of claim 1, wherein the current motion vector is determined by comparing pixel values in blocks of pixels in the current image and pixel values in blocks of pixels in a prior image preceding the current image.

### Claim 7 (depends on 6)

The method of claim 6, wherein comparing the pixel values in blocks of pixels in the current image and the pixel values in blocks of pixels in the prior image comprises:
determining first cross-correlation scores of row sums of pixel values in each row of the blocks of pixels in the current image with row sums of pixel values in each row of the blocks of pixels in the prior image; and
determining second cross-correlation scores of column sums of pixel values in each column of the blocks of pixels in the current image with column sums of pixel values in each column of the blocks of pixels in the prior image.

### Claim 8 (depends on 7)

The method of claim 7, wherein the current motion vector is further determined by:
identifying a vertical shift corresponding to a greatest one of the first cross-correlation scores; and
identifying a horizontal shift corresponding to a greatest one of the second cross-correlation scores, wherein the location of the autofocus window is shifted by the horizontal shift and the vertical shift.

### Claim 9 (depends on 1)

The method of claim 1, wherein the threshold vertical distance further accounts for a threshold vertical shift and coordinates of the motion detection window in a vertical axis.

### Claim 10 (depends on 9)

The method of claim 9, wherein the shifted location of the autofocus window overlaps within the threshold vertical distance, and wherein the location of the autofocus window is shifted less than or equal to the threshold vertical shift along the vertical axis.

### Claim 11 (depends on 9)

The method of claim 9, wherein the threshold vertical shift and a threshold horizontal shift of the autofocus window are each a multiple of a power of two.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
selecting the autofocus window from a plurality of autofocus windows, each of the plurality of autofocus windows having a different location in the current image.

### Claim 13

A non-transitory computer-readable storage medium storing instructions for image processing, the instructions when executed by a processor causing the processor to:
determine a current motion vector of a motion detection window in a current image;
determine a location of an autofocus window in the current image;
determine whether the motion detection window is followed by the autofocus window after at least a threshold vertical distance based on the location of the autofocus window, the threshold vertical distance accounting for at least a period of time for determining the current motion vector; and
responsive to determining that the motion detection window is followed by the autofocus window after at least the threshold vertical distance, adjusting at least one property of the autofocus window according to at least the current motion vector.

### Claim 14 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein adjusting the at least one property of the autofocus window includes:
shifting a horizontal or vertical location of the autofocus window by a horizontal element or vertical element of the current motion vector, respectively.

### Claim 15 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the at least one property of the autofocus window includes the location, a shape, a size or an orientation of the autofocus window.

### Claim 16 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, the instructions when executed by the processor causing the processor to:
responsive to determining that the motion detection window is followed by the autofocus window within the threshold vertical distance, generate a signal indicating that the current motion vector is not to be used for performing autofocusing on the autofocus window.

### Claim 17 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, the instructions when executed by the processor causing the processor to:
responsive to determining that the motion detection window is followed by the autofocus window within the threshold vertical distance, perform autofocusing on a different autofocus window using another motion vector corresponding to the different autofocus window.

### Claim 18 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the current motion vector is determined by comparing pixel values in blocks of pixels in the current image and pixel values in blocks of pixels in a prior image preceding the current image.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, wherein comparing the pixel values in blocks of pixels in the current image and the pixel values in blocks of pixels in the prior image comprises:
determining first cross-correlation scores of row sums of pixel values in each row of the blocks of pixels in the current image with row sums of pixel values in each row of the blocks of pixels in the prior image; and
determining second cross-correlation scores of column sums of pixel values in each column of the blocks of pixels in the current image with column sums of pixel values in each column of the blocks of pixels in the prior image.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein the current motion vector is further determined by:
identifying a vertical shift corresponding to a greatest one of the first cross-correlation scores; and
identifying a horizontal shift corresponding to a greatest one of the second cross-correlation scores, wherein the location of the autofocus window is shifted by the horizontal shift and the vertical shift.
