# US 2015/0103192 A1 · Refocusable images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0103192 A1
- Document ID: US20150103192A1
- Status: Published application
- Kind: A1
- Application number: 14/053,436
- Filed: 2013-10-14
- Published: 2015-04-16
- Earliest claimed: 2013-10-14
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Subramaniam Venkatraman; Victor Kulik
- Web page: https://patentgrep.com/patent/US20150103192A1

## CPC

- H04N 23/676: {Bracketing for image capture at varying focusing conditions}
- G06T 7/00: Image analysis
- H04N 23/63: by using electronic viewfinders
- H04N 23/673: {based on contrast or high frequency components of image signals, e.g. hill climbing method}
- H04N 23/681: {Motion detection}
- H04N 23/6812: {based on additional sensors, e.g. acceleration sensors}
- H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory}
- H04N 23/74: by influencing the scene brightness using illuminating means
- H04N 23/743: Bracketing, i.e. taking a series of images with varying exposure conditions
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Embodiments of the present invention are directed toward providing refocusable image. Refocusable images are images that, after being captured, can be refocused, or adjusted to have a different focal length. The embodiments described herein can utilize a mobile user device, such as a smartphone, without the need for specialized hardware. A camera of the mobile device can be configured to take a series of images in succession over a short period of time in which the focal length of the camera is varied so that each image has a different focal length. An image stack is created by aligning the images, and images are processed to determine which image has the highest contrast (i.e., is in focus) for each region of the image stack. Embodiments can further include displaying a first image of the series of images.

## Claims

### Claim 1

A method of image capture and processing, the method comprising:
capturing a plurality of images in succession over a period of time with a camera, wherein the capturing is triggered by a user input;
varying a focal length of the camera during the period of time such that each image of the plurality of images is captured using a unique focal length;
aligning the plurality of images using motion sensor data taken during the period of time;
processing, with a processing unit, the plurality of images by:
dividing each image into a plurality of regions, each region having a corresponding region in each of the plurality of images;
calculating a level of contrast for each region of a first set of corresponding regions;
comparing the levels of contrast for regions of the first set of corresponding regions; and
determining a first image having a highest-level of contrast in the first set of corresponding regions; and
causing a display to show the first image when one of the first set of corresponding regions is selected.

### Claim 2 (depends on 1)

The method of claim 1, wherein one of the first set of corresponding regions is selected from an initial displayed image of the plurality of images.

### Claim 3 (depends on 1)

The method of claim 1, wherein the aligning occurs during the period of time in which the plurality of images is captured.

### Claim 4 (depends on 1)

The method of claim 1, wherein processing the plurality of images is performed by a mobile device.

### Claim 5 (depends on 1)

The method of claim 1, wherein processing the plurality of images is performed by a computing device other than a mobile device.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
receiving a selection, from the first image, of one of a second set of corresponding regions;
determining a second image, from the plurality of images, having a highest-level of contrast in the second set of corresponding regions; and
causing the display to show the second image.

### Claim 7 (depends on 1)

The method of claim 1, further comprising removing, from the plurality of images, one or more images in which none of the plurality of regions of the image is determined to be the highest-contrast region among the corresponding regions of the plurality of images.

### Claim 8 (depends on 1)

The method of claim 1, wherein aligning the plurality of images comprises at least one of:
scaling one or more images of the plurality of images,
shifting one or more images of the plurality of images,
rotating one or more images of the plurality of images, or
adjusting a brightness of one or more images of the plurality of images.

### Claim 9 (depends on 1)

The method of claim 1, wherein the motion sensor data is obtained from at least one of:
a gyroscope,
an accelerometer, or
the camera.

### Claim 10

An apparatus comprising:
a memory;
a display; and
a processing unit communicatively coupled with the memory, wherein the processing unit is configured to perform functions including:
obtaining a plurality of images captured in succession over a period of time by a camera of a mobile device, wherein:
the capturing is triggered by a user input; and
a focal length of the camera is varied during the period of time such that each image of the plurality of images is captured using a unique focal length;
aligning the plurality of images using motion sensor data taken during the period of time;
processing the plurality of images by:
dividing each image into a plurality of regions, each region having a corresponding region in each of the plurality of images;
calculating a level of contrast for each region of a first set of corresponding regions;
comparing the levels of contrast for regions of the first set of corresponding regions; and
determining a first image having a highest-level of contrast in the first set of corresponding regions; and
causing the display to show the first image when one of the first set of corresponding regions is selected.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the processing unit is configured to perform the aligning during the period of time in which the plurality of images is captured.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein the apparatus comprises the mobile device.

### Claim 13 (depends on 10)

The apparatus of claim 10, wherein the apparatus comprises a computing device other than the mobile device.

### Claim 14 (depends on 10)

The apparatus of claim 10, wherein the processing unit is further configured to:
receive a selection, from the first image, of one of a second set of corresponding regions;
determine a second image, from the plurality of images, having a highest-level of contrast in the second set of corresponding regions; and
cause the display to show the second image.

### Claim 15 (depends on 10)

The apparatus of claim 10, wherein the processing unit is further configured to remove, from the plurality of images, one or more images in which none of the plurality of regions of the image is determined to be the highest-contrast region among the corresponding regions of the plurality of images.

### Claim 16 (depends on 10)

The apparatus of claim 10, the processing unit is further configured to align the plurality of images by:
scaling one or more images of the plurality of images,
shifting one or more images of the plurality of images,
rotating one or more images of the plurality of images, or
adjusting a brightness of one or more images of the plurality of images.

### Claim 17

A non-transitory computer-readable storage medium having instructions embedded thereon for providing refocusable images, the instructions comprising computer code for:
obtaining a plurality of images in succession over a period of time with a camera of a mobile device, wherein the capturing is triggered by a user input;
causing the camera to vary a focal length during the period of time such that each image of the plurality of images is captured using a unique focal length;
aligning the plurality of images using motion sensor data taken during the period of time;
processing the plurality of images by:
dividing each image into a plurality of regions, each region having a corresponding region in each of the plurality of images;
calculating a level of contrast for each region of a first set of corresponding regions;
comparing the levels of contrast for regions of the first set of corresponding regions; and
determining a first image having a highest-level of contrast of the first set in corresponding regions; and
causing a display to show the first image when one of the first set of corresponding regions is selected.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the code for aligning is configured to cause the aligning to occur during the period of time in which the plurality of images is captured.

### Claim 19 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, further comprising code for:
receiving a selection, from the first image, of one of a second set of corresponding regions;
determining a second image, from the plurality of images, having a highest-level of contrast in the second set of corresponding regions; and
causing the display to show the second image.

### Claim 20 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, further comprising code for removing, from the plurality of images, one or more images in which none of the plurality of regions of the image is determined to be the highest-contrast region among the corresponding regions of the plurality of images.

### Claim 21 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the code for aligning the plurality of images comprises code for performing at least one of:
scaling one or more images of the plurality of images,
shifting one or more images of the plurality of images,
rotating one or more images of the plurality of images, or
adjusting a brightness of one or more images of the plurality of images.

### Claim 22

A device comprising:
means for obtaining a plurality of images in succession over a period of time with a camera of a mobile device, wherein the capturing is triggered by a user input;
means for causing the camera to vary a focal length during the period of time such that each image of the plurality of images is captured using a unique focal length;
means for aligning the plurality of images using motion sensor data taken during the period of time;
means for processing the plurality of images by:
dividing each image into a plurality of regions, each region having a corresponding region in each of the plurality of images;
calculating a level of contrast for each region of a first set of corresponding regions;
comparing the levels of contrast for regions of the first set of corresponding regions; and
determining a first image having a highest-level of contrast of the first set in corresponding regions; and
means for causing a display to show the first image when one of the first set of corresponding regions is selected.

### Claim 23 (depends on 22)

The device of claim 22, wherein the means for aligning is configured to cause the aligning to occur during the period of time in which the plurality of images is captured.

### Claim 24 (depends on 22)

The device of claim 22, further comprising means for:
receiving a selection, from the first image, of one of a second set of corresponding regions;
determining a second image, from the plurality of images, having a highest-level of contrast in the second set of corresponding regions; and
causing the display to show the second image.

### Claim 25 (depends on 22)

The device of claim 22, further comprising means for removing, from the plurality of images, one or more images in which none of the plurality of regions of the image is determined to be the highest-contrast region among the corresponding regions of the plurality of images.

### Claim 26 (depends on 22)

The device of claim 22, wherein the means for aligning the plurality of images comprises at least one of:
means for scaling one or more images of the plurality of images,
means for shifting one or more images of the plurality of images,
means for rotating one or more images of the plurality of images, or
means for adjusting a brightness of one or more images of the plurality of images.
