# US 2015/0104101 A1 · Method and ui for z depth image segmentation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0104101 A1
- Document ID: US20150104101A1
- Status: Published application
- Kind: A1
- Application number: 14/053,581
- Filed: 2013-10-14
- Published: 2015-04-16
- Earliest claimed: 2013-10-14
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Andrew E. Bryant; Daniel Pettigrew
- Web page: https://patentgrep.com/patent/US20150104101A1

## CPC

- G06T 7/557: from light fields, e.g. from plenoptic cameras
- G06T 7/507: from shading (G06T7/586 takes precedence)
- G06T 2200/21: involving computational photography
- G06T 2207/10052: Images from lightfield camera
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An application that receives and edits image data from a light field camera. The application determines a distance from the light field camera for each portion of the image. The application of some embodiments uses the depth information to break the image data down into layers based on the depths of the objects in the image. In some embodiments, the layers are determined based on a histogram that plots the fraction of an image at a particular depth against the depths of the image.

## Claims

### Claim 1

A method of editing image data produced by a light field camera, the method comprising:
receiving the image data produced by the light field camera;
generating histogram data based on an amount of image data representing each of a plurality of depth levels in the image data;
determining a plurality of layers of the image data based on the generated histogram data;
providing a set of controls for the plurality of layers of the image data; and
generating an image based on the image data and settings of the set of controls.

### Claim 2 (depends on 1)

The method of claim 1, wherein the set of controls comprises a control for hiding at least one layer of the image data when generating the image.

### Claim 3 (depends on 2)

The method of claim 2, wherein generating the image comprises generating the image without displaying the hidden layer of the image data.

### Claim 4 (depends on 1)

The method of claim 1, wherein the set of controls comprises a control for each of a plurality of identified layers.

### Claim 5 (depends on 1)

The method of claim 1 further comprising displaying a graphical representation of the histogram data.

### Claim 6 (depends on 1)

The method of claim 1, wherein the image is comprised of pixels, the method further comprising:
receiving a selection of a location in the image, wherein the location identifies a portion of the image in a particular layer of the image; and
removing from the image a plurality of pixels that are (i) in the particular layer, and (ii) contiguously connected to the selected location by portions of the image in the particular layer.

### Claim 7 (depends on 6)

The method of claim 6, wherein the removing does not remove pixels that are not connected to the selected location by portions of the image in the particular layer.

### Claim 8 (depends on 6)

The method of claim 6, wherein receiving the selection of a location in the image comprises receiving a click on the location in the image with a cursor control device.

### Claim 9 (depends on 6)

The method of claim 6 further comprising receiving a selection of an object removal control before receiving the selection of the location in the image.

### Claim 10 (depends on 6)

The method of claim 6, wherein removing the plurality of pixels produces a void in the image smaller than the area of pixels removed.

### Claim 11

A method of editing image data produced by a light field camera, the method comprising:
receiving the image data produced by the light field camera;
identifying a depth for each of a plurality of portions of the image data;
determining a threshold depth; and
generating an image, based on the image data, with portions of the image that represent depths beyond the threshold depth obscured.

### Claim 12 (depends on 11)

The method of claim 11, wherein the obscured portions of the image are grayed out.

### Claim 13 (depends on 12)

The method of claim 12, wherein non-obscured portions of the image are displayed without being grayed out.

### Claim 14 (depends on 11)

The method of claim 11 further comprising determining an initial threshold depth based on a set of histogram data that relates a set of depths of the image to portions of the image at each particular depth of the set of depths.

### Claim 15 (depends on 11)

The method of claim 11 further comprising providing a control for setting the threshold depth, wherein determining the threshold depth comprises determining a setting of the control.

### Claim 16 (depends on 15)

The method of claim 15, wherein the control is a first control, the method further comprising providing a second control for setting a depth of focus for the image.

### Claim 17 (depends on 11)

The method of claim 11, wherein the obscured portions of the image are not displayed.

### Claim 18

A non-transitory machine readable medium storing a program which when executed by at least one processing unit edits image data produced by a light field camera, the program comprising sets of instructions for:
receiving the image data produced by the light field camera;
generating histogram data based on an amount of image data representing each of a plurality of depth levels in the image data;
determining a plurality of layers of the image data based on the generated histogram data;
providing a set of controls for the plurality of layers of the image data; and
generating an image based on the image data and settings of the set of controls.

### Claim 19 (depends on 18)

The non-transitory machine readable medium of claim 18, wherein the set of controls comprises a control for hiding at least one layer of the image data when generating the image.

### Claim 20 (depends on 19)

The non-transitory machine readable medium of claim 19, wherein the set of instructions for generating the image comprises a set of instructions for generating the image without displaying the hidden layer of the image data.

### Claim 21 (depends on 18)

The non-transitory machine readable medium of claim 18, wherein the set of controls comprises a control for each of a plurality of identified layers.

### Claim 22 (depends on 18)

The non-transitory machine readable medium of claim 18, wherein the program further comprises a set of instructions for displaying a graphical representation of the histogram data.

### Claim 23 (depends on 18)

The non-transitory machine readable medium of claim 18, wherein the image is comprised of pixels, wherein the program further comprises sets of instructions for:
receiving a selection of a location in the image, wherein the location identifies a portion of the image in a particular layer of the image; and
removing from the image a plurality of pixels that are (i) in the particular layer, and (ii) contiguously connected to the selected location by portions of the image in the particular layer.
