# US 10,455,220 B2 · Image processing device, method of controlling image processing device and program causing computer to execute method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,455,220 B2
- Document ID: US10455220B2
- Status: Granted
- Kind: B2
- Application number: 15/425,563
- Filed: 2017-02-06
- Published: 2017-05-25
- Granted: 2019-10-22
- Earliest claimed: 2011-08-24
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: Naho Suzuki; Hideki Nabesako; Takami Mizukura
- Web page: https://patentgrep.com/patent/US10455220B2

## CPC

- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- G06T 5/70: Denoising; Smoothing
- G06T 5/94: based on local image properties, e.g. for local contrast enhancement
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

There is provided an image processing device including an image acquisition part acquiring an image; a depth acquisition part acquiring a depth associated with a pixel in the image; a depth conversion part converting the depth in accordance with a function having a characteristic to nonlinearly approach a predetermined value with an increase in the depth; and a storage part storing the converted depth in association with the image.

## Claims

### Claim 1

An image processing device comprising:
a memory; and
circuitry, coupled with the memory, configured to
set an initial value of a coefficient of a function used to convert depth associated with a pixel within image data to have a nonlinear characteristic with an increase in the depth,
after setting the initial value, determine whether a shooting mode from among a plurality of shooting modes is a landscape shooting mode or a macro shooting mode,
modify the initial value of the coefficient to a second value that is smaller than the initial value, in response to determining the shooting mode is the landscape shooting mode,
modify the initial value of the coefficient to a third value that is greater than the initial value, in response to determining the shooting mode is the macro shooting mode, and
convert the depth associated with the pixel within the image data to have the nonlinear characteristic with the increase in the depth, based on one of the initial value, the second value, and the third value, the nonlinear characteristic being different between the plurality of shooting modes, and different shooting modes having different converted depths.

### Claim 2 (depends on 1)

The image processing device according to claim 1, wherein the circuitry is further configured to perform smoothing processing on the image data to a degree depending on the converted depth corresponding to the pixel in the image data based on a converted depth corresponding to a predetermined pixel in the image data.

### Claim 3 (depends on 2)

The image processing device according to claim 2, wherein the circuitry
converts the depth associated with the pixel within the image data in accordance with the function to nonlinearly approach a predetermined value with the increase in the depth, the function being a function with a characteristic that varies depending on the coefficient, and
performs the smoothing processing based on the depth converted in accordance with the characteristic.

### Claim 4 (depends on 3)

The image processing device according to claim 3, wherein the function is an exponential function defined by the following formula:
Y=α×e{circumflex over ( )}(−βx),
where x is the depth, y is an output, e is a base of natural logarithm, α is a predetermined constant and β is the coefficient.

### Claim 5 (depends on 3)

The image processing device according to claim 3, wherein the function is an exponential function defined by the following formula:
Y=α×{1−e{circumflex over ( )}(−βx)},
where x is the depth, y is an output, e is a base of natural logarithm, α is a predetermined constant and β is the coefficient.

### Claim 6 (depends on 1)

The image processing device according to claim 1, wherein the circuitry is further configured to create an aggregation of pixels as a depth image, each pixel having the converted depth value as a pixel value.

### Claim 7 (depends on 6)

The image processing device according to claim 6, wherein
the circuitry is further configured to compress the depth image in accordance with a predetermined image compression format, and
the circuitry is configured to store the compressed depth image in association with the image data.

### Claim 8

A method of controlling an image processing device, the method comprising:
setting an initial value of a coefficient of a function used to convert depth associated with a pixel within image data to have a nonlinear characteristic with an increase in the depth;
after setting the initial value, determining whether a shooting mode from among a plurality of shooting modes is a landscape shooting mode or a macro shooting mode;
modifying the initial value of the coefficient to a second value that is smaller than the initial value, in response to determining the shooting mode is the landscape shooting mode;
modifying the initial value of the coefficient to a third value that is greater than the initial value, in response to determining the shooting mode is the macro shooting mode; and
converting the depth associated with the pixel within the image data to have the nonlinear characteristic with the increase in the depth, based on one of the initial value, the second value, and the third value, the nonlinear characteristic being different between the plurality of shooting modes, and different shooting modes having different converted depths.

### Claim 9

A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method, the method comprising:
setting an initial value of a coefficient of a function used to convert depth associated with a pixel within image data to have a nonlinear characteristic with an increase in the depth;
after setting the initial value, determining whether a shooting mode from among a plurality of shooting modes is a landscape shooting mode or a macro shooting mode;
modifying the initial value of the coefficient to a second value that is smaller than the initial value, in response to determining the shooting mode is the landscape shooting mode;
modifying the initial value of the coefficient to a third value that is greater than the initial value, in response to determining the shooting mode is the macro shooting mode; and
converting the depth associated with the pixel within the image data to have the nonlinear characteristic with the increase in the depth, based on one of the initial value, the second value, and the third value, the nonlinear characteristic being different between the plurality of shooting modes, and different shooting modes having different converted depths.