# US 2017/0251171 A1 · Image processing device, image display device, image processing method, and image display method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0251171 A1
- Document ID: US20170251171A1
- Status: Published application
- Kind: A1
- Application number: 15/521,761
- Filed: 2015-10-15
- Published: 2017-08-31
- Earliest claimed: 2014-11-06
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: HIDESHI YAMADA; TOMOO MITSUNAGA; KENSEI JO; TOHRU KURATA
- Web page: https://patentgrep.com/patent/US20170251171A1

## CPC

- H04N 7/08: Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band {, e.g. by time division (H04N7/007 takes precedence)}
- H04B 10/116: Visible light communication
- H04N 5/60: for the sound signals
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04B: TRANSMISSION

## Abstract

[Object] To provide an image processing device capable of outputting audio acquired as an audio data signal by capturing an arbitrary image. [Solution] Provided is an image processing device, including: a separation unit that separates a flicker pattern from an image obtained by capturing, at multiple exposure times, an image containing a scene that emits light on a basis of a flicker pattern converted from an audio signal; and a conversion unit that converts the separated flicker pattern to an original audio signal.

## Claims

### Claim 1

An image processing device, comprising:
a separation unit that separates a flicker pattern from an image obtained by capturing, at multiple exposure times, an image containing a scene that emits light on a basis of a flicker pattern converted from an audio signal; and
a conversion unit that converts the separated flicker pattern to an original audio signal.

### Claim 2 (depends on 1)

The image processing device according to claim 1, wherein
the audio signal that acts as a basis for the flicker pattern is a signal that is earlier than the image by a certain amount of time.

### Claim 3 (depends on 1)

The image processing device according to claim 1, wherein
the separation unit separates the flicker pattern from an image captured at a second exposure time that is shorter than a first exposure time.

### Claim 4 (depends on 1)

The image processing device according to claim 1, further comprising:
an imaging unit that captures the image containing the scene over at least two exposure times of a first exposure time, and a second exposure time that is shorter than the first exposure time.

### Claim 5 (depends on 4)

The image processing device according to claim 4, wherein
the imaging unit conducts the capture over the second exposure time at a higher frame rate than the capture over the first exposure time.

### Claim 6 (depends on 5)

The image processing device according to claim 5, wherein
the imaging unit conducts the capture over the second exposure time at a higher frame rate than the capture over the first exposure time by sampling pixels in a horizontal direction when conducting the capture over the second exposure time.

### Claim 7 (depends on 5)

The image processing device according to claim 5, wherein
the imaging unit conducts the capture over the second exposure time at a higher frame rate than the capture over the first exposure time by conducting pixel addition in a horizontal direction when conducting the capture over the second exposure time.

### Claim 8 (depends on 5)

The image processing device according to claim 5, wherein
the imaging unit conducts the capture over the second exposure time at a higher frame rate than the capture over the first exposure time only in a range of pixels containing the scene occupying a field of view of a captured image.

### Claim 9 (depends on 1)

The image processing device according to claim 1, further comprising:
a playback control unit that controls playback of audio based on the audio signal, on a basis of an image containing the scene occupying a field of view of a captured image.

### Claim 10 (depends on 9)

The image processing device according to claim 9, wherein
the playback control unit controls playback of audio based on the audio signal, on a basis of a size of the image containing the scene occupying the field of view.

### Claim 11 (depends on 10)

The image processing device according to claim 10, wherein
the playback control unit controls a volume of audio based on the audio signal, on a basis of a size of the image containing the scene occupying the field of view.

### Claim 12 (depends on 9)

The image processing device according to claim 9, wherein
the playback control unit controls a localization of audio based on the audio signal, on a basis of a position of the image containing the scene occupying the field of view.

### Claim 13 (depends on 1)

The image processing device according to claim 1, further comprising:
an orientation analysis unit that analyzes an orientation of an image containing the scene occupying a field of view of a captured image.

### Claim 14 (depends on 13)

The image processing device according to claim 13, wherein
the orientation analysis unit recognizes a certain marker included in the image containing the scene and analyzes an orientation.

### Claim 15 (depends on 13)

The image processing device according to claim 13, wherein
when the separation unit separates the flicker pattern, the orientation analysis unit outputs information specifying a range of the image containing the scene occupying the field of view of the captured image.

### Claim 16 (depends on 1)

The image processing device according to claim 1, wherein
the audio signal that the conversion unit converts is AM-modulated or FM-modulated.

### Claim 17 (depends on 1)

The image processing device according to claim 1, wherein
the audio signal that the conversion unit converts is coded by a certain coding scheme.

### Claim 18

An image display device, comprising:
a delay unit that delays an image signal by a certain delay time;
a flicker pattern conversion unit that converts, into a flicker pattern, an audio signal with attached information about the delay time by which the delay unit delays; and
an image display unit that displays an image based on an image signal delayed by the delay unit, while also causing a backlight to emit light on a basis of the flicker pattern converted by the flicker pattern conversion unit.

### Claim 19

An image processing method, comprising:
separating a flicker pattern from an image obtained by capturing, at multiple exposure times, an image containing a scene that emits light on a basis of a flicker pattern converted from an audio signal; and
converting the separated flicker pattern to an original audio signal.

### Claim 20

An image display method, comprising:
delaying an image signal by a certain delay time;
converting, into a flicker pattern, an audio signal with attached information about the delay time by which to delay; and
displaying an image based on the delayed image signal, while also causing a backlight to emit light on a basis of the converted flicker pattern.
