# US 2019/0172423 A1 · Image processing apparatus and image processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0172423 A1
- Document ID: US20190172423A1
- Status: Published application
- Kind: A1
- Application number: 16/321,958
- Filed: 2017-08-10
- Published: 2019-06-06
- Earliest claimed: 2016-08-24
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: Masuyoshi KUROKAWA
- Web page: https://patentgrep.com/patent/US20190172423A1

## CPC

- G09G 5/36: characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
- G09G 5/00: Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators (image data processing or generation, in general G06T)
- H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)}
- H04N 9/3188: {Scale or resolution adjustment (scaling in general G06T3/40; resolution modifying circuits for displays in general G09G5/391)}
- H04N 9/3194: {including sensor feedback}
- G09G 2340/0407: Resolution change, inclusive of the use of different resolutions for different screen areas
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present disclosure relates to an image processing apparatus and an image processing method capable of realizing grasping of a state of a projected image at a lower cost. A difference between discontinuous frames of a captured image configured with a plurality of frames obtained by capturing a projected image which is obtained by sequentially projecting predetermined pattern images opposite in sign, which are positive and negative pattern images, at a higher frame rate than a frame rate of the projected image by a rolling shutter scheme is calculated. The present disclosure can be applied to, for example, an image processing apparatus, a projection apparatus, an imaging apparatus, a projection imaging apparatus, a controller, a projection imaging system, or the like.

## Claims

### Claim 1

An image processing apparatus comprising:
a difference calculation section calculating a difference between discontinuous frames of a captured image configured with a plurality of frames obtained by capturing a projected image which is obtained by sequentially projecting predetermined pattern images opposite in sign, which are positive and negative pattern images, by a projection section at a higher frame rate than a frame rate of the projected image by an imaging section driven by a rolling shutter scheme.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1, wherein
the difference calculation section is configured to calculate a difference between the frames at an interval in accordance with a ratio of the frame rate of the projected image to the frame rate of the captured image.

### Claim 3 (depends on 2)

The image processing apparatus according to claim 2, wherein
the frame rate of the captured image is twice as high as the frame rate of the projected image, and
the difference calculation section calculates a difference between the frames at an interval of two frames.

### Claim 4 (depends on 3)

The image processing apparatus according to claim 3, further comprising:
an extraction section extracting a region where the pattern images are detectable from the difference between the frames calculated by the difference calculation section; and
a joint section joining the region extracted from each of a plurality of differences by the extraction section together to generate the pattern image corresponding to one frame.

### Claim 5 (depends on 4)

The image processing apparatus according to claim 4, further comprising:
a correspondence point detection section detecting correspondence points between the projection section and the imaging section using the pattern image generated by the joint section.

### Claim 6 (depends on 5)

The image processing apparatus according to claim 5, further comprising:
an image correction section correcting an image projected by the projection section using the correspondence points detected by the correspondence point detection section.

### Claim 7 (depends on 4)

The image processing apparatus according to claim 4, wherein
the pattern images each include patterns available for detecting correspondence points between the projection section and the imaging section and patterns available for identifying a pixel row of the projected image, and
the extraction section is configured to extract the regions where the pattern images are detectable on the basis of the patterns available for identifying the pixel row of the projected image.

### Claim 8 (depends on 7)

The image processing apparatus according to claim 7, wherein
the pattern images each further include patterns available for identifying a pixel column of the projected image.

### Claim 9 (depends on 8)

The image processing apparatus according to claim 8, wherein
the patterns available for identifying the pixel row of the projected image and the patterns available for identifying the pixel column of the projected image in each of the pattern images are formed near left, right, upper, and lower ends of the projected image.

### Claim 10 (depends on 8)

The image processing apparatus according to claim 8, wherein
the patterns available for identifying the pixel row of the projected image and the patterns available for identifying the pixel column of the projected image in the pattern image are formed in a grid form on entirety of the projected image.

### Claim 11 (depends on 4)

The image processing apparatus according to claim 4, wherein
the pattern images each include patterns available for both detecting correspondence points between the projection section and the imaging section and identifying a pixel row of the projected image, and
the extraction section is configured to extract the regions where the pattern images are detectable on the basis of the patterns available for both detecting the correspondence points between the projection section and the imaging section and identifying the pixel row of the projected image.

### Claim 12 (depends on 2)

The image processing apparatus according to claim 2, further comprising:
a synchronous processing section synchronizing timing of projection by the projection section with timing of imaging by the imaging section.

### Claim 13 (depends on 2)

The image processing apparatus according to claim 2, wherein
the frame rate of the captured image is 2N (where N is a natural number) times as high as the frame rate of the projected image, and
the difference calculation section is configured to calculate a difference between the frames at an interval of 2N frames.

### Claim 14 (depends on 13)

The image processing apparatus according to claim 13, further comprising:
an image selection section selecting frames for which the difference calculation section calculates the difference from the captured image of the plurality of frames.

### Claim 15 (depends on 14)

The image processing apparatus according to claim 14, wherein
the image selection section selects frames in each of which a plurality of frames of the projected image are mixed, in the captured image.

### Claim 16 (depends on 14)

The image processing apparatus according to claim 14, wherein
the image selection section selects frames in each of which only a single frame of the projected image is included, in the captured image.

### Claim 17 (depends on 2)

The image processing apparatus according to claim 2, wherein
the projected image is obtained by sequentially projecting a superimposed image on which the pattern image that is positive is superimposed, and a superimposed image on which the pattern image that is negative is superimposed, onto predetermined frames of a moving image.

### Claim 18 (depends on 1)

The image processing apparatus according to claim 1, further comprising:
the imaging section.

### Claim 19 (depends on 1)

The image processing apparatus according to claim 1, further comprising:
the projection section.

### Claim 20

An image processing method, comprising:
calculating a difference between discontinuous frames of a captured image configured with a plurality of frames obtained by capturing a projected image which is obtained by sequentially projecting predetermined pattern images opposite in sign, which are positive and negative pattern images, at a higher frame rate than a frame rate of the projected image by a rolling shutter scheme.
