# US 2016/0261799 A1 · Image processing apparatus and method, and image capturing apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0261799 A1
- Document ID: US20160261799A1
- Status: Published application
- Kind: A1
- Application number: 15/059,617
- Filed: 2016-03-03
- Published: 2016-09-08
- Earliest claimed: 2015-03-04
- Assignees: Canon Kabushiki Kaisha
- Applicants: Canon Kabushiki Kaisha
- Inventors: Yohei Horikawa
- Web page: https://patentgrep.com/patent/US20160261799A1

## CPC

- H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory}
- H04N 23/687: {by shifting the lens or sensor position}
- H04N 23/959: by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing apparatus that corrects an image using an image signal obtained by performing shooting by an image sensor which includes a plurality of photoelectric conversion portions that correspond to each of a plurality of microlenses and performs photoelectric conversion on a light flux that enters via an imaging optical system including an image stabilization optical system configured to correct a camera shake, comprising: a first obtaining unit that obtains the image signal and a driving state of the image stabilization optical system at a time of image shooting; a second obtaining unit that obtains correction values for correcting variation in defocus amounts occurred due to driving of the image stabilization optical system; and a refocus unit that corrects the variation in defocus amounts by refocusing on the basis of the correction values.

## Claims

### Claim 1

An image processing apparatus that corrects an image using an image signal obtained by performing shooting by an image sensor which includes a plurality of photoelectric conversion portions that correspond to each of a plurality of microlenses and is configured to perform photoelectric conversion on a light flux that enters via an imaging optical system including an image stabilization optical system configured to correct a camera shake, comprising:
a first obtaining unit that obtains the image signal and a driving state of the image stabilization optical system at a time of image shooting;
a second obtaining unit that obtains correction values for correcting variation in defocus amounts due to shift of a focus plane at different image heights in the image sensor that occurred due to driving of the image stabilization optical system; and
a refocus unit that performs refocusing on the basis of the obtained image signal and generates the image,
wherein the refocus unit corrects the variation in defocus amounts by refocusing on the basis of the correction values.

### Claim 2 (depends on 1)

The image processing apparatus according to claim 1, wherein information of defocus amounts at different image height of the image sensor with respective to a plurality of driving states of the image stabilization optical system is stored in a storage unit, and
the second obtaining unit obtains the correction values on the basis of the defocus amounts read out from the storage unit.

### Claim 3 (depends on 2)

The image processing apparatus according to claim 2, wherein the information of the defocus amounts is defocus amounts of images at different image heights, wherein the images are obtained by shooting a reference image in a plurality of driving states of the image stabilization optical system.

### Claim 4 (depends on 2)

The image processing apparatus according to claim 2, wherein the information of the defocus amounts is an approximate equation for obtaining defocus amounts at different image height in a plurality of driving states of the image stabilization optical system.

### Claim 5 (depends on 2)

The image processing apparatus according to claim 2, wherein the information of the defocus amount is provided for each of a plurality of blocks obtained by dividing the image sensor into rectangles.

### Claim 6 (depends on 2)

The image processing apparatus according to claim 2, wherein the information of the defocus amount is provided for each of a plurality of blocks obtained by dividing the image sensor by angle and distance with a center of image height as a center position of polar coordinates.

### Claim 7 (depends on 1)

The image processing apparatus according to claim 1, wherein the second obtaining unit converts the variation in the defocus amounts to shift amounts for shifting a signal read out from the plurality of photoelectric conversion portions in order to correct the variation by the refocusing, and sets the shift amounts as the correction values.

### Claim 8 (depends on 7)

The image processing apparatus according to claim 7, wherein the second obtaining unit obtains, for the plurality of photoelectric conversion portions except for a reference photoelectric conversion portion, that is predetermined one of the plurality of photoelectric conversion portions, conversion coefficients for converting shift amounts of the signal read out from the plurality of photoelectric conversion portions except for the reference photoelectric conversion portion to the defocus amounts by the refocusing, and divides the defocus amounts by the conversion coefficients to obtain the shift amounts of the signal obtained from the plurality of photoelectric conversion portions except for the reference photoelectric conversion portion.

### Claim 9 (depends on 1)

The image processing apparatus according to claim 1, wherein the second obtaining unit changes, in a case where the defocus amounts are beyond a range in which refocusing is possible by the refocus unit, the defocus amounts to the upper limit of the range in which the refocusing is possible.

### Claim 10 (depends on 1)

The image processing apparatus according to claim 1, wherein the defocus amounts are defocus amounts at the different image heights with the center of the image height of the image sensor being focused.

### Claim 11

An image capturing apparatus comprising:
an image sensor; and
an image processing apparatus that corrects an image using an image signal obtained by performing shooting by an image sensor which includes a plurality of photoelectric conversion portions that correspond to each of a plurality of microlenses and is configured to perform photoelectric conversion on a light flux that enters via an imaging optical system including an image stabilization optical system configured to correct a camera shake, comprising:
a first obtaining unit that obtains the image signal and a driving state of the image stabilization optical system at a time of image shooting;
a second obtaining unit that obtains correction values for correcting variation in defocus amounts due to shift of a focus plane at different image heights in the image sensor that occurred due to driving of the image stabilization optical system; and
a refocus unit that performs refocusing on the basis of the obtained image signal and generates the image,
wherein the refocus unit corrects the variation in defocus amounts by refocusing on the basis of the correction values.

### Claim 12

An image processing method of correcting an image using an image signal obtained by performing shooting by an image sensor which includes a plurality of photoelectric conversion portions that correspond to each of a plurality of microlenses and is configured to perform photoelectric conversion on a light flux that enters via an imaging optical system including an image stabilization optical system configured to correct a camera shake, the method comprising:
obtaining the image signal and a driving state of the image stabilization optical system at a time of image shooting;
obtaining correction values for correcting variation in defocus amounts due to shift of a focus plane at different image heights in the image sensor that occurred due to driving of the image stabilization optical system; and
performing refocusing on the basis of the obtained image signal and generates the image,
wherein the variation in defocus amounts is corrected by the refocusing on the basis of the correction values.

### Claim 13

A non-transitory readable storage medium having stored thereon a program which is executable by a computer, the program having a program code for causing the computer to function as each unit of an image processing apparatus that corrects an image using an image signal obtained by performing shooting by an image sensor which includes a plurality of photoelectric conversion portions that correspond to each of a plurality of microlenses and is configured to perform photoelectric conversion on a light flux that enters via an imaging optical system including an image stabilization optical system configured to correct a camera shake, the apparatus comprising:
a first obtaining unit that obtains the image signal and a driving state of the image stabilization optical system at a time of image shooting;
a second obtaining unit that obtains correction values for correcting variation in defocus amounts due to shift of a focus plane at different image heights in the image sensor that occurred due to driving of the image stabilization optical system; and
a refocus unit that performs refocusing on the basis of the obtained image signal and generates the image,
wherein the refocus unit corrects the variation in defocus amounts by refocusing on the basis of the correction values.
