# US 2020/0396369 A1 · Image capturing apparatus, image capturing method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0396369 A1
- Document ID: US20200396369A1
- Status: Published application
- Kind: A1
- Application number: 17/006,901
- Filed: 2020-08-31
- Published: 2020-12-17
- Earliest claimed: 2018-03-29
- Assignees: FUJIFILM Corporation
- Applicants: FUJIFILM Corporation
- Inventors: Shinya FUJIWARA
- Web page: https://patentgrep.com/patent/US20200396369A1

## CPC

- H04N 23/73: by influencing the exposure time
- G03B 7/093: for control of exposure time
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/92: based on global image properties
- H04N 23/60: Control of cameras or camera modules
- H04N 23/71: Circuitry for evaluating the brightness variation
- H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
- H04N 23/76: by influencing the image signals
- G06T 2207/20208: High dynamic range [HDR] image processing
- G06T 2207/20216: Image averaging
- G06T 2207/20221: Image fusion; Image merging
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image capturing apparatus including: an imaging sensor and a processor configured to set a first exposure time and a second exposure time obtained by dividing the first exposure time by an integer m where m is an integer equal to or larger than 2; control the imaging sensor to capture frames successively by repeatedly performing exposure including at least exposure for one frame having the first exposure time and exposure for m frames having the second exposure time as one set; acquire image data of each frame from the imaging sensor; generate image data of an average frame that is an average of pieces of image data of the m frames having the second exposure time in the one set; and generate a composite frame for recording or displaying a motion picture of an extended dynamic range according to image data of the one frame having the first exposure time and the image data of the average frame.

## Claims

### Claim 1

An image capturing apparatus comprising:
an imaging sensor; and
a processor configured to
set a first exposure time and a second exposure time shorter than the first exposure time, the second exposure time being an exposure time obtained by dividing the first exposure time by an integer m where m is an integer equal to or larger than 2;
control the imaging sensor to capture frames successively by repeatedly performing exposure including at least exposure for one frame having the first exposure time and exposure for m frames having the second exposure time as one set;
acquire image data of each frame from the imaging sensor;
generate image data of an average frame that is an average of pieces of image data of the m frames having the second exposure time in the one set; and
generate a composite frame for recording or displaying a motion picture of an extended dynamic range according to image data of the one frame having the first exposure time and the image data of the average frame,
wherein the processor is further configured to calculate a proportion of blown-out highlight pixels in at least one of the image data of the one frame having the first exposure time or image data of one frame having the second exposure time, and
make an increase change or a decrease change to the integer m in a next set on the basis of the proportion of blown-out highlight pixels.

### Claim 2 (depends on 1)

The image capturing apparatus according to claim 1, wherein
the processor is further configured to make an increase change to the integer m in the next set in a case where the proportion of blown-out highlight pixels is equal to or larger than a predetermined threshold value.

### Claim 3 (depends on 1)

The image capturing apparatus according to claim 1, wherein
the processor is further configured to make a decrease change to the integer m in the next set in a case where the proportion of blown-out highlight pixels is equal to or smaller than a predetermined threshold value

### Claim 4 (depends on 2)

The image capturing apparatus according to claim 2, wherein
the processor is further configured to make a decrease change to the integer m in the next set in a case where the proportion of blown-out highlight pixels is equal to or smaller than a predetermined threshold value

### Claim 5 (depends on 1)

The image capturing apparatus according to claim 1, wherein
the processor is further configured to make the increase change in a case where the proportion of blown-out highlight pixels is equal to or larger than a first threshold value and sets the second exposure time to an exposure time obtained by dividing the first exposure time by an integer m+1 in the next set.

### Claim 6 (depends on 1)

The image capturing apparatus according to claim 1, wherein
the processor is further configured to
make the decrease change in a case where the proportion of blown-out highlight pixels is equal to or smaller than a second threshold value; and
set the second exposure time to an exposure time obtained by dividing the first exposure time by an integer m−1 in the next set.

### Claim 7 (depends on 2)

The image capturing apparatus according to claim 2, wherein
the processor is further configured to
make the decrease change in a case where the proportion of blown-out highlight pixels is equal to or smaller than a second threshold value; and
set the second exposure time to an exposure time obtained by dividing the first exposure time by an integer m−1 in the next set.

### Claim 8 (depends on 3)

The image capturing apparatus according to claim 3, wherein
the processor is further configured to
make the decrease change in a case where the proportion of blown-out highlight pixels is equal to or smaller than a second threshold value; and
set the second exposure time to an exposure time obtained by dividing the first exposure time by an integer m−1 in the next set.

### Claim 9 (depends on 4)

The image capturing apparatus according to claim 4, wherein
the processor is further configured to
make the decrease change in a case where the proportion of blown-out highlight pixels is equal to or smaller than a second threshold value; and
set the second exposure time to an exposure time obtained by dividing the first exposure time by an integer m−1 in the next set.

### Claim 10 (depends on 5)

The image capturing apparatus according to claim 5, wherein
the processor is further configured to
make the decrease change in a case where the proportion of blown-out highlight pixels is equal to or smaller than a second threshold value; and
set the second exposure time to an exposure time obtained by dividing the first exposure time by an integer m−1 in the next set.

### Claim 11 (depends on 1)

The image capturing apparatus according to any one of claim 1, wherein
the processor is further configured to
apply a check image digital gain to image data of one frame having the second exposure time before the decrease change is made to generate blown-out highlight check image data;
calculate a proportion of blown-out highlight pixels in the blown-out highlight check image data, and
make the decrease change in a case where the proportion of blown-out highlight pixels in the blown-out highlight check image data is equal to or smaller than a third threshold value.

### Claim 12 (depends on 8)

The image capturing apparatus according to claim 8, wherein
the processor is further configured to
apply the check image digital gain of m/(m−1)-fold to the image data of the one frame having the second exposure time to generate the blown-out highlight check image data, and
set the second exposure time in the next set to an exposure time obtained by dividing the first exposure time by an integer m−1 in a case where the proportion of blown-out highlight pixels in the blown-out highlight check image data is equal to or smaller than the third threshold value.

### Claim 13 (depends on 1)

The image capturing apparatus according to claim 1, wherein
the processor is further configured to generate the first gain-up image data in which a first digital gain is decreased in a stepwise manner in accordance with a frame rate for recording or displaying the motion picture from the image data of the average frame after the increase change has been made; and
generate the composite frame according to the first gain-up image data.

### Claim 14 (depends on 13)

The image capturing apparatus according to claim 13, wherein
the first gain-up image data generation unit decreases the first digital gain from m_b/m_a-fold to 1-fold in a stepwise manner in a case of the increase change from an integer m_a to an integer m_b.

### Claim 15 (depends on 1)

The image capturing apparatus according to claim 1, wherein
the processor is further configured to
generate second gain-up image data in which a second digital gain is increased in a stepwise manner in accordance with a frame rate for recording or displaying the motion picture from the image data of the average frame before the decrease change is made, and
generate the composite frame according to the second gain-up image data.

### Claim 16 (depends on 15)

The image capturing apparatus according to claim 15, wherein
the processor is further configured to increase the second digital gain from 1-fold to m_c/m_d-fold in a stepwise manner in a case of the decrease change from an integer m_c to an integer m_d.

### Claim 17

An image capturing method comprising:
setting a first exposure time and a second exposure time shorter than the first exposure time, the second exposure time being an exposure time obtained by dividing the first exposure time by an integer m where m is an integer equal to or larger than 2;
controlling an imaging sensor to capture frames successively by repeatedly performing exposure including at least exposure for one frame having the first exposure time and exposure for m frames having the second exposure time as one set;
acquiring image data of each frame from the imaging sensor;
generating image data of an average frame that is an average of pieces of image data of the m frames having the second exposure time in the one set;
generating a composite frame for recording or displaying a motion picture of an extended dynamic range according to image data of the one frame having the first exposure time and the image data of the average frame;
calculating a proportion of blown-out highlight pixels in at least one of the image data of the one frame having the first exposure time or image data of one frame having the second exposure time; and
making an increase change or a decrease change to the integer m in a next set on the basis of the proportion of blown-out highlight pixels.

### Claim 18

A non-transitory computer readable medium for storing a program causing a computer to perform an image capturing method comprising:
setting a first exposure time and a second exposure time shorter than the first exposure time, the second exposure time being an exposure time obtained by dividing the first exposure time by an integer m where m is an integer equal to or larger than 2;
controlling an imaging sensor to capture frames successively by repeatedly performing exposure including at least exposure for one frame having the first exposure time and exposure for m frames having the second exposure time as one set to control exposure;
acquiring image data of each frame from the imaging unit;
generating image data of an average frame that is an average of pieces of image data of the m frames having the second exposure time in the one set;
generating a composite frame for recording or displaying a motion picture of an extended dynamic range according to image data of the one frame having the first exposure time and the image data of the average frame;
calculating a proportion of blown-out highlight pixels in at least one of the image data of the one frame having the first exposure time or image data of one frame having the second exposure time; and
making an increase change or a decrease change to the integer m in a next set on the basis of the proportion of blown-out highlight pixels.
