# US 2019/0156516 A1 · Method and system of generating multi-exposure camera statistics for image processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0156516 A1
- Document ID: US20190156516A1
- Status: Published application
- Kind: A1
- Application number: 16/235,970
- Filed: 2018-12-28
- Published: 2019-05-23
- Earliest claimed: 2018-12-28
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Jarno Nikkanen; Rastislav Lukac
- Web page: https://patentgrep.com/patent/US20190156516A1

## CPC

- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/92: based on global image properties
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/90: Determination of colour characteristics
- H04N 23/10: for generating image signals from different wavelengths
- H04N 23/73: by influencing the exposure time
- H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
- H04N 25/589: with different integration times, e.g. short and long exposures
- G06T 2207/20208: High dynamic range [HDR] image processing
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method, system, and article is directed to generating multi-exposure camera statistics for image processing by rearranging and sampling image data of multiple exposure images.

## Claims

### Claim 1

A computer-implemented method of image processing, comprising:
receiving image data of multiple image captures each resulting from a different exposure;
generating a single frame of image data to be input to generate statistics of the single frame, the generating comprising selecting image data of different pixel locations of the multiple image captures to form image data of pixel locations on the single frame depending on the exposure level of the image capture and depending on at least one of:
an assigned color of a pixel location of one of the image captures and assigned from a color filter array pattern, and
a position of a pixel location on one of the image captures;
generating the statistics of the single frame; and
modifying camera capture parameters by using the statistics to capture a subsequent frame or modifying image data of a frame by using the statistics.

### Claim 2 (depends on 1)

The method of claim 1 wherein the multiple image captures are of the same captured frame.

### Claim 3 (depends on 1)

The method of claim 1 wherein each image capture of the multiple image captures is of a separate input frame.

### Claim 4 (depends on 1)

The method of claim 1 wherein the image data of a pixel location on one of the multiple image captures is selected to form the image data of one pixel location of the single frame.

### Claim 5 (depends on 1)

The method of claim 1 wherein pixel locations of the image captures are selected without considering the image data values of the pixel location to make the selection.

### Claim 6 (depends on 1)

The method of claim 1 wherein one or more pixel locations are selected to be included in the single frame depending on whether the pixel location is on a maximum or minimum exposure image capture versus an intermediate exposure image capture.

### Claim 7 (depends on 6)

The method of claim 6 wherein only pixel locations of predefined colors are selected from at least one image capture.

### Claim 8 (depends on 6)

The method of claim 6 wherein pixel locations of two colors are selected from the intermediate image capture when three of the image captures are used, and wherein pixel locations of one color are selected from two intermediate image captures are being used, and when a three color filter array is used with pixel camera sensors used to obtain the image data.

### Claim 9 (depends on 1)

The method of claim 1 wherein blocks of input pixel image data from the image captures are not the same size as output blocks receiving the image data on the single frame.

### Claim 10 (depends on 1)

The method of claim 1 comprising determining input pixel locations of the image captures to be placed at output pixel locations of the single frame comprising determining a descriptor code pre-assigned to pixel location addresses within an output block of the single frame and that indicates which image capture and which position in an input block to use as the input pixel location for one of the pixel location addresses.

### Claim 11 (depends on 10)

The method of claim 10 wherein the descriptor codes are pre-assigned to the pixel location addresses depending on available color pixel locations from a color filter array pattern used to capture the image data.

### Claim 12

A computer-implemented system of image processing, comprising:
at least one image capture device to generate multiple image captures each resulting from a different exposure;
memory communicatively coupled to at least one processor and storing image data of the multiple image captures; and
at least one processor communicatively coupled to the at least one image capture device and memory, and the at least one processor to operate by:
generating a single frame of image data to be input to generate statistics of the single frame, the generating comprising selecting image data of different pixel locations of the multiple image captures to form image data of pixel locations on the single frame depending on the exposure level of the image capture and depending on at least one of:
an assigned color of a pixel location of one of the image captures and assigned from a color filter array pattern, and
a position of a pixel location on one of the image captures;
generating the statistics of the single frame; and
modifying camera capture parameters by using the statistics to capture a subsequent frame or modifying image data of a frame by using the statistics.

### Claim 13 (depends on 12)

The system of claim 12 comprising adding a digital signal processor (DSP) operating a mapper to generate the single frame and added upstream to relatively and generally downstream existing image signal processing hardware that generates statistics.

### Claim 14 (depends on 12)

The system of claim 12 comprising providing blocks of the single frame block by block to statistics generation hardware wherein the single frame blocks are not the same size as blocks received from the image captures block by block.

### Claim 15 (depends on 12)

The system of claim 12 wherein blocks of the image captures received to form a single frame overlap so that individual pixel locations at least initially may be placed in multiple output blocks of the single frame; and wherein the single frame is either (a) expanded relative to the size of the image captures or (b) formed by combining image data of pixel locations of the multiple blocks of the same pixel location on the multiple blocks.

### Claim 16 (depends on 12)

The system of claim 12 wherein the at least one processor rearranges pixel image data of a pixel location of one of the image captures according to a descriptor code and rearranged to a location on an output block of the single frame.

### Claim 17 (depends on 16)

The system of claim 16 wherein the descriptor code indicates an exposure level of one of the image captures where the pixel image data was obtained as well as input block coordinates indicating where on an input block the pixel image data should be obtained.

### Claim 18 (depends on 16)

The system of claim 16 wherein the at least one processor operates by selecting a descriptor code comprising inputting image-wide output pixel coordinates of the single frame to an index formula that outputs a pixel location address within an output block, and wherein descriptor codes are individually assigned to the pixel location addresses.

### Claim 19 (depends on 18)

The system of claim 18 wherein the formula is adaptable to support varying sizes of output blocks of the single frame.

### Claim 20

At least one non-transitory article having a computer readable medium comprising a plurality of instructions that in response to being executed on a computing device, cause the computing device to operate by:
receiving image data of multiple image captures each resulting from a different exposure;
generating a single frame of image data to be input to generate statistics of the single frame, the generating comprising selecting image data of different pixel locations of the multiple image captures to form image data of pixel locations on the single frame depending on the exposure level of the image capture and depending on at least one of:
an assigned color of a pixel location of one of the image captures and assigned from a color filter array pattern, and
a position of a pixel location on one of the image captures;
generating the statistics of the single frame; and
modifying camera capture parameters by using the statistics to capture a subsequent frame or modifying image data of a frame by using the statistics.

### Claim 21 (depends on 20)

The article of claim 20 wherein the image data is selected without considering the values of the image data.

### Claim 22 (depends on 20)

The article of claim 20 wherein the instructions cause the computing device to determine a descriptor code assigned to an output pixel location address within an output block; and use the descriptor to determine an input image-wide pixel location corresponding to an image-wide output pixel location of the same frame used to determine the output pixel location address.

### Claim 23 (depends on 20)

The article of claim 20 wherein the instructions cause the computing device to operate by combining, or selecting among, image values of two or more pixel locations of the same color of at least one of the image captures to form a single pixel location on an output block of the single frame.

### Claim 24 (depends on 23)

The article of claim 23 wherein the combining, or selecting among, image values comprises at least one of: averaging the image values, selecting a minimum or maximum image value, and using a median value among the image values.

### Claim 25 (depends on 20)

The article of claim 20 wherein the image data is selected output block by output block of the single frame, and wherein an output block window used to select corresponding input pixel locations of the image captures are overlapped.
