# US 2019/0379848 A1 · Image sensor post processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0379848 A1
- Document ID: US20190379848A1
- Status: Published application
- Kind: A1
- Application number: 16/432,072
- Filed: 2019-06-05
- Published: 2019-12-12
- Earliest claimed: 2018-06-08
- Assignees: Facebook Technologies, LLC
- Applicants: Facebook Technologies, LLC
- Inventors: Andrew Samuel BERKOVICH; Wei GAO; Xinqiao LIU; Song CHEN
- Web page: https://patentgrep.com/patent/US20190379848A1

## CPC

- H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
- H04N 25/701: Line sensors
- H04N 23/73: by influencing the exposure time
- H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
- H04N 25/571: involving a non-linear response
- H04N 25/573: {the logarithmic type}
- H04N 25/587: acquired sequentially, e.g. using the combination of odd and even image fields
- H04N 25/59: by controlling the amount of charge storable in the pixel, e.g. modification of the charge conversion ratio of the floating node capacitance
- H04N 25/633: by using optical black pixels
- H04N 25/71: Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
- H04N 25/77: Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
- H04N 25/771: comprising storage means other than floating diffusion
- H04N 25/772: comprising A/D, V/T, V/F, I/T or I/F converters
- H04N 25/78: Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters
- H04N 25/532: by controlling global shutters in CMOS SSIS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In one example, an apparatus is provided. The apparatus comprises an image sensor configured to generate a first raw output to represent a first intensity of incident light based on a first relationship, and to generate a second raw output to represent a second intensity of incident light based on a second relationship. The apparatus further comprises a post processor configured to: generate a first post-processed output based on the first raw output and based on the first relationship such that the first post-processed output is linearly related to the first intensity based on a third relationship, and to generate a second post-processed output based on the second raw output and based on the second relationship such that the second post-processed output is linearly related to the second intensity based on the third relationship.

## Claims

### Claim 1

An apparatus comprising:
an image sensor configured to:
generate a first raw output to represent a first intensity of incident light based on a first relationship, and
generate a second raw output to represent a second intensity of incident light based on a second relationship; and
a post processor configured to:
generate a first post-processed output based on the first raw output and based on the first relationship such that the first post-processed output is linearly related to the first intensity based on a third relationship; and
generate a second post-processed output based on the second raw output and based on the second relationship such that the second post-processed output is linearly related to the second intensity based on the third relationship.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein:
the first relationship between the first raw output and the first intensity is associated with a first intensity range;
the second relationship between the second raw output and the second intensity is associated with a second intensity range; and
the third relationship is associated with a third intensity range comprising the first intensity range and the second intensity range.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the image sensor comprises:
a photodiode configured to:
generate charge in response to incident light within an integration period, and
accumulate at least a part of the charge as residual charge until the photodiode saturates; and
a charge storage unit configured to accumulate the remaining charge as overflow charge until the charge storage unit saturates.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein the third relationship linearly relates a post-processed output with an estimated quantity of charge generated by the photodiode within the integration period.

### Claim 5 (depends on 4)

The apparatus of claim 4,
wherein the first raw output measures a saturation time when the charge storage unit saturates before the integration period ends; and
wherein the second raw output measures a quantity of the overflow charge accumulated by the charge storage unit within the integration period.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the post processor is configured to:
determine, based on the saturation time, a rate of accumulation of the overflow charge by the charge storage unit;
determine, based on the rate of accumulation of the overflow charge and the integration period, the estimated quantity of charge generated by the photodiode within the integration period; and
determine the first post-processed output based on the estimated quantity of charge.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein the post processor is configured to determine the estimated quantity of charge based on a first quantity of dark current received by the charge storage unit and a second quantity of dark current received by the photodiode within the integration period.

### Claim 8 (depends on 6)

The apparatus of claim 6, wherein the post processor is configured to determine the rate of accumulation of the overflow charge based on a first charge storage capacity of the charge storage unit and a second charge storage capacity of the photodiode.

### Claim 9 (depends on 4)

The apparatus of claim 4, wherein the post processor is configured to:
determine the estimated quantity of charge based on the second raw output and a charge storage capacity of the photodiode; and
determine the second post-processed output based on the estimated quantity of charge.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the post processor is configured to determine the estimated quantity of charge based on a first quantity of dark current received by the charge storage unit and a second quantity of dark current received by the photodiode within the integration period.

### Claim 11 (depends on 3)

The apparatus of claim 3,
wherein the first raw output measures a quantity of the overflow charge accumulated by the charge storage unit within the integration period; and
wherein the second raw output measures a quantity of the residual charge accumulated by the photodiode within the integration period.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the image sensor is configured to generate a third raw output to represent a third intensity of incident light, the third raw output measuring a saturation time when the charge storage unit saturates and is related to the third intensity of the incident light based on a fourth relationship; and
wherein the post processor is configured to generate a third post-processed output based on the third raw output and based on the fourth relationship such that the third post-processed output is linearly related to the third intensity based on the third relationship.

### Claim 13 (depends on 11)

The apparatus of claim 11, wherein the charge storage unit has a configurable charge storage capacity;
wherein the image sensor is configured to store the overflow charge at the charge storage unit having a first charge storage capacity to generate the first raw output;
wherein the image sensor is configured to transfer the residual charge from the photodiode to the charge storage unit having a second charge storage capacity to generate the second raw output; and
wherein the first post-processed output is generated based on the first charge storage capacity and the second charge storage capacity.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the first post-processed output is generated based on a first quantity of dark current received by the charge storage unit and a second quantity of dark current received by the photodiode within the integration period; and
wherein the second post-processed output is generated based on the second quantity of dark current received by the photodiode within the integration period.

### Claim 15 (depends on 1)

The apparatus of claim 1, wherein the post processor comprises one or more lookup tables that maps the first raw output to the first post-processed output and maps the second raw output to the second post-processed output; and
wherein the post processor is configured to generate the first post-processed output and the second post-processed output based on the one or more lookup tables.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the one or more lookup tables comprises a first lookup table that maps the first raw output to the first post-processed output and a second lookup table that maps the second raw output to the second post-processed output; and
wherein the post processor is configured to select the first lookup table based on the first intensity and to select the second lookup table based on the second intensity.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the first raw output and the second raw output are generated based on a non-uniform quantization process.

### Claim 18

A method, comprising:
generating, by an image sensor, a first raw output to represent a first intensity of incident light based on a first relationship;
generating, by the image sensor, a second raw output to represent a second intensity of the incident light based on a second relationship;
generating, by a post processor, a first post-processed output based on the first raw output and based on the first relationship such that the first post-processed output is linearly related to the first intensity based on a third relationship; and
generating, by the post processor, a second post-processed output based on the second raw output and based on the second relationship such that the second post-processed output is linearly related to the second intensity based on the third relationship.

### Claim 19 (depends on 18)

The method of claim 18, further comprising:
generating, by the image sensor, a third raw output to represent a third intensity of incident light, wherein the third raw output measures a saturation time when a charge storage unit of the image sensor saturates and is related to the third intensity of the incident light based on a fourth relationship; and
generating, by the post processor, a third post-processed output based on the third raw output and based on the fourth relationship such that the third post-processed output is linearly related to the third intensity based on the third relationship.

### Claim 20

A non-transitory computer readable medium storing instructions that, when executed by a hardware processor, causes the hardware processor to:
receive, from an image sensor, a first raw output to represent a first intensity of incident light based on a first relationship;
receive, from the image sensor, a second raw output to represent a second intensity of the incident light based on a second relationship;
generate a first post-processed output based on the first raw output and based on the first relationship such that the first post-processed output is linearly related to the first intensity based on a third relationship; and
generate a second post-processed output based on the second raw output and based on the second relationship such that the second post-processed output is linearly related to the second intensity based on the third relationship
