# US 9,436,983 B2 · Systems and methods for non-linear processing of image frames

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,436,983 B2
- Document ID: US9436983B2
- Status: Granted
- Kind: B2
- Application number: 14/304,773
- Filed: 2014-06-13
- Published: 2014-12-18
- Granted: 2016-09-06
- Earliest claimed: 2013-06-14
- Assignees: nLightn, Inc.
- Applicants: nLightn, Inc.
- Inventors: Mikael Nilsson; To Tran; Börje Anders Hellsberg; James Frank Thomas
- Web page: https://patentgrep.com/patent/US9436983B2

## CPC

- G06T 5/40: using histogram techniques
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/90: Determination of colour characteristics
- G06T 2207/10024: Color image
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Embodiments of the present disclosure include systems and methods for image processing for enhancing, e.g., photos, videos, and graphics representations. Particularly, some embodiments perform a histogram-based SMQT decomposition of various attributes in an original image. The SMQT decomposition of these attributes may then be used to create a look-up-table indicating how to modify pixel attributes in the same or a subsequent image so as to enhance the same or subsequent image. The results of the SMQT decomposition for each attribute may be interpolated based, e.g., on user preferences, to achieve a more optimal enhancement.

## Claims

### Claim 1

A computer-implemented method comprising:
obtaining a first image;
generating a first histogram for a first attribute associated with pixels in the first image;
generating a second histogram for a second attribute associated with pixels in the first image;
generating a first Successive Mean Quantization Transform (SMQT) output using the first histogram; and
generating a second SMQT output using the second histogram; and
generating an interpolation look-up-table based at least in part upon the first SMQT output, the second SMQT output, and an interpolation parameter; and
adjusting a pixel in a second image using the interpolation look-up-table.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein generating a first SMQT output comprises generating an SMQT table for a first attribute.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the first attribute is one of a red, green, or blue pixel value attribute and the second attribute is a luminance value attribute.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the first attribute comprises a red pixel value attribute and the second attribute comprises a luminance pixel value attribute, the method further comprising:
generating a third histogram for a green pixel value attribute;
generating a fourth histogram for a blue pixel value attribute;
generating a third SMQT output using the third histogram; and
generating a fourth SMQT output using the fourth histogram, and wherein
the generation of the interpolation look-up-table is further based upon the third SMQT output and the fourth SMQT output, the interpolation look-up-table specifying an RGB output value for a given RGB input value.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the first SMQT output is derived from an SMQT comprising 8 levels and the second SMQT output is derived from an SMQT comprising 8 levels.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein generating a first histogram for a first attribute associated with pixels in the first image comprises iterating over less than all of the pixels in the image.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the interpolation parameter is received from a human user.

### Claim 8

A non-transitory computer-readable medium comprising instructions configured to cause at least one processor to perform a method, comprising:
obtaining a first image;
generating a first histogram for a first attribute associated with pixels in the first image;
generating a second histogram for a second attribute associated with pixels in the first image;
generating a first SMQT output using the first histogram; and
generating a second SMQT output using the second histogram; and
generating an interpolation look-up-table based at least in part upon the first SMQT output, the second SMQT output, and an interpolation parameter; and
adjusting a pixel in a second image using the interpolation look-up-table.

### Claim 9 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein generating a first SMQT output comprises generating an SMQT table for the first attribute.

### Claim 10 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the first attribute is one of a red, green, or blue pixel value attribute and the second attribute is a luminance value attribute.

### Claim 11 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the first attribute comprises a red pixel value attribute and the second attribute comprises a luminance pixel value attribute, the method further comprising:
generating a third histogram for a green pixel value attribute;
generating a fourth histogram for a blue pixel value attribute;
generating a third SMQT output using the third histogram; and
generating a fourth SMQT output using the fourth histogram, and wherein
the generation of the interpolation look-up-table is further based upon the third SMQT output and the fourth SMQT output, the interpolation look-up-table specifying an RGB output value for a given RGB input value.

### Claim 12 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the first SMQT output is derived from an SMQT comprising 8 levels and the second SMQT output is derived from an SMQT comprising 8 levels.

### Claim 13 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein generating a first histogram for a first attribute associated with pixels in the first image comprises iterating over less than all of the pixels in the image.

### Claim 14 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the interpolation parameter is received from a human user.

### Claim 15

A computer system, comprising:
one or more hardware modules configured to:
receive a first image;
generate a first histogram for a first attribute associated with pixels in the first image;
generate a second histogram for a second attribute associated with pixels in the first image;
generate a first SMQT output using the first histogram; and
generate a second SMQT output using the second histogram; and
generate an interpolation look-up-table based at least in part upon the first SMQT output, the second SMQT output, and an interpolation parameter; and
adjust a pixel in a second image using the interpolation look-up-table.

### Claim 16 (depends on 15)

The computer system of claim 15, wherein generating a first SMQT output comprises generating an SMQT table for the first attribute.

### Claim 17 (depends on 15)

The computer system of claim 15, wherein the first attribute is one of a red, green, or blue pixel value attribute and the second attribute is a luminance value attribute.

### Claim 18 (depends on 15)

The computer system of claim 15, wherein the first attribute comprises a red pixel value attribute and the second attribute comprises a luminance pixel value attribute, the method further comprising:
generating a third histogram for a green pixel value attribute;
generating a fourth histogram for a blue pixel value attribute;
generating a third SMQT output using the third histogram; and
generating a fourth SMQT output using the fourth histogram, and wherein
the generation of the interpolation look-up-table is further based upon the third SMQT output and the fourth SMQT output, the interpolation look-up-table specifying an RGB output value for a given RGB input value.

### Claim 19 (depends on 15)

The computer system of claim 15, wherein the first SMQT output is derived from an SMQT comprising 8 levels and the second SMQT output is derived from an SMQT comprising 8 levels.

### Claim 20 (depends on 15)

The computer system of claim 15, wherein generating a first histogram for a first attribute associated with pixels in the first image comprises iterating over less than all of the pixels in the image.

### Claim 21 (depends on 15)

The computer system of claim 15, wherein the interpolation parameter is received from a human user.