# US 11,451,752 B2 · Grid gain calculation circuit, image sensing device and operation method of the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,451,752 B2
- Document ID: US11451752B2
- Status: Granted
- Kind: B2
- Application number: 17/089,156
- Filed: 2020-11-04
- Published: 2021-11-18
- Granted: 2022-09-20
- Earliest claimed: 2020-05-18
- Assignees: SK hynix Inc.
- Applicants: SK hynix Inc.
- Inventors: Jae Yoon Yoo
- Web page: https://patentgrep.com/patent/US11451752B2

## CPC

- H04N 17/002: {for television cameras}
- H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values}
- G06T 5/80: Geometric correction
- H04N 23/55: Optical parts specially adapted for electronic image sensors; Mounting thereof
- H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A grid gain calculation circuit includes a region setting component suitable for setting a grid region corresponding to a pixel array; a virtual grid point setting component suitable for setting virtual grid points with respect to outer grid points located on a border of a grid region and with respect to an inner grid point located in an inner area of the grid region close to the outer grid points; and a gain calculation component suitable for calculating a gain of the outer grid points based on a gain of the inner grid point and a gain of the virtual grid points.

## Claims

### Claim 1

A grid gain calculation circuit, comprising:
a region setting component suitable for setting a grid region corresponding to a pixel array during a calibration operation;
a virtual grid point setting component suitable for setting virtual grid points with respect to outer grid points located on a border of a grid region and with respect to an inner grid point located in an inner area of the grid region close to the outer grid points; and
a gain calculation component suitable for calculating a gain of the outer grid points based on a gain of the inner grid point and a gain of the virtual grid points,
wherein the gain is associated with the calibration operation for balancing a pixel color level,
wherein the virtual grid points are virtually set to an outer area of each set of the outer grid points based on the corresponding outer grid points and the corresponding inner grid point.

### Claim 2 (depends on 1)

The grid gain calculation circuit of claim 1, wherein the virtual grid point setting component sets the virtual grid points such that a distance between two vertically neighboring inner grid points is set to be the same as a distance between a vertical direction virtual grid point and the corresponding inner grid point, and such that a distance between two horizontally neighboring inner grid points is set to be the same as a distance between a horizontal direction virtual grid point and the corresponding inner grid point.

### Claim 3 (depends on 2)

The grid gain calculation circuit of claim 2, wherein the virtual grid point setting component sets external division points with respect to the outer grid points and the inner grid point to the virtual grid points.

### Claim 4 (depends on 3)

The grid gain calculation circuit of claim 3, wherein the virtual grid point setting component sets a horizontal direction external division point with respect to a horizontal direction outer grid point of the outer grid points and the inner grid point to a horizontal direction virtual grid point, sets a vertical direction external division point with respect to a vertical direction outer grid point of the outer grid points and the inner grid point to a vertical direction virtual grid point, sets a horizontal direction external division point with respect to a corner direction outer grid point of the outer grid points and the vertical direction outer grid point to a temporary virtual grid point, and sets a vertical direction external division point with respect to the horizontal direction virtual grid point and the temporary virtual grid point to a corner direction virtual grid point.

### Claim 5 (depends on 4)

The grid gain calculation circuit of claim 4, wherein the gain calculation component calculates a gain of the corner direction outer grid point based on a gain of the vertical direction virtual grid point, a gain of the horizontal direction virtual grid point, a gain of the corner direction virtual grid point and a gain of the inner grid point.

### Claim 6 (depends on 5)

The grid gain calculation circuit of claim 5, wherein the gain calculation component calculates the gain of the corner direction outer grid point according to the following equation:
gain
⁢
⁢
(
x
,
y
)
=
{
gain
⁢
⁢
(
x
1
,
y
1
)
×
d
x
⁢
⁢
2
×
d
y
⁢
⁢
2
+
gain
⁢
⁢
(
x
2
,
y
2
)
×
d
x
⁢
⁢
1
×
d
y
⁢
⁢
2
+
gain
⁢
⁢
(
x
3
,
y
3
)
×
d
x
⁢
⁢
2
×
d
y
⁢
⁢
1
+
gain
⁢
⁢
(
x
4
,
y
4
)
×
d
x
⁢
⁢
1
×
d
y
⁢
⁢
1
}
(
d
x
⁢
⁢
1
+
d
x
⁢
⁢
2
)
×
(
d
y
⁢
⁢
1
+
d
y
⁢
⁢
2
)
where ‘gain(x,y)’ denotes the gain of the corner direction outer grid point, ‘gain(x1,y1)’ denotes the gain of the corner direction virtual grid point, ‘gain(x2,y2)’ denotes the gain of the vertical direction virtual gird point, ‘gain(x3,y3)’ denotes the gain of the horizontal direction virtual grid point, ‘gain(x4,y4)’ denotes the gain of the inner grid point, ‘dx1’ denotes a horizontal distance between the corner direction outer grid point and the temporary virtual grid point, ‘dx2’ denotes a horizontal distance between the vertical direction outer grid point and the corner direction outer grid point, ‘dy1’ denotes a vertical distance between the vertical direction virtual grid point and the corner direction outer grid point, and ‘dy2’ denotes a vertical distance between the horizontal direction outer grid point and the corner direction outer grid point.

### Claim 7 (depends on 5)

The grid gain calculation circuit of claim 5, wherein the gain calculation component sets the gain of each of the vertical direction virtual grid point, the horizontal direction virtual grid point and the corner direction virtual grid point to be the same as the gain of the inner grid point.

### Claim 8

An image sensing device, comprising:
an image sensor including a pixel array having a plurality of pixels;
an image signal processor suitable for processing an output signal of the image sensor; and
a grid gain calculation circuit,
wherein the grid gain calculation circuit comprises
a region setting component suitable for setting a grid region corresponding to the pixel array during a calibration operation;
a virtual grid point setting component suitable for setting virtual grid points with respect to outer grid points located on a border of a grid region and with respect to an inner grid point located in an inner area of the grid region close to the outer grid points; and
a calculation component suitable for calculating a gain of the outer grid points based on a gain of the inner grid point and a gain of the virtual grid points,
wherein the gain is associated with the calibration operation for balancing a pixel color level,
wherein the virtual grid points are virtually set to an outer area of each set of the outer grid points based on the corresponding outer grid points and the corresponding inner grid point.

### Claim 9 (depends on 8)

The image sensing device of claim 8, wherein the virtual grid point setting component sets the virtual grid points such that a distance between the inner grid point and another inner grid point is the same as a distance between each of the virtual grid points and the inner grid point.

### Claim 10 (depends on 9)

The image sensing device of claim 9, wherein the virtual grid point setting component sets external division points with respect to the outer grid points and the inner grid point to the virtual grid points.

### Claim 11 (depends on 10)

The image sensing device of claim 10, wherein the virtual grid point setting component sets a horizontal direction external division point with respect to a horizontal direction outer grid point of the outer grid points and the inner grid point to a horizontal direction virtual grid point, sets a vertical direction external division point with respect to a vertical direction outer grid point of the outer grid points and the inner grid point to a vertical direction virtual grid point, sets a horizontal direction external division point with respect to a corner direction outer grid point of the outer grid points and the vertical direction outer grid point to a temporary virtual grid point, and sets a vertical direction external division point with respect to the horizontal direction virtual grid point and the temporary virtual grid point to a corner direction virtual grid point.

### Claim 12 (depends on 11)

The image sensing device of claim 11, wherein the gain calculation component calculates a gain of the corner direction outer grid point based on a gain of the vertical direction virtual grid point, a gain of the horizontal direction virtual grid point, a gain of the corner direction virtual grid point and a gain of the inner grid point.

### Claim 13 (depends on 12)

The image sensing device of claim 12, wherein the gain calculation component calculates the gain of the corner direction outer grid point according to the following equation:
gain
⁢
⁢
(
x
,
y
)
=
{
gain
⁢
⁢
(
x
1
,
y
1
)
×
d
x
⁢
⁢
2
×
d
y
⁢
⁢
2
+
gain
⁢
⁢
(
x
2
,
y
2
)
×
d
x
⁢
⁢
1
×
d
y
⁢
⁢
2
+
gain
⁢
⁢
(
x
3
,
y
3
)
×
d
x
⁢
⁢
2
×
d
y
⁢
⁢
1
+
gain
⁢
⁢
(
x
4
,
y
4
)
×
d
x
⁢
⁢
1
×
d
y
⁢
⁢
1
}
(
d
x
⁢
⁢
1
+
d
x
⁢
⁢
2
)
×
(
d
y
⁢
⁢
1
+
d
y
⁢
⁢
2
)
where ‘gain(x,y)’ denotes the gain of the corner direction outer grid point, ‘gain(x1,y1)’ denotes the gain of the corner direction virtual grid point, ‘gain(x2,y2)’ denotes the gain of the vertical direction virtual gird point, ‘gain(x3,y3)’ denotes the gain of the horizontal direction virtual grid point, ‘gain(x4,y4)’ denotes the gain of the inner grid point, ‘dx1’ denotes a horizontal distance between the corner direction outer grid point and the temporary virtual grid point, ‘dx2’ denotes a horizontal distance between the vertical direction outer grid point and the corner direction outer grid point, ‘dy1’ denotes a vertical distance between the vertical direction virtual grid point and the corner direction outer grid point, and ‘dy2’ denotes a vertical distance between the horizontal direction outer grid point and the corner direction outer grid point.

### Claim 14 (depends on 12)

The image sensing device of claim 12, wherein the gain calculation component sets the gain of each of the vertical direction virtual grid point, the horizontal direction virtual grid point and the corner direction virtual grid point to be the same as the gain of the inner grid point.

### Claim 15

An image sensing device comprising:
an image sensor including a pixel array having a plurality of pixels; and
a grid gain circuit suitable for:
determining a grid region having a plurality of squares and a plurality of rectangles, which correspond to the pixel array during a calibration operation;
determining an inner area including the plurality of squares and an outer area including the plurality of rectangles, from the grid region;
determining corner regions of the outer area, each corner region including an inner grid point in the inner area and outer grid points;
determining virtual grid points such that each corner region has the same size as each square; and
calculating a gain of the outer grid points based on a gain of the inner grid point and a gain of the virtual grid points,
wherein the gain is associated with the calibration operation for balancing a pixel color level,
wherein the virtual grid points are virtually set to an outer area of each set of the outer grid points based on the outer grid points and the inner grid point.

### Claim 16 (depends on 15)

The image sensing device of claim 15, wherein the virtual grid points are set to external division points with respect to the outer grid points and the inner grid point.

### Claim 17 (depends on 16)

The image sensing device of claim 16, wherein a horizontal direction external division point with respect to a horizontal direction outer grid point of the outer grid points and the inner grid point is set to a horizontal direction virtual grid point.

### Claim 18 (depends on 17)

The image sensing device of claim 17, wherein a vertical direction external division point with respect to a vertical direction outer grid point of the outer grid points and the inner grid point is set to a vertical direction virtual grid point.

### Claim 19 (depends on 18)

The image sensing device of claim 18, wherein a horizontal direction external division point with respect to a corner direction outer grid point of the outer grid points and the vertical direction outer grid point is set to a temporary virtual grid point.

### Claim 20 (depends on 19)

The image sensing device of claim 19, wherein a corner direction virtual grid point is set to a vertical direction external division point with respect to the horizontal direction virtual grid point and the temporary virtual grid point.