# US 2024/0155254 A1 · Image Processing Method and Related Electronic Device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0155254 A1
- Document ID: US20240155254A1
- Status: Published application
- Kind: A1
- Application number: 18/006,183
- Filed: 2022-08-29
- Published: 2024-05-09
- Earliest claimed: 2021-12-31
- Assignees: Honor Device Co., Ltd.
- Applicants: Honor Device Co., Ltd.
- Inventors: Yanlin Qian
- Web page: https://patentgrep.com/patent/US20240155254A1

## CPC

- H04N 1/6086: {by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature}
- H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control
- H04N 23/631: {Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters}
- H04N 23/71: Circuitry for evaluating the brightness variation
- H04N 1/608: {within the L, C1, C2 colour signals}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

This application proposes an image processing method and a related electronic device. The method includes: detecting a first operation; starting, by an electronic device, a camera in response to the first operation; detecting a second operation; acquiring, by the camera, a first image in response to the second operation; performing first image processing on the first image to obtain a second image in a second color space, where the second color space is different from a first color space; performing second image processing on the second image to obtain a target image; and storing the target image, where the performing second image processing on the second image includes: obtaining a second parameter based on a first parameter of a first ambient light source, where the first parameter includes a white balance parameter of the first ambient light source, the second parameter has a correspondence with the first parameter, the second parameter is used to identify light source information of the first ambient light source in a third color space, and the third color space is different from the first color space and the second color space; and performing second image processing on the second image based on the second parameter.

## Claims

### Claim 1

An image processing method, applied to an electronic device comprising a camera, the method comprising:
detecting a first operation;
starting, by the electronic device, the camera in response to the first operation;
detecting a second operation;
acquiring, by the camera, a first image under a first ambient light source in response to the second operation, wherein the first image is an image in a first color space, and the first ambient light source is at least one ambient light source for acquisition of the first image;
obtaining a second parameter based on a first parameter of the first ambient light source, wherein the second parameter is a parameter related to a third color space, the first parameter comprises an x1 value and a y1 value of the first ambient light source in an xy chromaticity diagram, the second parameter has a correspondence with the first parameter, the second parameter identifies lightsource information of the first ambient light source in the third color space, the third color space is different from the first color space, and the third color space is different from the second color space;
performing second image processing on the second image based on the second parameter to obtain a target image; and
storing the target image.

### Claim at position 2

-3. (canceled)

### Claim 4 (depends on 1)

The method of claim 1, wherein performing first image processing on the first image to obtain the second image in the second color space comprises:
adjusting the first image based on a formula
[
R
1
⁢
i
′
G
1
⁢
i
′
B
1
⁢
i
′
]
=
[
R
gain
0
0
0
1
0
0
0
B
gain
]
·
[
R
1
⁢
i
G
1
⁢
i
B
1
⁢
i
]
to obtain an adjusted first image, wherein
[
R
1
⁢
i
G
1
⁢
i
B
1
⁢
i
]
are RGB values of an i-th pixel of the first image,
[
R
1
⁢
i
′
G
1
⁢
i
′
B
1
⁢
i
′
]
are RGB values of an i-th pixel of the adjusted first image, Rgain=G/R, Bgain=G/B, and, R, G, and B are RGB values, respectively, of the first ambient light source in the first color space; and
adjusting the adjusted first image based on a formula
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
=
CCM
·
[
R
1
⁢
i
′
G
1
⁢
i
′
B
1
⁢
i
′
]
to obtain the second image, wherein CCM is a color correction matrix obtained based on the first parameter, and
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
are RGB values of the i-th pixel of the second image in the second color space.

### Claim 5 (depends on 1)

The method of claim 1, wherein obtaining the second parameter based on the first parameter of the first ambient light source comprises:
calculating a degree D of chromatic adaptation based on the first parameter;
calculating, based on the first parameter, LMS values of a white point corresponding to the first ambient light source in the third color space;
calculating a gain matrix based on a formula
M
Gain
⁢
1
=
[
L
dst
/
L
src
0
0
0
M
dst
/
M
src
0
0
0
S
dst
/
S
src
]
,
wherein L, M, and S are LMS values of a reference white point in the third color space, a light source corresponding to the reference white point is a reference light source, and the L, M, and S values are the LMS values of the white point corresponding to the first ambient light source in the third color space;
calculating LMS values of a target white point in the third color space based on a formula
[
L
cur
M
cur
S
cur
]
=
(
D
·
M
Gain
⁢
1
+
1
-
D
)
×
Mcat
1
×
[
X
src
Y
src
Z
src
]
,
wherein a light source corresponding to the target white point is a target light source,
[
X
src
Y
src
Z
src
]
are XYZ values of the white point corresponding to the first ambient light source in an XYZ color space,
[
X
src
Y
src
Z
src
]
are obtained based on the L, M, and S values, Mcat1 is a conversion matrix for conversion from the XYZ color space to an LMS color space, and
[
L
cur
M
cur
S
cur
]
are the LMS values of the target white point in the third color space; and
calculating the second parameter based on a formula
CA
=
[
L
cur
/
L
dst
0
0
0
M
cur
/
M
dst
0
0
0
S
cur
/
S
dst
]
,
wherein CA is the second parameter.

### Claim 6 (depends on 5)

The method of claim 5, wherein calculating the degree D of chromatic adaptation based on the first parameter comprises:
calculating a first variable based on a formula Pxy=√{square root over ((x1−xN)2+(y1−yN)2)}, wherein Pxy is the first variable, xN is an x value of the reference white point in the xy chromaticity diagram, and yN is a y value of the reference white point in the xy chromaticity diagram;
calculating a second variable based on a formula Dc=1−e−5.3·Pxy, wherein Dc is the second variable; and
obtaining the degree D of chromatic adaptation based on a formula D=0.96·Dc((1−e(−4.28·log La))406.5−1)+1, wherein La is a lighting value of the first ambient light source.

### Claim 7 (depends on 5)

The method of claim 5, wherein calculating, based on the first parameter, LMS values of the white point corresponding to the first ambient light source in the third color space comprises:
calculating a third variable based on a formula
i
y
=
1
y
1
,
wherein iy is the third variable;
calculating XYZ values of the white point corresponding to the first ambient light source in the XYZ color space based on a formula Xsrc=x1·Ysrc·iy and a formula Zsrc=Ysrc·(1−x1−y1)·iy, wherein X, Y, and Z are the XYZ values of the white point corresponding to the first ambient light source in the XYZ color space, Ysrc=m, and m is an image brightness preset on the electronic device; and
calculating LMS values of the white point corresponding to the first ambient light source in the third color space based on a formula
[
L
src
M
src
S
src
]
=
Mcat
1
·
[
X
src
Y
src
Z
s
⁢
r
⁢
c
]
,
and
_
⁢
wherein
[
L
s
⁢
r
⁢
c
M
s
⁢
r
⁢
c
S
s
⁢
r
⁢
c
]
are LMS values of the white point corresponding to the first ambient light source in the third color space.

### Claim 8 (depends on 1)

The method of claim 1, wherein performing second image processing on the second image based on the second parameter to obtain the target image comprises:
performing third image processing on the second image to obtain a third image;
performing chromatic adaptation processing on the third image based on a formula
[
L
4
⁢
i
M
4
⁢
i
S
4
⁢
i
]
=
CA
·
[
L
3
⁢
i
M
3
⁢
i
S
3
⁢
i
]
to obtain a fourth image, wherein CA is the second parameter,
[
L
3
⁢
i
M
3
⁢
i
S
3
⁢
i
]
are LMS values of the third image in the third color space, and
[
L
4
⁢
i
M
4
⁢
i
S
4
⁢
i
]
are LMS values of the fourth image in the third color space; and
performing fourth image processing on the fourth image to obtain the target image.

### Claim 9 (depends on 8)

The method of claim 8, wherein performing third image processing on the second image to obtain the third image specifically comprises:
converting the second image from the second color space to an XYZ color space based on a formula
[
X
2
⁢
i
Y
2
⁢
i
Z
2
⁢
i
]
=
M
1
·
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
to obtain a first converted image, wherein
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
are RGB values of an i-th pixel of the second image in the second color space, M1 is a conversion matrix for conversion from the second color space to the XYZ color space, and
[
X
2
⁢
i
Y
2
⁢
i
Z
2
⁢
i
]
are XYZ values of an i-th pixel of the first converted image in the XYZ color space; and
converting the first converted image from the XYZ color space to the third color space based on a formula
[
L
3
⁢
i
M
3
⁢
i
S
3
⁢
i
]
=
Mcat
1
·
[
X
2
⁢
i
Y
2
⁢
i
Z
2
⁢
i
]
to obtain the third image, wherein Mcat1 is a conversion matrix for conversion from the XYZ color space to an LMS color space, and
[
L
3
⁢
i
M
3
⁢
i
S
3
⁢
i
]
are LMS values of the third image in the third color space.

### Claim 10 (depends on 8)

The method of claim 8, wherein performing fourth image processing on the fourth image to obtain the target image comprises:
converting the fourth image from the third color space to an XYZ color space based on a formula
[
X
5
⁢
i
Y
5
⁢
i
Z
5
⁢
i
]
=
Mcat
2
·
[
L
4
⁢
i
M
4
⁢
i
S
4
⁢
i
]
to obtain a second converted image, wherein
[
X
5
⁢
i
Y
5
⁢
i
Z
5
⁢
i
]
are XYZ values of an i-th pixel of the second converted image in the XYZ color space, and Mcat2 is a conversion matrix for conversion from an LMS color space to the XYZ color space;
converting the second converted image from the XYZ color space to the second color space based on a formula
[
R
6
⁢
i
G
6
⁢
i
B
6
⁢
i
]
=
M
2
×
[
X
5
⁢
i
Y
5
⁢
i
Z
5
⁢
i
]
to obtain a third converted image, wherein M2 is a conversion matrix for conversion from the XYZ color space to the second color space, and
[
R
6
⁢
i
G
6
⁢
i
B
6
⁢
i
]
are RGB values of an i-th pixel of the third converted image in the second color space;
performing RGB color space image processing on the third converted image to obtain a fourth converted image;
converting the fourth converted image from the second color space to a YUV color space to obtain a fifth converted image; and
performing YUV color space processing on the fifth converted image to obtain the target image.

### Claim 11 (depends on 1)

The method of claim 1, wherein performing second image processing on the second image based on the second parameter to obtain the target image comprises:
performing fifth image processing on the second image based on a formula
[
R
6
⁢
i
G
6
⁢
i
B
6
⁢
i
]
=
C
⁢
A
1
×
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
to obtain a third converted image, wherein
[
R
6
⁢
i
G
6
⁢
i
B
6
⁢
i
]
are RGB values of an i-th pixel of the third converted image in the second color space,
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
are RGB values of an i-th pixel of the second image in the second color space, and CA1 is a target matrix;
performing RGB color space image processing on the third converted image to obtain a fourth converted image;
converting the fourth converted image from the second color space to a YUV color space to obtain a fifth converted image; and
performing YUV color space processing on the fifth converted image to obtain the target image,
wherein CA1=M2·Mcat2·CA·Mcat1·M1, M1 is a conversion matrix for conversion from the second color space to an XYZ color space, M2 is a conversion matrix for conversion from the XYZ color space to the first color space, Mcat1 is a conversion matrix for conversion from the XYZ color space to an LMS color space, Mcat2 is a conversion matrix for conversion from the LMS color space to the XYZ color space, and CA is the second parameter.

### Claim at position 11

-14. (canceled)

### Claim 15

An electronic device, comprising:
a processor; and
a non-transitory memory coupled to the processor and configured to store instructions that, when executed by the processor, causes the electronic device to be configured to:
detect a first operation;
start a camera of the electronic device in response to the first operation;
detect a second operation;
acquire, by the camera, a first image under a first ambient light source in response to the second operation, wherein the first image is an image in a first color space, and the first ambient light source is at least one ambient light source for acquisition of the first image;
perform first image processing on the first image to obtain a second image in a second color space, wherein the second color space is different from the first color space;
obtain a second parameter based on a first parameter of the first ambient light source, wherein the second parameter is a parameter related to a third color space, the first parameter comprises a white balance parameter of the first ambient light source, the second parameter has a correspondence with the first parameter, the second parameter identifies light source information of the first ambient light source in the third color space, the third color space is different from the first color space, and the third color space is different from the second color space;
perform second image processing on the second image based on the second parameter to obtain a target image; and
store the target image.

### Claim 16 (depends on 15)

The electronic device of claim 15, wherein the first color space is a RAW space, the second color space is an sRGB color space, and the third color space is an LMS color space or an XYZ color space, and wherein the first parameter comprises an x1 value and a y1 value of the first ambient light source in an xy chromaticity diagram.

### Claim 17 (depends on 15)

The electronic device of claim 15, wherein performing first image processing on the first image to obtain the second image in the second color space comprises:
adjusting the first image based on a formula
[
R
1
⁢
i
′
G
1
⁢
i
′
B
1
⁢
i
′
]
=
[
R
gain
0
0
0
1
0
0
0
B
gain
]
·
[
R
1
⁢
i
G
1
⁢
i
B
1
⁢
i
]
to obtain an adjusted first image, wherein
[
R
1
⁢
i
G
1
⁢
i
B
1
⁢
i
]
are RGB values of an i-th pixel of the first image,
[
R
1
⁢
i
′
G
1
⁢
i
′
B
1
⁢
i
′
]
are RGB values of an i-th pixel of the adjusted first image, Rgain=G/R, Bgain=G/B, and R, G, and B are RGB values, respectively, of the first ambient light source in the first color space; and
adjusting the adjusted first image based on a formula
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
=
CCM
·
[
R
1
⁢
i
′
G
1
⁢
i
′
B
1
⁢
i
′
]
to obtain the second image, wherein CCM is a color correction matrix obtained based on the first parameter, and
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
are RGB values of the i-th pixel of the second image in the second color space.

### Claim 18 (depends on 15)

The electronic device of claim 15, wherein obtaining the second parameter based on the first parameter of the first ambient light source comprises:
calculating a degree D of chromatic adaptation based on the first parameter;
calculating, based on the first parameter, LMS values of a white point corresponding to the first ambient light source in the third color space;
calculating a gain matrix based on a formula
M
Gain
⁢
1
=
[
L
dst
/
L
src
0
0
0
M
dst
/
M
src
0
0
0
S
dst
/
S
src
]
,
wherein Ldst, Mdst, and Sdst are LMS values of a reference white point in the third color space, a light source corresponding to the reference white point is a reference light source, and the Lsrc, Msrc, and Ssrc values are the LMS values of the white point corresponding to the first ambient light source in the third color space;
calculating LMS values of a target white point in the third color space based on a formula
[
L
cur
M
cur
S
cur
]
=
(
D
·
M
Gain
⁢
1
+
1
-
D
)
×
Mcat
1
×
[
X
src
Y
src
Z
src
]
,
wherein a light source corresponding to the target white point is a target light source,
[
X
src
Y
src
Z
src
]
are XYZ values of the white point corresponding to the first ambient light source in an XYZ color space,
[
X
src
Y
src
Z
src
]
are obtained based on the Lsrc, Msrc, and Ssrc values, Mcat1 is a conversion matrix for conversion from the XYZ color space to an LMS color space, and
[
L
cur
M
cur
S
cur
]
are the LMS values of the target white point in the third color space; and
calculating the second parameter based on a formula
CA
=
[
L
cur
/
L
dst
0
0
0
M
cur
/
M
dst
0
0
0
S
cur
/
S
dst
]
,
wherein CA is the second parameter.

### Claim 19 (depends on 18)

The electronic device of claim 18, wherein calculating the degree D of chromatic adaptation based on the first parameter comprises:
calculating a first variable based on a formula Pxy=√{square root over ((x1−xN)+2+(y1−yN)2)}, wherein Pxy is the first variable, xN is an x value of the reference white point in an xy chromaticity diagram, and yN is a y value of the reference white point in the xy chromaticity diagram;
calculating a second variable based on a formula Dc=1−e−5.3·Pxy, wherein Dc is the second variable; and
obtaining the degree D of chromatic adaptation based on a formula D=0.96·Dc((1−e(−4.28·log La))406.5−1)+1, wherein La is a lighting value of the first ambient light source.

### Claim 20 (depends on 18)

The electronic device of claim 18, wherein calculating, based on the first parameter, LMS values of the white point corresponding to the first ambient light source in the third color space comprises:
calculating a third variable based on a formula
i
y
=
1
y
1
,
wherein iy is the third variable;
calculating XYZ values of the white point corresponding to the first ambient light source in the XYZ color space based on a formula Xsrc=x1·Ysrc·iy and a formula Zsrc·=Ysrc·(1−x1−y1)·iy, wherein Xsrc, Ysrc, and Zsrc are the XYZ values of the white point corresponding to the first ambient light source in the XYZ color space, Ysrcm, and m is an image brightness preset on the electronic device; and
calculating LMS values of the white point corresponding to the first ambient light source in the third color space based on a formula
[
L
src
M
src
S
src
]
=
Mcat
1
·
[
X
src
Y
src
Z
src
]
,
and
⁢
wherein
[
L
src
M
src
S
src
]
are LMS values of the white point corresponding to the first ambient light source in the third color space.

### Claim 21 (depends on 15)

The electronic device of claim 15, wherein performing second image processing on the second image based on the second parameter to obtain the target image comprises:
performing third image processing on the second image to obtain a third image;
performing chromatic adaptation processing on the third image based on a formula
[
L
4
⁢
i
M
4
⁢
i
S
4
⁢
i
]
=
CA
·
[
L
3
⁢
i
M
3
⁢
i
S
3
⁢
i
]
to obtain a fourth image, wherein CA is the second parameter,
[
L
3
⁢
i
M
3
⁢
i
S
3
⁢
i
]
are LMS values of the third image in the third color space, and
[
L
4
⁢
i
M
4
⁢
i
S
4
⁢
i
]
are LMS values of the fourth image in the third color space; and
performing fourth image processing on the fourth image to obtain the target image.

### Claim 22 (depends on 21)

The electronic device of claim 21, wherein performing third image processing on the second image to obtain the third image comprises:
converting the second image from the second color space to an XYZ color space based on a formula
[
X
2
⁢
i
Y
2
⁢
i
Z
2
⁢
i
]
=
M
1
·
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
to obtain a first converted image, wherein
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
are RGB values of an i-th pixel of the second image in the second color space, M1 is a conversion matrix for conversion from the second color space to the XYZ color space, and
[
X
2
⁢
i
Y
2
⁢
i
Z
2
⁢
i
]
are XYZ values of an i-th pixel of the first converted image in the XYZ color space; and
converting the first converted image from the XYZ color space to the third color space based on a formula
[
L
3
⁢
i
M
3
⁢
i
S
3
⁢
i
]
=
Mcat
1
·
[
X
2
⁢
i
Y
2
⁢
i
Z
2
⁢
i
]
to obtain the third image, wherein Mcat1 is a conversion matrix for conversion from the XYZ color space to an LMS color space, and
[
L
3
⁢
i
M
3
⁢
i
S
3
⁢
i
]
are LMS values of the third image in the third color space.

### Claim 23 (depends on 21)

The electronic device of claim 21, wherein performing fourth image processing on the fourth image to obtain the target image comprises:
converting the fourth image from the third color space to an XYZ color space based on a formula
[
X
5
⁢
i
Y
5
⁢
i
Z
5
⁢
i
]
=
Mcat
2
·
[
L
4
⁢
i
M
4
⁢
i
S
4
⁢
i
]
to obtain a second converted image, wherein
[
X
5
⁢
i
Y
5
⁢
i
Z
5
⁢
i
]
are XYZ values of an i-th pixel of the second converted image in the XYZ color space, and Mcat2 is a conversion matrix for conversion from an LMS color space to the XYZ color space;
converting the second converted image from the XYZ color space to the second color space based on a formula
[
R
6
⁢
i
G
6
⁢
i
B
6
⁢
i
]
=
M
2
×
[
X
5
⁢
i
Y
5
⁢
i
Z
5
⁢
i
]
to obtain a third converted image, wherein M2 is a conversion matrix for conversion from the XYZ color space to the second color space, and
[
R
6
⁢
i
G
6
⁢
i
B
6
⁢
i
]
are RGB values of an i-th pixel of the third converted image in the second color space;
performing RGB color space image processing on the third converted image to obtain a fourth converted image;
converting the fourth converted image from the second color space to a YUV color space to obtain a fifth converted image; and
performing YUV color space processing on the fifth converted image to obtain the target image.

### Claim 24 (depends on 15)

The electronic device of claim 15, wherein performing second image processing on the second image based on the second parameter to obtain the target image comprises:
performing fifth image processing on the second image based on a formula
[
R
6
⁢
i
G
6
⁢
i
B
6
⁢
i
]
=
CA
1
×
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
to obtain a third converted image, wherein
[
R
6
⁢
i
G
6
⁢
i
B
6
⁢
i
]
are RGB values of an i-th pixel of the third converted image in the second color space, are RGB values of an i-th pixel of the second image in the second color space,
[
R
2
⁢
i
G
2
⁢
i
B
2
⁢
i
]
and CA1 is a target matrix;
performing RGB color space image processing on the third converted image to obtain a fourth converted image;
converting the fourth converted image from the second color space to a YUV color space to obtain a fifth converted image; and
performing YUV color space processing on the fifth converted image to obtain the target image,
wherein CA1=M2·Mcat2·CA·Mcat1·M1, M1 is a conversion matrix for conversion from the second color space to an XYZ color space, M2 is a conversion matrix for conversion from the XYZ color space to the first color space, Mcat1 is a conversion matrix for conversion from the XYZ color space to an LMS color space, Mcat2 is a conversion matrix for conversion from the LMS color space to the XYZ color space, and CA is the second parameter.

### Claim 25

A non-transitory computer-readable storage medium comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to be configured to:
detect a first operation;
start a camera of the electronic device in response to the first operation;
detect a second operation;
acquire, by the camera, a first image under a first ambient light source in response to the second operation, wherein the first image is an image in a first color space, and the first ambient light source is at least one ambient light source for acquisition of the first image;
perform first image processing on the first image to obtain a second image in a second color space, wherein the second color space is different from the first color space;
obtain a second parameter based on a first parameter of the first ambient light source, wherein the second parameter is a parameter related to a third color space, the first parameter comprises a white balance parameter of the first ambient light source, the second parameter has a correspondence with the first parameter, the second parameter identifies light source information of the first ambient light source in the third color space, the third color space is different from the first color space, and the third color space is different from the second color space;
perform second image processing on the second image based on the second parameter to obtain a target image; and
store the target image.
