# US 2025/0166239 A1 · Method and apparatus of image processing, electronic device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166239 A1
- Document ID: US20250166239A1
- Status: Published application
- Kind: A1
- Application number: 18/725,698
- Filed: 2022-12-26
- Published: 2025-05-22
- Earliest claimed: 2021-12-29
- Assignees: Beijing Zitiao Network Technology Co., Ltd.
- Applicants: Beijing Zitiao Network Technology Co., Ltd.
- Inventors: Jiabin Huang
- Web page: https://patentgrep.com/patent/US20250166239A1

## CPC

- G06T 11/00: Two-dimensional [2D] image generation
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/08: Learning methods
- G06T 3/00: Geometric image transformations in the plane of the image
- G06T 5/20: using local operators
- G06T 7/11: Region-based segmentation
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/443: {by matching or filtering}
- G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/82: using neural networks
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20028: Bilateral filtering
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The disclosure provides a method and apparatus of image processing, an electronic device, and a storage medium. The method of image processing includes: obtaining a video frame to be processed and determining a target normal map of the video frame to be processed; determining, based on the target normal map and preset attribute information of a light source, target lighting intensity information of at least one pixel of the video frame to be processed; and determining display information of the at least one pixel based on the target lighting intensity information of the at least one pixel, and determining, based on the display information, a target video frame corresponding to the video frame to be processed.

## Claims

### Claim 1

A method of image processing, the method comprising:
obtaining a video frame to be processed and determining a target normal map of the video frame to be processed;
determining, based on the target normal map and preset attribute information of a light source, target lighting intensity information of at least one pixel of the video frame to be processed; and
determining display information of the at least one pixel based on the target lighting intensity information of the at least one pixel, to determine, based on the display information, a target video frame corresponding to the video frame to be processed.

### Claim 2 (depends on 1)

The method of claim 1, wherein obtaining the video frame to be processed and determining the target normal map of the video frame to be processed comprise:
sequentially obtaining at least one video frame to be processed from a target video;
determining gradient information, in a first direction and a second direction, of at least one pixel of the video frame to be processed to obtain normal information of the at least one pixel; and
obtaining the target normal map based on the normal information of the at least one pixel.

### Claim 3 (depends on 1)

The method of claim 1, wherein obtaining the video frame to be processed and determining the target normal map of the video frame to be processed comprise:
obtaining the video frame to be processed, and determining, based on a pre-trained image segmentation model, a target segmentation region corresponding to the video frame to be processed; and
determining gradient information, in a first direction and a second direction, of at least one pixel in the target segmentation region to obtain normal information of the at least one pixel, and determining, based on the normal information, the target normal map of the video frame to be processed.

### Claim 4 (depends on 2)

The method of claim 2, wherein determining the gradient information, in the first direction and the second direction, of the at least one pixel to obtain the normal information of the at least one pixel comprises:
filtering, via joint bilateral filtering, the video frame to be processed to obtain a video frame to be used; and
determining, using a Sobel operator, gradient information, in the first direction and the second direction, of each pixel in the video frame to be used, and determining the normal information of the at least one pixel.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining, based on the target normal map and the preset attribute information of the light source, the target lighting intensity information of at least one pixel of the video frame to be processed comprises, for each of the at least one pixel:
determining target normal information corresponding to a current pixel in the target normal map, and
determining target lighting intensity information of the current pixel based on the target normal information, the attribute information of the light source, and shooting angle information of a video frame to which the current pixel belongs.

### Claim 6 (depends on 5)

The method of claim 5, wherein the attribute information of the light source comprises position information of the light source, and wherein determining the target lighting intensity information of the current pixel based on the target normal information, the attribute information of the light source, and the shooting angle information of a video frame to which the current pixel belongs comprises:
determining light direction information of the current pixel based on the position information of the light source;
determining a diffuse lighting value of the current pixel based on the light direction information, the target normal information, and a preset diffuse coefficient;
determining a target reflection angle based on the light direction information and the target normal information, and determining a reflection intensity value of the current pixel based on the target reflection angle, the shooting angle information and a preset reflection coefficient; and
determining the target lighting intensity information based on the diffuse lighting value, the reflection intensity value, and an ambient lighting intensity value corresponding to the attribute information of the light source.

### Claim 7 (depends on 6)

The method of claim 6, wherein determining the target lighting intensity information based on the diffuse lighting value, the reflection intensity value, and an ambient lighting intensity value corresponding to the attribute information of the light source comprises:
determining the target lighting intensity information as a maximum value among the diffuse lighting value, the reflection intensity value, and the ambient lighting intensity value.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining the display information of the at least one pixel based on the target lighting intensity information of the at least one pixel comprises:
updating the display information of the at least one pixel based on the target lighting intensity information and pixel value information of the at least one pixel.

### Claim 9

(canceled)

### Claim 10

An electronic device, comprising:
at least one processor; and
a memory storing at least one program;
wherein the at least one processor, when executing the at least one program, is caused to implement a method comprising:
obtaining a video frame to be processed and determining a target normal map of the video frame to be processed;
determining, based on the target normal map and preset attribute information of a light source, target lighting intensity information of at least one pixel of the video frame to be processed; and
determining display information of the at least one pixel based on the target lighting intensity information of the at least one pixel, to determine, based on the display information, a target video frame corresponding to the video frame to be processed.

### Claim 11

A non-transitory readable storage medium comprising computer-executable instructions which, when executed by a computer processor, implement a method comprising:
obtaining a video frame to be processed and determining a target normal map of the video frame to be processed;
determining, based on the target normal map and preset attribute information of a light source, target lighting intensity information of at least one pixel of the video frame to be processed; and
determining display information of the at least one pixel based on the target lighting intensity information of the at least one pixel, to determine, based on the display information, a target video frame corresponding to the video frame to be processed.

### Claim 12

(canceled)

### Claim 13 (depends on 10)

The electronic device of claim 10, wherein obtaining the video frame to be processed and determining the target normal map of the video frame to be processed comprise:
sequentially obtaining at least one video frame to be processed from a target video;
determining gradient information, in a first direction and a second direction, of at least one pixel of the video frame to be processed to obtain normal information of the at least one pixel; and
obtaining the target normal map based on the normal information of the at least one pixel.

### Claim 14 (depends on 1)

The electronic device of claim 1, wherein obtaining the video frame to be processed and determining the target normal map of the video frame to be processed comprise:
obtaining the video frame to be processed, and determining, based on a pre-trained image segmentation model, a target segmentation region corresponding to the video frame to be processed; and
determining gradient information, in a first direction and a second direction, of at least one pixel in the target segmentation region to obtain normal information of the at least one pixel, and determining, based on the normal information, the target normal map of the video frame to be processed.

### Claim 15 (depends on 13)

The electronic device of claim 13, wherein determining the gradient information, in the first direction and the second direction, of the at least one pixel to obtain the normal information of the at least one pixel comprises:
filtering, via joint bilateral filtering, the video frame to be processed to obtain a video frame to be used; and
determining, using a Sobel operator, gradient information, in the first direction and the second direction, of each pixel in the video frame to be used, and determining the normal information of the at least one pixel.

### Claim 16 (depends on 10)

The electronic device of claim 10, wherein determining, based on the target normal map and the preset attribute information of the light source, the target lighting intensity information of at least one pixel of the video frame to be processed comprises, for each of the at least one pixel:
determining target normal information corresponding to a current pixel in the target normal map, and
determining target lighting intensity information of the current pixel based on the target normal information, the attribute information of the light source, and shooting angle information of a video frame to which the current pixel belongs.

### Claim 17 (depends on 16)

The electronic device of claim 16, wherein the attribute information of the light source comprises position information of the light source, and wherein determining the target lighting intensity information of the current pixel based on the target normal information, the attribute information of the light source, and the shooting angle information of a video frame to which the current pixel belongs comprises:
determining light direction information of the current pixel based on the position information of the light source;
determining a diffuse lighting value of the current pixel based on the light direction information, the target normal information, and a preset diffuse coefficient;
determining a target reflection angle based on the light direction information and the target normal information, and determining a reflection intensity value of the current pixel based on the target reflection angle, the shooting angle information and a preset reflection coefficient; and
determining the target lighting intensity information based on the diffuse lighting value, the reflection intensity value, and an ambient lighting intensity value corresponding to the attribute information of the light source.

### Claim 18 (depends on 17)

The electronic device of claim 17, wherein determining the target lighting intensity information based on the diffuse lighting value, the reflection intensity value, and an ambient lighting intensity value corresponding to the attribute information of the light source comprises:
determining the target lighting intensity information as a maximum value among the diffuse lighting value, the reflection intensity value, and the ambient lighting intensity value.

### Claim 19 (depends on 10)

The electronic device of claim 10, wherein determining the display information of the at least one pixel based on the target lighting intensity information of the at least one pixel comprises:
updating the display information of the at least one pixel based on the target lighting intensity information and pixel value information of the at least one pixel.

### Claim 20 (depends on 11)

The non-transitory readable storage medium of claim 11, wherein obtaining the video frame to be processed and determining the target normal map of the video frame to be processed comprise:
sequentially obtaining at least one video frame to be processed from a target video;
determining gradient information, in a first direction and a second direction, of at least one pixel of the video frame to be processed to obtain normal information of the at least one pixel; and
obtaining the target normal map based on the normal information of the at least one pixel.

### Claim 21 (depends on 11)

The non-transitory readable storage medium of claim 11, wherein obtaining the video frame to be processed and determining the target normal map of the video frame to be processed comprise:
obtaining the video frame to be processed, and determining, based on a pre-trained image segmentation model, a target segmentation region corresponding to the video frame to be processed; and
determining gradient information, in a first direction and a second direction, of at least one pixel in the target segmentation region to obtain normal information of the at least one pixel, and determining, based on the normal information, the target normal map of the video frame to be processed.

### Claim 22 (depends on 20)

The non-transitory readable storage medium of claim 20, wherein determining the gradient information, in the first direction and the second direction, of the at least one pixel to obtain the normal information of the at least one pixel comprises:
filtering, via joint bilateral filtering, the video frame to be processed to obtain a video frame to be used; and
determining, using a Sobel operator, gradient information, in the first direction and the second direction, of each pixel in the video frame to be used, and determining the normal information of the at least one pixel.
