# US 2022/0036068 A1 · Method and apparatus for recognizing image, electronic device and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036068 A1
- Document ID: US20220036068A1
- Status: Published application
- Kind: A1
- Application number: 17/504,188
- Filed: 2021-10-18
- Published: 2022-02-03
- Earliest claimed: 2021-03-25
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science Technology Co., Ltd.
- Inventors: Yan PENG; Xiang LONG; Honghui ZHENG; Zhuang JIA; Bin ZHANG; Xiaodi WANG; Ying XIN; Yi GU; Yunhao WANG; Chao LI; Yuan FENG; Shumin HAN
- Web page: https://patentgrep.com/patent/US20220036068A1

## CPC

- G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60)
- G06V 20/194: {using hyperspectral data, i.e. more or other wavelengths than RGB}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06N 3/08: Learning methods
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The disclosure provides a method for recognizing an image, an apparatus for recognizing an image, an electronic device and a storage medium. An image to be processed is obtained. The number of first channels of the image is greater than the number of second channels of a red-green-blue (RGB) image. For each pixel of the image, a semantic type of the pixel is determined based on a value of the pixel on each channel. A recognition result of the image is generated based on the image and the semantic type of each pixel.

## Claims

### Claim 1

A method for recognizing an image, comprising:
obtaining an image to be processed, wherein the number of first channels of the image is greater than the number of second channels of a red-green-blue (RGB) image;
for each pixel of the image, determining a semantic type of the pixel based on a value of the pixel on each channel; and
generating a recognition result of the image based on the image and the semantic type of each pixel.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining the semantic type of the pixel based on the value of the pixel on each channel comprises:
obtaining a semantic segmentation model matching the number of the first channels; and
obtaining the semantic type of the pixel by inputting the image into the matched semantic segmentation model.

### Claim 3 (depends on 2)

The method of claim 2, wherein obtaining the semantic segmentation model matching the number of the first channels comprises:
for each of at least one trained candidate semantic segmentation model, determining a difference between the number of inputting channels of the trained candidate semantic segmentation model and the number of the first channels;
obtaining a first candidate semantic segmentation model corresponding to a minimum difference; and
obtaining the matched semantic segmentation model by adjusting the inputting channels of the first candidate semantic segmentation model based on the number of the first channels.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
obtaining an initial candidate semantic segmentation model;
obtaining training data for the initial candidate semantic segmentation model, wherein the training data comprises sample images and sample semantic types of pixels of the sample images; and
obtaining the trained candidate semantic segmentation model by training the initial candidate semantic segmentation model based on the sample images and the semantic types.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating the recognition result of the image based on the image and the semantic type of each pixel comprises:
generating a corresponding RGB image based on the image; and
generating the recognition result of the image based on the corresponding RGB image and the semantic type of each pixel.

### Claim 6 (depends on 5)

The method of claim 5, wherein generating the corresponding RGB image based on the image comprises:
for each pixel of the image, removing values of the pixel on channels other than RGB channels from the image, obtaining a normalized value by normalizing the value of the pixel on each of the RGB channels, and obtaining a processed value by multiplying the normalized value by a value threshold of the RGB channels; and
generating the corresponding RGB image based on the processed value of each pixel on the RGB channels.

### Claim 7 (depends on 6)

The method of claim 6, wherein normalizing the value of the pixel on each of the RGB channels comprises:
determining a maximum value threshold by one of:
obtaining a maximum value from values of pixels of the image on a corresponding channel or on the RGB channels, obtaining a maximum value from values of pixels of each image stored in a preset image collection on a corresponding channel or on the RGB channels; or obtaining a maximum value from value of pixels of a part of images stored in a preset image collection on a corresponding channel or on the RGB channels; and
normalizing the value of pixel on each of the RGB channels based on the maximum value threshold.

### Claim 8

An electronic device, comprising:
at least one processor; and
a memory communicatively connected with the at least one processor; wherein,
the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is configured to:
obtain an image to be processed, wherein the number of first channels of the image is greater than the number of second channels of a red-green-blue (RGB) image;
for each pixel of the image, determine a semantic type of the pixel based on a value of the pixel on each channel; and
generate a recognition result of the image based on the image and the semantic type of each pixel.

### Claim 9 (depends on 8)

The electronic device of claim 8, wherein the at least one processor is further configured to:
obtain a semantic segmentation model matching the number of the first channels; and
obtain the semantic type of the pixel by inputting the image into the matched semantic segmentation model.

### Claim 10 (depends on 9)

The electronic device of claim 9, wherein the at least one processor is further configured to:
for each of at least one trained candidate semantic segmentation model, determine a difference between the number of inputting channels of the trained candidate semantic segmentation model and the number of the first channels;
obtain a first candidate semantic segmentation model corresponding to a minimum difference; and
obtain the matched semantic segmentation model by adjusting the inputting channels of the first candidate semantic segmentation model based on the number of the first channels.

### Claim 11 (depends on 10)

The electronic device of claim 10, wherein the at least one processor is further configured to:
obtain an initial candidate semantic segmentation model;
obtain training data for the initial candidate semantic segmentation model, wherein the training data comprises sample images and sample semantic types of pixels of the sample images; and
obtain the trained candidate semantic segmentation model by training the initial candidate semantic segmentation model based on the sample images and the semantic types.

### Claim 12 (depends on 8)

The electronic device of claim 8, wherein the at least one processor is further configured to:
generate a corresponding RGB image based on the image; and
generate the recognition result of the image based on the corresponding RGB image and the semantic type of each pixel.

### Claim 13 (depends on 12)

The electronic device of claim 12, wherein the at least one processor is further configured to:
for each pixel of the image, remove values of the pixel on channels other than RGB channels from the image, obtain a normalized value by normalizing the value of the pixel on each of the RGB channels, and obtain a processed value by multiplying the normalized value by a value threshold of the RGB channels; and
generate the corresponding RGB image based on the processed value of each pixel on the RGB channels.

### Claim 14 (depends on 13)

The electronic device of claim 13, wherein the at least one processor is further configured to:
determine a maximum value threshold by one of: obtaining a maximum value from values of pixels of the image on a corresponding channel or on the RGB channels, obtaining a maximum value from values of pixels of each image stored in a preset image collection on a corresponding channel or on the RGB channels; or obtaining a maximum value from value of pixels of a part of images stored in a preset image collection on a corresponding channel or on the RGB channels; and
normalize the value of pixel on each of the RGB channels based on the maximum value threshold.

### Claim 15

A non-transitory computer-readable storage medium storing computer instructions, wherein when the computer instructions are executed by a processor, a method for identifying an image is executed, the method comprising:
obtaining an image to be processed, wherein the number of first channels of the image is greater than the number of second channels of a red-green-blue (RGB) image;
for each pixel of the image, determining a semantic type of the pixel based on a value of the pixel on each channel; and
generating a recognition result of the image based on the image and the semantic type of each pixel.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein determining the semantic type of the pixel based on the value of the pixel on each channel comprises:
obtaining a semantic segmentation model matching the number of the first channels; and
obtaining the semantic type of the pixel by inputting the image into the matched semantic segmentation model.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein obtaining the semantic segmentation model matching the number of the first channels comprises:
for each of at least one trained candidate semantic segmentation model, determining a difference between the number of inputting channels of the trained candidate semantic segmentation model and the number of the first channels;
obtaining a first candidate semantic segmentation model corresponding to a minimum difference; and
obtaining the matched semantic segmentation model by adjusting the inputting channels of the first candidate semantic segmentation model based on the number of the first channels.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the method further comprises:
obtaining an initial candidate semantic segmentation model;
obtaining training data for the initial candidate semantic segmentation model, wherein the training data comprises sample images and sample semantic types of pixels of the sample images; and
obtaining the trained candidate semantic segmentation model by training the initial candidate semantic segmentation model based on the sample images and the semantic types.

### Claim 19 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein generating the recognition result of the image based on the image and the semantic type of each pixel comprises:
generating a corresponding RGB image based on the image; and
generating the recognition result of the image based on the corresponding RGB image and the semantic type of each pixel.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein generating the corresponding RGB image based on the image comprises:
for each pixel of the image, removing values of the pixel on channels other than RGB channels from the image, obtaining a normalized value by normalizing the value of the pixel on each of the RGB channels, and obtaining a processed value by multiplying the normalized value by a value threshold of the RGB channels; and
generating the corresponding RGB image based on the processed value of each pixel on the RGB channels.
