# US 12,112,556 B2 · Image recognition method and apparatus, recognition model training method and apparatus, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,556 B2
- Document ID: US12112556B2
- Status: Granted
- Kind: B2
- Application number: 17/402,500
- Filed: 2021-08-13
- Published: 2021-12-02
- Granted: 2024-10-08
- Earliest claimed: 2019-06-21
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Xinrui Zhuang; Yuexiang Li; Yefeng Zheng
- Web page: https://patentgrep.com/patent/US12112556B2

## CPC

- G06V 20/64: Three-dimensional [3D] objects
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/096: Transfer learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/82: using neural networks
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An image recognition method includes: obtaining a target three-dimensional (3D) image; inputting the target 3D image to a first recognition model; and obtaining the image type of the target 3D image outputted by the first recognition model. The first recognition model is configured to perform image recognition on the target 3D image to obtain an image type of the target 3D image. A convolutional block of the first recognition model is the same as a convolutional block of a second recognition model, and configured to perform image recognition on the target 3D image. The second recognition model is obtained by training an original recognition model using a target training sample, the target training sample including cubes obtained by rotating and sorting N target cubes obtained from a 3D sample image, N being a natural number greater than 1.

## Claims

### Claim 1

An image recognition method, performed by a terminal, the method comprising:
obtaining a target three-dimensional (3D) image;
inputting the target 3D image to a first recognition model, the first recognition model being configured to perform image recognition on the target 3D image to obtain a type of the target 3D image; and
obtaining the type of the target 3D image outputted by the first recognition model,
wherein the first recognition model is obtained by:
determining N target cubes from a 3D sample image to be rotated, N being a natural number greater than 1;
determining a plurality of candidate permutation sequences of the N target cubes;
obtaining, based on the 3D sample image, a target training sample, the target training sample comprising cubes obtained by rotating and sorting the N target cubes based on a target permutation sequence randomly selected from the candidate permutation sequences;
using the target training sample without tag annotation to train an original recognition model and obtain a convolutional block of the second recognition model; and
after the second recognition model is trained, using the 3D sample image with the tag annotation and using the convolutional block of the trained second recognition model as a convolutional block of the first recognition model, to train the first recognition model, the tag annotation including a type of the 3D sample.

### Claim 2 (depends on 1)

The method according to claim 1, wherein before obtaining the target 3D image, the method further comprises:
obtaining the 3D sample image;
determining an original cube from the 3D sample image; and
splitting the original cube into the N target cubes.

### Claim 3 (depends on 2)

The method according to claim 2, wherein a value of N is a third power of a positive integer greater than 1, and splitting the original cube into the N target cubes comprises:
splitting the N target cubes from the original cube while keeping an interval of M voxels between two adjacent target cubes, M being a positive integer less than J−1, J being a side length of the target cube.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the method further comprises:
determining a first target cube from the N target cubes;
rotating the first target cube by a first angle; and
rearranging the first target cube rotated by the first angle with other target cubes in the N target cubes based on the target permutation sequence to obtain the target training sample.

### Claim 5 (depends on 1)

The method according to claim 1, wherein before obtaining the target 3D image, the method further comprises:
obtaining a recognition result outputted after the original recognition model recognizes the target training sample, wherein the recognition result comprises probabilities of multiple sorting sequences of the target cubes in the target training sample and a rotation angle of each target cube; and
determining the original recognition model as the second recognition model when a probability that the recognition result meets a first target function is greater than a first threshold.

### Claim 6 (depends on 1)

The method according to claim 1, wherein before obtaining the target 3D image, the method further comprises:
determining the convolutional block of the second recognition model as the convolutional block of the first recognition model; and
training the first recognition model by using a first training sample until accuracy of the first recognition model is greater than a target threshold, the first training sample comprising a first 3D image and a type of the first 3D image.

### Claim 7 (depends on 1)

The method according to claim 1, wherein similarities among the candidate permutation sequences are greater than a similarity measurement threshold.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the similarities among the candidate permutation sequences are measured by Hamming distance.

### Claim 9

An image recognition apparatus, comprising a memory and a processor, the memory storing a computer program, when executing the computer program, the processor being configured top:
obtain a target 3D image;
input the target 3D image to a first recognition model, the first recognition model being configured to perform image recognition on the target 3D image to obtain a type of the target 3D image; and
obtain the type of the target 3D image outputted by the first recognition model,
wherein the first recognition model is obtained by:
determining N target cubes from a 3D sample image to be rotated, N being a natural number greater than 1;
determining a plurality of candidate permutation sequences of the N target cubes;
obtaining, based on the 3D sample image, a target training sample, the target training sample comprising cubes obtained by rotating and sorting the N target cubes based on a target permutation sequence randomly selected from the candidate permutation sequences;
using the target training sample without tag annotation to train an original recognition model and obtain a convolutional block of the second recognition model; and
after the second recognition model is trained, using the 3D sample image with the tag annotation and using the convolutional block of the trained second recognition model as a convolutional block of the first recognition model, to train the first recognition model, the tag annotation including a type of the 3D sample.

### Claim 10 (depends on 9)

The apparatus according to claim 9, wherein the processor is further configured to:
obtain the 3D sample image before the target 3D image is obtained;
determine an original cube from the 3D sample image; and
split the original cube into the N target cubes.

### Claim 11 (depends on 10)

The apparatus according to claim 10, wherein a value of N is a third power of a positive integer greater than 1, and the splitting the original cube into the N target cubes comprises:
splitting the N target cubes from the original cube while keeping an interval of M voxels between two adjacent target cubes, M being a positive integer less than J−1, J being a side length of the target cube.

### Claim 12 (depends on 9)

The apparatus according to claim 9, wherein the processor is further configured to:
determine a first target cube from the N target cubes;
rotate the first target cube by a first angle; and
rearrange the first target cube rotated by the first angle with other target cubes in the N target cubes based on the target permutation sequence to obtain the target training sample.

### Claim 13 (depends on 9)

The apparatus according to claim 9, wherein before obtaining the target 3D image, the processor is further configured to:
obtain a recognition result outputted after the original recognition model recognizes the target training sample, wherein the recognition result comprises probabilities of multiple sorting sequences of the target cubes in the target training sample and a rotation angle of each target cube; and
determine the original recognition model as the second recognition model when a probability that the recognition result meets a first target function is greater than a first threshold.

### Claim 14 (depends on 9)

The apparatus according to claim 9, wherein before obtaining the target 3D image, the processor is further configured to:
determine the convolutional block of the second recognition model as the convolutional block of the first recognition model; and
train the first recognition model by using a first training sample until accuracy of the first recognition model is greater than a target threshold, the first training sample comprising a first 3D image and a type of the first 3D image.

### Claim 15

A non-transitory computer storage medium, storing a computer program, the computer program, when executed, causing a processor to perform:
obtaining a target three-dimensional (3D) image;
inputting the target 3D image to a first recognition model, the first recognition model being configured to perform image recognition on the target 3D image to obtain a type of the target 3D image; and
obtaining the type of the target 3D image outputted by the first recognition model,
wherein the first recognition model is obtained by:
determining N target cubes from a 3D sample image to be rotated, N being a natural number greater than 1;
determining a plurality of candidate permutation sequences of the N target cubes;
obtaining, based on the 3D sample image, a target training sample, the target training sample comprising cubes obtained by rotating and sorting the N target cubes based on a target permutation sequence randomly selected from the candidate permutation sequences;
using the target training sample without tag annotation to train an original recognition model and obtain a convolutional block of the second recognition model; and
after the second recognition model is trained, using the 3D sample image with the tag annotation and using the convolutional block of the trained second recognition model as a convolutional block of the first recognition model, to train the first recognition model, the tag annotation including a type of the 3D sample.

### Claim 16 (depends on 15)

The storage medium according to claim 15, wherein before obtaining the target 3D image, the computer program further causes the processor to perform:
obtaining the 3D sample image;
determining an original cube from the 3D sample image; and
splitting the original cube into the N target cubes.

### Claim 17 (depends on 16)

The storage medium according to claim 16, wherein a value of N is a third power of a positive integer greater than 1, and splitting the original cube into the N target cubes comprises:
splitting the N target cubes from the original cube while keeping an interval of M voxels between two adjacent target cubes, M being a positive integer less than J−1, J being a side length of the target cube.

### Claim 18 (depends on 15)

The storage medium according to claim 15, wherein the computer program further causes the processor to perform:
determining a first target cube from the N target cubes;
rotating the first target cube by a first angle; and
rearranging the first target cube rotated by the first angle with other target cubes in the N target cubes based on the target permutation sequence to obtain the target training sample.

### Claim 19 (depends on 15)

The storage medium according to claim 15, wherein before obtaining the target 3D image, the computer program further causes the processor to perform:
obtaining a recognition result outputted after the original recognition model recognizes the target training sample, wherein the recognition result comprises probabilities of multiple sorting sequences of the target cubes in the target training sample and a rotation angle of each target cube; and
determining the original recognition model as the second recognition model when a probability that the recognition result meets a first target function is greater than a first threshold.