# US 10,769,493 B2 · Method and apparatus for neural network training and construction and method and apparatus for object detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,769,493 B2
- Document ID: US10769493B2
- Status: Granted
- Kind: B2
- Application number: 15/660,494
- Filed: 2017-07-26
- Published: 2018-02-01
- Granted: 2020-09-08
- Earliest claimed: 2016-07-27
- Assignees: Beijing Kuangshi Technology Co., Ltd.; Megvii (Beijing) Technology Co., Ltd.
- Applicants: Beijing Kuangshi Technology Co., Ltd.; Megvii (Beijing) Technology Co., Ltd.
- Inventors: Jiahui Yu; Qi Yin
- Web page: https://patentgrep.com/patent/US10769493B2

## CPC

- G06V 10/82: using neural networks
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/24: Classification techniques
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 7/11: Region-based segmentation
- G06T 7/337: {involving reference images or patches}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 40/172: {Classification, e.g. identification}
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20132: Image cropping
- G06T 2207/30201: Face
- G06T 2210/12: Bounding box
- G06V 40/168: {Feature extraction; Face representation}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The embodiments of the present invention provide training and construction methods and apparatus of a neural network for object detection, an object detection method and apparatus based on a neural network and a neural network. The training method of the neural network for object detection, comprises: inputting a training image including a training object to the neural network to obtain a predicted bounding box of the training object; acquiring a first loss function according to a ratio of the intersection area to the union area of the predicted bounding box and a true bounding box, the true bounding box being a bounding box of the training object marked in advance in the training image; and adjusting parameters of the neural network by utilizing at least the first loss function to train the neural network.

## Claims

### Claim 1

A training method of a neural network for object detection, comprising:
inputting a training image including a training object to the neural network to obtain a predicted bounding box of the training object;
acquiring a first loss function according to a ratio of an intersection area to a union area of the predicted bounding box and a true bounding box, wherein the true bounding box is a bounding box of the training object marked in advance in the training image; and
adjusting parameters of the neural network by utilizing at least the first loss function to train the neural network;
wherein the first loss function is a negative value of a natural logarithm of the ratio of the intersection area to the union area of the predicted bounding box and the true bounding box;
wherein the method further comprising:
selecting a second loss function reflecting a difference between a predicted confidence and a true confidence of each pixel point in the training image, the predicted confidence being the confidence, predicted by employing the neural network, that a certain pixel point in the training image belongs to the training object, and the true confidence representing the confidence, marked in advance in the training image, that the certain pixel point belongs to the training object,
wherein said adjusting the parameters of the neural network by utilizing at least the first loss function to train the neural network comprises:
adjusting the parameters of the neural network by utilizing the first loss function and the second loss function to maximize a ratio of an intersection area to an union area of the predicted bounding box and the true bounding box and to minimize the second loss function so as to obtain the trained neural network.

### Claim 2 (depends on 1)

The training method according to claim 1, wherein said adjusting the parameters of the neural network by utilizing at least the first loss function to train the neural network comprises:
adjusting the parameters of the neural network by utilizing at least the first loss function to maximize the ratio of the intersection area to the union area of the predicted bounding box and the true bounding box.

### Claim 3 (depends on 1)

The training method according to claim 1, wherein said adjusting the parameters of the neural network by utilizing at least the first loss function to train the neural network comprises:
determining values of the parameters when a number of training rounds reaches to a set threshold as values of parameters of the trained neural network.

### Claim 4 (depends on 1)

The training method according to claim 1, wherein said acquiring the first loss function according to the ratio of the intersection area to the union area of the predicted bounding box and the true bounding box comprises:
for at least one pixel point located within the true bounding box in the training image, determining a first 4-dimensional vector and a second 4-dimensional vector, wherein elements in the first 4-dimensional vector are distances from the pixel point to the top, bottom, left and right bounds of the true bounding box respectively, and elements in the second 4-dimensional vector represent distances from the pixel point to the top, bottom, left and right bounds of the predicted bounding box corresponding to the pixel point respectively;
acquiring an area of the true bounding box according to the first 4-dimensional vector;
acquiring an area of the predicted bounding box according to the second 4-dimensional vector;
acquiring an intersection area of the predicted bounding box and the true bounding box according to the first 4-dimensional vector and the second 4-dimensional vector; and
taking a difference between a sum of the acquired area of the predicted bounding box and the acquired area of the true bounding box and the acquired intersection area as the union area.

### Claim 5 (depends on 1)

The training method according to claim 1, wherein
the neural network comprises at least one input layer, n intermediate layers and two output layers, wherein n is a positive integer greater than or equal to 2, and the two output layers comprise a confidence output layer and a bounding box output layer,
wherein in a training step, the input layer is configured to receive the training image, the confidence output layer is configured to output a predicted confidence map which represents the confidence, predicted by employing the neural network, that each pixel point in the training image including the training object belongs to the training object,
the bounding box output layer is configured to output a bounding box prediction map which contains a 4-dimensional vector of each pixel point located in the training image, wherein, when the pixel point is located within the true bounding box in the training image, elements in the 4-dimensional vector of the pixel point are distances from the pixel point to the top, bottom, left and right bounds of the predicted bounding box corresponding to the pixel point.

### Claim 6 (depends on 5)

The training method according to claim 5, further comprising:
for each input training image, inputting to the input layer a true confidence map and a training bounding box map whose sizes are same as that of the training image, wherein the training bounding box map contains a 4-dimensional vector of each pixel point located in the training image, wherein, when the pixel point is located within the true bounding box in the training image, elements in the 4-dimensional vector corresponding to the pixel point are distances from the pixel point to the top, bottom, left and right bounds of the true bounding box, and each pixel point in the true confidence map represents the confidence that the corresponding pixel in the training image belongs to the training object.

### Claim 7 (depends on 5)

The training method according to claim 5, wherein the m-th intermediate layer of the neural network is connected to the confidence output layer in cascade, the n-th intermediate layer is connected to the bounding box output layer in cascade, and m is less than n.