# US 2021/0279513 A1 · Target detection method and apparatus, model training method and apparatus, device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279513 A1
- Document ID: US20210279513A1
- Status: Published application
- Kind: A1
- Application number: 17/313,497
- Filed: 2021-05-06
- Published: 2021-09-09
- Earliest claimed: 2019-03-21
- Assignees: Tecent Technology (Shenzhen) Company Limited
- Applicants: Tecent Technology (Shenzhen) Company Limited
- Inventors: Ze Qun JIE
- Web page: https://patentgrep.com/patent/US20210279513A1

## CPC

- G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The disclosure provides a target detection method and apparatus, a model training method and apparatus, a device, and a storage medium. The target detection method includes: obtaining a first image; obtaining a second image corresponding to the first image, the second image belonging to a second domain; and obtaining a detection result corresponding to the second image through a cross-domain image detection model, the detection result including target localization information and target class information of a target object, the cross-domain image detection model including a first network model configured to convert an image from a first domain into an image in the second domain, and the second network model configured to perform region localization on the image in the second domain.

## Claims

### Claim 1

A model training method, comprising:
obtaining a test style image corresponding to a training image through a first network model, and obtaining a training style image corresponding to a test image through the first network model, the training image and the training style image belonging to a first domain, the test image and the test style image belonging to a second domain, and the first domain and the second domain being different domains;
obtaining, through a second network model, a predicted probability value corresponding to a probability of a candidate region with respect to each class, the candidate region being a candidate bounding box extracted from the test image or the test style image;
obtaining a first prediction class probability value and a first prediction localization value, which are prediction values corresponding to the test style image and outputted by a region proposal network (RPN) of the second network model, and obtaining a second prediction class probability value and a second prediction localization value, which are prediction values corresponding to the test style image outputted by a target detection network of the second network model; and
training the first network model and the second network model by using a target loss function and according to the predicted probability value, the test style image, the training style image, the first prediction class probability value, the first prediction localization value, the second prediction class probability value, and the second prediction localization value, to obtain a cross-domain image detection model.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the training the first network model and the second network model comprises:
determining a first loss function according to the test style image and the training style image;
determining a second loss function according to the predicted probability value;
determining a third loss function according to the first prediction class probability value, the first prediction localization value, the second prediction class probability value, and the second prediction localization value;
generating the target loss function according to the first loss function, the second loss function, and the third loss function; and
training the first network model and the second network model by using the target loss function.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the determining the first loss function comprises:
determining the first loss function in the following manner:
Lcyc(Gt-s,Gs-t)=Et˜X(t)[∥Gs-t(Gt-s(xt))−xt∥]+Es˜X(s)[∥Gt-s(Gs-t(xs))−xs∥],
wherein Lcyc(Gt-s,Gs-t) represents the first loss function, xt represents the test image, xs represents the training image, Gs-t represents a generator for converting the training image into the test style image, Gt-s represents a generator for converting the test image into the training style image, Gt-s(xt) represents the training style image, Gs-t(xs) represents the test style image, Es˜X(s) represents calculating an expected value of each training image in a training image set, and Et˜X(t) represents calculating an expected value of each test image in a test image set.

### Claim 4 (depends on 1)

The method according to claim 1, further comprising, after the obtaining the test style image and the training style image:
performing reality detection on the test style image through a first discriminator that is obtained based on training by using a first adversarial loss function; and
performing reality detection on the training style image through a second discriminator that is obtained based on training by using a second adversarial loss function.

### Claim 5 (depends on 4)

The method according to claim 4, further comprising:
obtaining the first adversarial loss function in the following manner:
LGAN(Dt,Gs-t)=Et˜X(t)[log Dt(x)]+Es˜X(s)[log(1−Dt(Gs-t(s)))],
wherein LGAN (Dt,Gs-t) represents the first adversarial loss function, Es˜X(s) represents calculating an expected value of each training image in a training image set, Et˜X(t) represents calculating an expected value of each test image in a test image set, Dt( ) represents the first discriminator, s represents an image in the first domain, Gs-t( ) represents a network for converting the image in the first domain into an image in the second domain, and x represents an inputted image.

### Claim 6 (depends on 4)

The method according to claim 4, further comprising:
obtaining the second adversarial loss function in the following manner:
LGAN(Ds,Gt-s)=Es˜X(s)[log Ds(x)]+Et˜X(t)[log Ds(Gt-s(t)))],
wherein LGAN (Ds, Gt-s) represents the second adversarial loss function, Es˜X(s) represents calculating an expected value of each training image in a training image set, Et˜X(t) represents calculating an expected value of each test image in a test image set, Ds( ) represents the second discriminator, t represents an image in the second domain, Gt-s( ) represents a network for converting the image in the second domain into an image in the first domain, and x represents an inputted image.

### Claim 7 (depends on 2)

The method according to claim 2, wherein the determining the second loss function comprises determining the second loss function in the following manner:
Ldomain=Σi,j[d log pi,j+(1−d)log(1−pi,j)],
wherein Ldomain represents the second loss function, pi,j represents a predicted probability value of an ith candidate region on a jth class, and d represents an image source value.

### Claim 8 (depends on 2)

The method according to claim 2, wherein the determining the third loss function comprises determining the third loss function in the following manner:
Ldet(p,u,tm,v)+Lrpn-cls(prpn,urpn)+Lrpn-loc(trpnm,vrpn)+Lobj-cis(pobj,uobj)+Lobj-loc(tobjm,vobj),
where L (p,u,tm,v)det represents the third loss function, Lrpn-cls represents a classification loss function of the RPN in the second network model, prpn represents the first prediction class probability value of the RPN, urpn represents a first real class probability value of the RPN, Lrpn-loc represents a localization loss function of the RPN in the second network model, trpnm represents the first prediction localization value of the RPN, vrpn represents a first real localization value of the RPN, Lobj-cls represents a classification loss function of the target detection network in the second network model, pobj represents the second prediction class probability value of the target detection network, uobj represents a second real class probability value of the target detection network, Lobj-loc represents a localization loss function of the target detection network in the second network model, tobjm represents the second prediction localization value of the target detection network, and vobj represents a second real localization value of the target detection network.

### Claim 9 (depends on 2)

The method according to claim 2, wherein the generating the target loss function comprises determining the target loss function in the following manner:
L=Lcyc(Gt-s,Gs-t)+Ldomain+Ldet,
wherein L represents the target loss function, Lcyc(Gt-s,Gs-t) represents the first loss function, Ldomain represents the second loss function, and Ldet represents the third loss function.

### Claim 10 (depends on 1)

The method according to claim 1, further comprising:
obtaining a first image belonging to the first domain;
obtaining a second image corresponding to the first image, the second image belonging to the second domain; and
obtaining a detection result corresponding to the second image through the cross-domain image detection model, the detection result comprising target localization information and target class information of a target object, the target object being a detected object in the first image, the cross-domain image detection model comprising the first network model and the second network model.

### Claim 11

A model training apparatus, comprising:
at least one memory configured to store program code; and
at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising:
first obtaining code configured to cause at least one of the at least one processor to obtain a test style image corresponding to a training image through a first network model, and obtain a training style image corresponding to a test image through the first network model, the training image and the training style image belonging to a first domain, the test image and the test style image belonging to a second domain, and the first domain and the second domain being different domains
second obtaining code configured to cause at least one of the at least one processor to, through a second network model, obtain a predicted probability value corresponding to a probability of a candidate region with respect to each class, the candidate region being a candidate bounding box extracted from the test image or the test style image;
third obtaining code configured to cause at least one of the at least one processor to obtain a first prediction class probability value and a first prediction localization value, which are prediction values corresponding to the test style image and outputted by a region proposal network (RPN) of the second network model, and obtain a second prediction class probability value and a second prediction localization value, which are prediction values corresponding to the test style image outputted by a target detection network of the second network model; and
training code configured to cause at least one of the at least one processor to train the first network model and the second network model by using a target loss function and according to the predicted probability value, the test style image, the training style image, the first prediction class probability value, the first prediction localization value, the second prediction class probability value, and the second prediction localization value, to obtain a cross-domain image detection model.

### Claim 12

A terminal device comprising:
at least one memory configured to store program code; and
at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising:
first obtaining code configured to cause at least one of the at least one processor to obtain a first image belonging to a first domain, a second image corresponding to the first image, the second image belonging to a second domain, and the first domain and the second domain being different domains;
second obtaining code configured to cause at least one of the at least one processor to obtain a detection result corresponding to the second image through a cross-domain image detection model, the detection result comprising target localization information and target class information of a target object, the target object being a detected object in the first image, the cross-domain image detection model comprising a first network model and a second network model, the first network model configured to convert an image from the first domain into an image in the second domain, and the second network model configured to perform region localization on the image in the second domain.

### Claim 13 (depends on 12)

The terminal device according to claim 12, wherein the program code further comprises:
third obtaining code configured to cause at least one of the at least one processor to obtain a test style image corresponding to a training image through the first network model, and obtaining a training style image corresponding to the test image through the first network model, the training image and the training style image belonging to the first domain, the test image and the test style image belonging to the second domain;
first determining code configured to cause at least one of the at least one processor to determine a first loss function according to the test style image and the training style image;
second determining code configured to cause at least one of the at least one processor to determine a second loss function according to the predicted probability value;
third determining code configured to cause at least one of the at least one processor to determine a third loss function according to a first prediction class probability value and a first prediction localization value, which are prediction values corresponding to the test style image and outputted by a region proposal network (RPN) of the second network model, and obtaining a second prediction class probability value and a second prediction localization value, which are prediction values corresponding to the test style image outputted by a target detection network of the second network model;
generating code configured to cause at least one of the at least one processor to generate a target loss function according to the first loss function, the second loss function, and the third loss function; and
training code configured to cause at least one of the at least one processor to train the first network model and the second network model by using the target loss function.

### Claim 14 (depends on 13)

The terminal device according to claim 13, wherein the first determining code is further configured to cause at least one of the at least one processor to determine the first loss function in the following manner:
Lcyc(Gt-s,Gs-t)=Et˜X(t)[∥Gs-t(Gt-s(xt))−xt∥]+Es˜X(s)[∥Gt-s(Gs-t(xs))−xs∥],
wherein Lcyc(Gt-s,Gs-t) represents the first loss function, xt represents the test image, xs represents the training image, Gs-t represents a generator for converting the training image into the test style image, Gt-s represents a generator for converting the test image into the training style image, Gt-s(xt) represents the training style image, Gs-t(xs) represents the test style image, Es˜X(s) represents calculating an expected value of each training image in a training image set, and Et˜X(t) represents calculating an expected value of each test image in a test image set.

### Claim 15 (depends on 13)

The terminal device according to claim 13, wherein the program code further comprises:
first reality detection determining code configured to cause at least one of the at least one processor to perform reality detection on the test style image through a first discriminator, wherein the first discriminator is obtained based on training by using a first adversarial loss function; and
second reality detection determining code configured to cause at least one of the at least one processor to perform reality detection on the training style image through a second discriminator, wherein the second discriminator is obtained based on training by using a second adversarial loss function.

### Claim 16 (depends on 15)

The terminal device according to claim 15, wherein the program code further comprises fourth obtaining code configured to cause at least one of the at least one processor to obtain the first adversarial loss function in the following manner:
LGAN(Dt,Gs-t)=Et˜X(t)[log Dt(x)]+Es˜X(s)[log(1−Dt(Gs-t(s)))],
wherein LGAN (Dt,Gs-t) represents the first adversarial loss function, Es˜X(s) represents calculating an expected value of each training image in a training image set, Et˜X(t) represents calculating an expected value of each test image in a test image set, Dt( ) represents the first discriminator, s represents an image in the first domain, Gs-t( ) represents a network for converting the image in the first domain into an image in the second domain, and x represents an inputted image.

### Claim 17 (depends on 13)

The terminal device according to claim 13, wherein the second determining code is further configured to cause at least one of the at least one processor to determine the second loss function in the following manner:
Ldomain=Σi,j[d log pi,j+(1−d)log(1−pi,j)],
wherein Ldomain represents the second loss function, pi,j represents a predicted probability value of an ith candidate region on a jth class, and d represents an image source value.

### Claim 18 (depends on 13)

The terminal device according to claim 13, wherein the third determining code is further configured to cause at least one of the at least one processor to determine the third loss function in the following manner:
Ldet(p,u,tm,v)+Lrpn-cls(prpn,urpn)+Lrpn-loc(trpnm,vrpn)+Lobj-cis(pobj,uobj)+Lobj-loc(tobjm,vobj),
where L (p,u,tm,v)det represents the third loss function, Lrpn-cls represents a classification loss function of the RPN in the second network model, prpn represents the first prediction class probability value of the RPN, urpn represents a first real class probability value of the RPN, Lrpn-loc represents a localization loss function of the RPN in the second network model, trpnm represents the first prediction localization value of the RPN, vrpn represents a first real localization value of the RPN, Lobj-cls represents a classification loss function of the target detection network in the second network model, pobj represents the second prediction class probability value of the target detection network, uobj represents a second real class probability value of the target detection network, Lobj-loc represents a localization loss function of the target detection network in the second network model, tobjm represents the second prediction localization value of the target detection network, and vobj represents a second real localization value of the target detection network.

### Claim 19 (depends on 1)

A non-transitory computer-readable storage medium storing program code, the program code being executable by at least one processor to cause the at least one processor to perform operations of the method according to claim 1.

### Claim 20 (depends on 10)

A non-transitory computer-readable storage medium storing program code, the program code being executable by at least one processor to cause the at least one processor to perform operations of the method according to claim 10.
