# US 2022/0036187 A1 · Sample generation method and apparatus, computer device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036187 A1
- Document ID: US20220036187A1
- Status: Published application
- Kind: A1
- Application number: 17/502,847
- Filed: 2021-10-15
- Published: 2022-02-03
- Earliest claimed: 2019-08-28
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Dong WEI; Kai MA; Yefeng ZHENG
- Web page: https://patentgrep.com/patent/US20220036187A1

## CPC

- G06N 3/08: Learning methods
- G06F 18/211: Selection of the most significant subset of features
- G06F 18/213: Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/2155: {characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling}
- G06F 18/25: Fusion techniques
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A sample generation method outputs a dummy sample set generated by a trained sample generation network that operates on spliced vectors formed by combining real category feature vectors extracted from real samples with real category label vectors corresponding to the real samples. The trained sample generation network is trained using real samples and dummy samples that are generated by an intermediate sample generation network operating on the spliced vectors. The training includes inputting the real samples and the dummy samples to an intermediate sample discrimination network, performing iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network until an iteration stop condition is met. As a result, the dummy sample set output by the trained sample generation network includes dummy samples that are not easily differentiated from real samples and that are already labeled with category information, for accurate use in training classifiers.

## Claims

### Claim 1

A sample generation method comprising:
obtaining real category feature vectors extracted from real samples respectively;
combining the real category feature vectors with real category label vectors corresponding to the real samples to obtain spliced vectors;
inputting the spliced vectors to an intermediate sample generation network, to obtain dummy samples through mapping;
determining mutual information between the dummy samples and the corresponding spliced vectors;
by processing circuitry of a computer device, inputting the real samples and the dummy samples to an intermediate sample discrimination network, performing iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network with reference to the mutual information, and iteratively maximizing the mutual information during the adversarial training until an iteration stop condition is met; and
inputting the spliced vectors to a trained sample generation network in response to a determination that the iteration stop condition is met, to output a dummy sample set.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the method further comprises:
obtaining real noise vectors in the real samples;
the combining comprises:
combining the real category feature vectors with the corresponding real category label vectors and the real noise vectors; the determining comprises:
determining mutual information between the dummy samples and corresponding real category feature combination vectors, the real category feature combination vectors being obtained by combining the real category feature vectors and the corresponding real category label vectors.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the method further comprises:
extracting feature vectors from the real samples respectively;
performing feature decomposition on the feature vectors, and sifting feature vectors for category discrimination, to obtain the real category feature vectors; and
obtaining feature vectors that remain after the sifting, to obtain the real noise vectors.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the performing comprises:
pre-assigning, for each real sample, corresponding category feature coefficients to feature sub-vectors in a feature vector of the real sample, to obtain category feature coefficient vectors;
constructing an objective function of the category feature coefficient vectors according to a sum of squared residuals of products between a real category label vector of the real sample and the feature sub-vectors;
iteratively adjusting the category feature coefficients of the feature sub-vectors in the category feature coefficient vectors, to iteratively search for a minimum value of the objective function of the category feature coefficient vectors; and
forming, in response to a determination that the iterative adjustment has stopped, the real category feature vector according to feature sub-vectors having category feature coefficients that are not 0.

### Claim 5 (depends on 2)

The method according to claim 2, wherein the method further comprises:
sampling from a first probability distribution according to a preset dummy category label vector, to obtain dummy category feature vectors, the first probability distribution being obtained by fitting the real category feature vectors; and
combining the dummy category feature vectors and the corresponding dummy category label vector, to obtain dummy spliced vectors, and inputting the dummy spliced vectors to the intermediate sample generation network, to output the dummy samples; and
the determining comprises:
determining mutual information between the dummy samples and the corresponding dummy spliced vectors.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the dummy spliced vectors further comprise dummy noise vectors, and the method further comprises:
obtaining the real noise vectors in the real samples, the real noise vectors being different from the real category feature vectors;
fitting the real noise vectors, to obtain a second probability distribution; and
sampling from the second probability distribution, to obtain the dummy noise vectors; and
the determining mutual information between the dummy samples and the corresponding dummy spliced vectors comprises:
determining mutual information between the dummy samples and corresponding dummy category feature combination vectors, the dummy category feature combination vectors being obtained by combining the dummy category feature vectors and the dummy category label vectors.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the inputting the spliced vectors to the trained sample generation network in response to the determination that the iteration stop condition is met, to output a dummy sample set comprises:
using the real category feature vectors and the dummy category feature vectors as levels in a category feature vector factor and the real noise vectors and the dummy noise vectors as levels in a noise vector factor, and performing cross-grouping on the levels in the category feature vector factor and on the levels in the noise vector factor, to obtain to-be-spliced combinations;
obtaining a category label vector corresponding to a category feature vector comprised in each to-be-spliced combination;
combining vectors comprised in each to-be-spliced combination with the corresponding category label vector; and
inputting vectors obtained by the combining the vectors in the same to-be-spliced combinations to the trained sample generation network in response to the determination that the iteration stop condition is met, to output final dummy samples.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the to-be-spliced combinations comprise at least one of the following combinations:
a to-be-spliced combination comprising a real noise vector, a real category feature vector, and a real label vector corresponding to the real category feature vector;
a to-be-spliced combination comprising a dummy noise vector, a dummy category feature vector, and a dummy label vector corresponding to the dummy category feature vector;
a to-be-spliced combination comprising a dummy noise vector, a real category feature vector, and a real label vector corresponding to the real category feature vector; and
a to-be-spliced combination comprising a real noise vector, a dummy category feature vector, and a dummy label vector corresponding to the dummy category feature vector.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the inputting the real samples and the dummy samples to the intermediate sample discrimination network, the performing the iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network with reference to the mutual information comprises:
inputting the real samples and the dummy samples to the intermediate sample discrimination network, to construct an objective function of the intermediate sample generation network and an objective function of the intermediate sample discrimination network, and performing iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network with reference to the mutual information serving as a regularization function, to iteratively search for a minimum value of the objective function of the intermediate sample generation network, search for a maximum value of the objective function of the intermediate sample discrimination network, and maximize the regularization function.

### Claim 10 (depends on 1)

The method according to claim 1, wherein a spliced vector comprising a real category feature vector and a corresponding real category label vector is a real spliced vector;
the method further includes:
obtaining a dummy sample obtained by mapping the real spliced vector, to obtain a reconstructed dummy sample;
obtaining a reconstruction loss function, the reconstruction loss function being configured to represent a difference between the reconstructed dummy sample and a corresponding real sample; and
the inputting the real samples and the dummy samples to the intermediate sample discrimination network, the performing the iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network with reference to the mutual information, and the iteratively maximizing the mutual information during the adversarial training until an iteration stop condition is met includes:
inputting the real samples and the dummy samples to the intermediate sample discrimination network, performing the iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network with reference to the mutual information and the reconstruction loss function, and iteratively maximizing the mutual information and searching for a minimum value of the reconstruction loss function during the adversarial training until the iteration stop condition is met.

### Claim 11 (depends on 1)

The method according to claim 1, wherein each real category feature vector includes a plurality of features used for representing real sample categories, and the method further includes:
selecting target features from the real category feature vectors;
evenly selecting feature values within a preset range for the target features, and maintaining feature values of non-target features of the real category feature vectors unchanged, to obtain a plurality of associated real category feature vectors; and
inputting the plurality of associated real category feature vectors to the trained sample generation network in response to the determination that the iteration stop condition is met, to output associated dummy samples.

### Claim 12 (depends on 1)

The method according to claim 1, wherein each real sample is a real image sample, each dummy sample is a dummy image sample, and each real category feature vector is a feature vector used for discriminating a category of the real image sample.

### Claim 13

A sample generation method comprising:
obtaining real category feature vectors extracted from real medical image samples respectively;
combining the real category feature vectors with real category label vectors corresponding to the real medical image samples to obtain spliced vectors;
inputting spliced vectors to an intermediate sample generation network, to output dummy medical image samples;
determining mutual information between the dummy medical image samples and the corresponding spliced vectors;
by processing circuitry of a computer device, inputting the real medical image samples and the dummy medical image samples to an intermediate sample discrimination network, performing iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network with reference to the mutual information, and iteratively maximizing the mutual information during the adversarial training until an iteration stop condition is met; and
inputting the spliced vectors to a trained sample generation network in response to a determination that the iteration stop condition is met, to output final dummy medical image samples.

### Claim 14 (depends on 13)

The method according to claim 13, wherein the method further comprises:
obtaining real noise vectors in the real medical image samples;
the combining includes:
combining the real category feature vectors with the corresponding real category label vectors and the real noise vectors; and
the determining includes:
determining mutual information between the dummy medical image sample and corresponding real category feature combination vectors, the real category feature combination vectors being obtained by combining the real category feature vectors and the corresponding real category label vectors.

### Claim 15 (depends on 14)

The method according to claim 14, wherein the method further comprises:
extracting feature vectors from the real medical image samples respectively;
performing feature decomposition on the feature vectors, and sifting feature vectors for category discrimination, to obtain the real category feature vectors; and
obtaining feature vectors that remain after the sifting, to obtain the real noise vectors.

### Claim 16 (depends on 14)

The method according to claim 14, wherein the method further comprises:
sampling from a first probability distribution according to a preset dummy category label vector, to obtain dummy category feature vectors, the first probability distribution being obtained by fitting the real category feature vectors; and
combining the dummy category feature vectors and the corresponding dummy category label vector, to obtain dummy spliced vectors, and inputting the dummy spliced vectors to the intermediate medical image sample generation network, to output dummy medical image samples; and
the determining includes:
determining mutual information between the dummy medical image samples and corresponding dummy spliced vectors.

### Claim 17

A sample generation apparatus comprising:
processing circuitry configured to
obtain real category feature vectors extracted from real samples respectively;
combine the real category feature vectors with real category label vectors corresponding to the real samples to obtain spliced vectors;
input the spliced vectors to an intermediate sample generation network, to obtain dummy samples through mapping;
determine mutual information between the dummy samples and the corresponding spliced vectors;
input the real samples and the dummy samples to an intermediate sample discrimination network, perform iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network with reference to the mutual information, and iteratively maximize the mutual information during the adversarial training until an iteration stop condition is met; and
input the spliced vectors to a trained sample generation network in response to a determination that the iteration stop condition is met, to output a dummy sample set.

### Claim 18 (depends on 17)

The apparatus according to claim 17, wherein
the processing circuitry is further configured to obtain real noise vectors in the real samples and combine the real category feature vectors with the corresponding real category label vectors and the real noise vectors to obtain the spliced vectors; and
the processing circuitry is further configured to determine mutual information between the dummy samples and corresponding real category feature combination vectors, the real category feature combination vectors being obtained by combining the real category feature vectors and the corresponding real category label vectors.

### Claim 19 (depends on 18)

The apparatus according to claim 18, wherein the processing circuitry is further configured to
extract feature vectors from the real samples respectively;
perform feature decomposition on the feature vectors, and sift feature vectors for category discrimination, to obtain the real category feature vectors; and
obtain feature vectors that remain after the sifting, to obtain the real noise vectors.

### Claim 20 (depends on 19)

The apparatus according to claim 19, wherein the processing circuitry is further configured to
pre-assign, for each real sample, corresponding category feature coefficients to feature sub-vectors in a feature vector of the real sample, to obtain category feature coefficient vectors;
construct an objective function of the category feature coefficient vectors according to a sum of squared residuals of products between a real category label vector of the real sample and the feature sub-vectors;
iteratively adjust the category feature coefficients of the feature sub-vectors in the category feature coefficient vectors, to iteratively search for a minimum value of the objective function of the category feature coefficient vectors; and
form, in response to a determination that the iterative adjustment has stopped, the real category feature vector according to feature sub-vectors having category feature coefficients are not 0.
