# US 2021/0241098 A1 · Method and apparatus with neural network meta-training and class vector training

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241098 A1
- Document ID: US20210241098A1
- Status: Published application
- Kind: A1
- Application number: 17/119,381
- Filed: 2020-12-11
- Published: 2021-08-05
- Earliest claimed: 2020-02-05
- Assignees: Samsung Electronics Co., Ltd.; Korea Advanced Institute of Science and Technology
- Applicants: Samsung Electronics Co., Ltd.; Korea Advanced Institute of Science and Technology
- Inventors: Seong-Jin PARK; Sung Ju HWANG; Seungju HAN; Insoo KIM; Jiwon BAEK; Jaejoon HAN
- Web page: https://patentgrep.com/patent/US20210241098A1

## CPC

- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06F 18/213: Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F 18/2431: Multiple classes
- G06N 3/045: Combinations of networks
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 40/168: {Feature extraction; Face representation}
- G06V 40/174: {Facial expression recognition}
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A processor-implemented neural network method includes: extracting, by a feature extractor of a neural network, a plurality of training feature vectors corresponding to a plurality of training class data of each of a plurality of classes including a first class and a second class; determining, by a feature sample generator of the neural network, an additional feature vector of the second class based on a mean vector and a variation vector of the plurality of training feature vectors of each of the first class and the second class; and training a class vector of the second class included in a classifier of the neural network based on the additional feature vector and the plurality of training feature vectors of the second class.

## Claims

### Claim 1

A processor-implemented neural network method, the method comprising:
extracting, by a feature extractor of a neural network, a plurality of training feature vectors corresponding to a plurality of training class data of each of a plurality of classes including a first class and a second class;
determining, by a feature sample generator of the neural network, an additional feature vector of the second class based on a mean vector and a variation vector of the plurality of training feature vectors of each of the first class and the second class; and
training a class vector of the second class included in a classifier of the neural network based on the additional feature vector and the plurality of training feature vectors of the second class.

### Claim 2 (depends on 1)

The method of claim 1, wherein the training comprises training the class vector of the second class to reduce a distance between the class vector of the second class and a feature vector of training input data corresponding to the second class with respect to a distance between a class vector of each of the plurality of classes and the feature vector of the training input data.

### Claim 3 (depends on 2)

The method of claim 2, wherein
the training comprises training the class vector of the second class using a loss function, and
the loss function includes a ratio of an exponential value of a negative distance of a correct answer class to a summation of exponential values of respective negative distances of the plurality of classes.

### Claim 4 (depends on 3)

The method of claim 3, wherein the loss function includes a softmax function.

### Claim 5 (depends on 1)

The method of claim 1, wherein the variation vector includes either one of a variance vector and a standard deviation vector of the plurality of training feature vectors.

### Claim 6 (depends on 5)

The method of claim 5, wherein
the mean vector is a vector of an elementwise mean value of the plurality of training feature vectors,
the variance vector is a vector of an elementwise variance of the plurality of training feature vectors, and
the standard deviation vector is a vector of an elementwise standard deviation of the plurality of training feature vectors.

### Claim 7 (depends on 1)

The method of claim 1, wherein the determining comprises:
combining the mean vector of the first class, the variation vector of the first class, the mean vector of the second class, and the variation vector of the second class;
outputting a compressed variation vector from a result of the combining; and
determining the additional feature vector based on the compressed variation vector and the mean vector of the second class.

### Claim 8 (depends on 7)

The method of claim 7, wherein the determining of the additional feature vector based on the compressed variation vector and the mean vector of the second class comprises determining the additional feature vector using a Gaussian probability distribution based on the compressed variation vector and the mean vector of the second class.

### Claim 9 (depends on 8)

The method of claim 8, wherein the determining of the additional feature vector based on the compressed variation vector and the mean vector of the second class comprises:
generating a plurality of feature vectors using the Gaussian probability distribution; and
determining the additional feature vector by sampling the generated plurality of feature vectors.

### Claim 10 (depends on 1)

A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 1.

### Claim 11

A processor-implemented neural network method, the method comprising:
extracting, by a feature extractor of a neural network, a plurality of training feature vectors corresponding to a plurality of training class data of each of a plurality of classes including a third class and a fourth class;
determining, by a feature sample generator of the neural network, an additional feature vector of the fourth class based on a mean vector and a variation vector of the plurality of training feature vectors of each of the third class and the fourth class;
determining, by a classifier of the neural network, a class vector of the fourth class based on the additional feature vector and the plurality of training feature vectors of the fourth class; and
training the feature extractor, the feature sample generator, and the classifier to reduce a distance between the class vector of the fourth class and a feature vector of training input data corresponding to the fourth class.

### Claim 12 (depends on 11)

The method of claim 11, wherein the training comprises training the feature extractor, the feature sample generator, and the classifier to reduce the distance between the class vector of the fourth class and the feature vector of the training input data corresponding to the fourth class with respect to a distance between a class vector of each of the plurality of classes including the fourth class and the feature vector of the training input data.

### Claim 13

A neural network apparatus, comprises:
a memory configured to store a neural network including a feature extractor, a feature sample generator, and a classifier; and
one or more processors configured to:
extract, by the feature extractor, a plurality of training feature vectors corresponding to a plurality of training class data of each of a plurality of classes including a first class and a second class,
determine, by the feature sample generator, an additional feature vector of the second class based on a mean vector and a variation vector of the plurality of training feature vectors of each of the first class and the second class,
determine, by the classifier, a class vector of the second class based on the additional feature vector and the plurality of training feature vectors of the second class, and
train the class vector of the second class to reduce a distance between the class vector of the second class and a feature vector of training input data.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the one or more processors are further configured to:
extract, by the feature extractor, a feature vector of received input data; and
recognize, by the classifier, a class of the received input data as the second class, based on the trained class vector of the second class.

### Claim 15

A neural network apparatus, comprising:
a memory configured to store a neural network including a feature extractor, a feature sample generator, and a classifier; and
one or more processors configured to:
extract, by the feature extractor, a plurality of training feature vectors corresponding to a plurality of training class data of each of a plurality of classes including a third class and a fourth class,
determine, by the feature sample generator, an additional feature vector of the fourth class based on a mean vector and a variation vector of the plurality of training feature vectors of each of the third class and the fourth class,
determine, by the classifier, a class vector of the fourth class based on the additional feature vector and the plurality of training feature vectors of the fourth class, and
train the feature extractor, the feature sample generator, and the classifier to reduce a distance between the class vector of the fourth class and a feature vector of training input data corresponding to the fourth class.

### Claim 16

A processor-implemented training method, the method comprising:
extracting, by a feature extractor of a neural network, a plurality of training feature vectors of training class data of a class;
generating, by a feature sample generator of a neural network, a variation vector of the class based on the extracted training feature vectors of training class data of the class;
generating, by the feature sample generator, an additional feature vector of another class based the variation vector of the class; and
training the neural network based on the additional feature vector.

### Claim 17 (depends on 16)

The method of claim 16, wherein the training of the neural network comprises either one or both of:
training the feature extractor, the feature sample generator, and a classifier of the neural network to reduce a distance between a class vector of the other class, determined based on the additional feature vector, and a feature vector of training input data corresponding to the other class; and
using the trained feature extractor, the trained sample generator, and the trained classifier, training the class vector of the other class based on the additional feature vector and a plurality of training feature vectors of training class data of the other class.

### Claim 18 (depends on 16)

The method of claim 16, wherein
the generating of the additional feature vector is further based on a variation vector of the other class,
the variation vector of the class indicates a variation between the training class data of the class in any one of illuminance, posture, facial expression, occlusion, shape change, and noise, and
the variation vector of the other class indicates a variation between the training class data of the other class in any other one of illuminance, posture, facial expression, occlusion, shape change, and noise.
