# US 2020/0394559 A1 · Collecting observations for machine learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394559 A1
- Document ID: US20200394559A1
- Status: Published application
- Kind: A1
- Application number: 16/507,025
- Filed: 2019-07-09
- Published: 2020-12-17
- Earliest claimed: 2019-06-13
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Cheng ZHANG; Wenbo GONG; Richard Eric TURNER; Sebastian TSCHIATSCHEK; José Miguel HERNÁNDEZ LOBATO
- Web page: https://patentgrep.com/patent/US20200394559A1

## CPC

- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 20/00: Machine learning
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/0499: Feedforward networks
- G06N 3/09: Supervised learning
- G06N 3/091: Active learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method of training a model comprising a generative network mapping a latent vector to a feature vector, wherein weights in the generative network are modelled as probabilistic distributions. The method comprises: a) obtaining one or more observed data points, each comprising an incomplete observation of the features in the feature vector; b) training the model based on the observed data points to learn values of the weights of the generative network which map the latent vector to the feature vector; c) from amongst a plurality of potential next features to observe, searching for a target feature of the feature vector which maximizes a measure of expected reduction in uncertainty in a distribution of said weights of the generative network given the observed data points so far; and d) outputting a request to collect a target data point comprising at least the target feature.

## Claims

### Claim 1

A computer-implemented method of training a model comprising one or more neural networks including at least a generative network, the generative network having a latent vector as an input vector and a feature vector as an output vector, each element of the feature vector representing a different one of a set of features, wherein weights applied by at least some nodes in the generative network are each modelled as a probabilistic distribution; the method comprising:
a) obtaining one or more observed data points, each comprising a respective subset of feature values, wherein within each subset, each feature value is a value of a corresponding one of a subset of the features in the feature vector;
b) training the model based on the observed data points to learn values of the weights of the generative network which map the latent vector to the feature vector;
c) from amongst a plurality of potential next features to observe, searching for a target feature of the feature vector which maximizes a measure of expected reduction in uncertainty in a distribution of said weights of the generative network given the observed data points so far; and
d) outputting a request to collect a target data point comprising at least the target feature.

### Claim 2 (depends on 1)

The method of claim 1, wherein the request comprises a message requesting a human user or group of human users to collect the target data point.

### Claim 3 (depends on 1)

The method of claim 1, wherein the request comprises a signal to an automated process requesting the automated process to collect the target data point.

### Claim 4 (depends on 1)

The method of claim 1, wherein at least some connections between nodes in the generative network are each modelled as a probabilistic distribution.

### Claim 5 (depends on 1)

The method of claim 1, wherein the neural networks of the model further include an inference network having the feature vector as an input vector and the latent vector as an output vector, the inference network and generative network thus forming an encoder and decoder respectively of a variational auto encoder; and the training further comprises learning weights of the inference network which map the feature vector to the latent vector.

### Claim 6 (depends on 5)

The method of claim 5, wherein the weights applied by at least some nodes in the inference network are each modelled as a probabilistic distribution.

### Claim 7 (depends on 5)

The method of claim 5, wherein at least some connections between nodes in the inference network are each modelled as a probabilistic distribution.

### Claim 8 (depends on 1)

The method of claim 1, comprising: e) receiving the target data point in response to said signal, and further training of the model based on the received target data point.

### Claim 9 (depends on 8)

The method of claim 8, comprising repeating a)-e) over multiple iterations, each iteration including the received target data point from the previous iteration amongst the observed data points.

### Claim 10 (depends on 9)

The method of claim 9, further comprising using the model to impute one or more of the features values of a further data point after one or more of the iterations of said further training.

### Claim 11 (depends on 1)

The method of claim 1, wherein the measure of uncertainty comprises a measure of:
a difference between i) an entropy of said distribution given the observed data points and ii) an expectation of the entropy given the input observed data points and the potential feature.

### Claim 12 (depends on 11)

The method of claim 11, wherein the measure of said difference comprises a measure of RI where:
RI(xid,XO)=H[p(θ|XO)]−p(xid|xO)[H[p(θ|XO,xid)]]
where H is the entropy, Ep is the expectation, p is said distribution, XO is a vector of the observed data points, θ is a vector of the weights of the generative network, and xid is the feature value of feature d of the feature vector in data point i.

### Claim 13 (depends on 1)

The method of claim 1, wherein each of the observed data points is labelled with a classification.

### Claim 14 (depends on 13)

The method of claim 13, comprising:
e) receiving the target data point in response to said signal, wherein the received target data point is labelled with a classification, and further training of the model based on the received target data point; and
repeating a)-e) over multiple iterations, each iteration including the received target data point from the previous iteration amongst the observed data points.

### Claim 15 (depends on 14)

The method of claim 14, further comprising using the model to predict a classification of a further data point after one or more of the iterations of said further training.

### Claim 16 (depends on 13)

The method of claim 13, wherein said measure of uncertainty comprises a measure of a combination of:
a difference between i) an entropy of said distribution given the observed data points and ii) an expectation of the entropy given the input observed data points and the potential feature; and
a measure of conditional mutual information in the weights of the generative network and a predicted classification given the observed data points and the potential feature.

### Claim 17 (depends on 16)

The method of claim 16, wherein said measure of mutual information comprises a measure of RP where:
RP(xid,XO)=p(xid|xO)[H[p(yi|xid,XO)]]−p(θ,xid|xO)[H[p(yi|θ,xid,XO)]]
where H is the entropy, Ep is the expectation, p is said distribution, XO is a vector of the observed data points, θ is a vector of the weights of the generative network, and xid is the feature value of feature d of the feature vector in data point i, and yi is a predicted classification.

### Claim 18 (depends on 17)

The method of claim 17, wherein said measure comprises a measure of RC where:
RC(xid,XO)=(1−α)RI(xid,XO)+αRP(xid,XO)
where α is a predetermined coefficient, and
RI(xid,XO)=H[p(θ|XO)]−p(xid|xO)[H[p(θ|XO,xid)]]

### Claim 19 (depends on 1)

A computer program embodied on computer-readable storage, the program comprising code configured so as when run on one or more processors to perform the operations of claim 1.

### Claim 20 (depends on 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)

A computer system comprising:
storage comprising one or more memory units, and
processing apparatus comprising one or more processing units;
wherein the storage stores code arranged to run on the processing apparatus, the code being configured so as when thus run to perform the operations of any of claims 1 to 18.
