# US 2020/0394512 A1 · Robustness against manipulations in machine learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394512 A1
- Document ID: US20200394512A1
- Status: Published application
- Kind: A1
- Application number: 16/508,281
- Filed: 2019-07-10
- Published: 2020-12-17
- Earliest claimed: 2019-06-13
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Cheng ZHANG; Yingzhen LI
- Web page: https://patentgrep.com/patent/US20200394512A1

## CPC

- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/08: Learning methods
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 7/023: {Learning or tuning the parameters of a fuzzy system}
- G06T 2207/20081: Training; Learning
- G06V 10/811: {the classifiers operating on different input data, e.g. multi-modal recognition}
- G06V 10/82: using neural networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method comprising: receiving observed data points each comprising a vector of feature values, wherein for each data point, the respective feature values are values of different features of a feature vector. Each observed data point represents a respective observation of a ground truth as observed in the form of the respective values of the feature vector. The method further comprises learning parameters of a machine-learning model based on the observed data points. The machine-learning model comprises one or more statistical models arranged to model a causal relationship between the feature vector and a latent vector, a classification, and a manipulation vector. The manipulation vector represents an effect of potential manipulations occurring between the ground truth and the observation thereof as observed via the feature vector. The learning comprises learning parameters of the one or more statistical models to map between the feature vector, latent vector, classification and manipulation vector.

## Claims

### Claim 1

A computer-implemented method of machine learning, the method comprising:
receiving a plurality of observed data points each comprising a respective vector of feature values, wherein for each observed data point, the respective feature values are values of a plurality of different features of a feature vector, and each observed data point represents a respective observation of a ground truth as observed in the form of the respective values of the feature vector; and
learning parameters of a machine-learning model based on the observed data points, wherein the machine-learning model comprises one or more statistical models arranged to model a causal relationship between the feature vector and a latent vector, a classification, and a manipulation vector, the manipulation vector representing an effect of potential manipulations occurring between the ground truth and the observation of the group truth as observed via said feature vector, wherein the learning comprises learning parameters of the one or more statistical models to map between the feature vector, latent vector, classification and manipulation vector.

### Claim 2 (depends on 1)

The method of claim 1, wherein the learning comprises at least a training phase wherein each of the data points used in the training phase further comprises a respective value of the classification.

### Claim 3 (depends on 1)

The method of claim 1, wherein the observed data points comprise a first group of the data points which do not include the effect of at least one manipulation, and a second group of said data points which do include the effect of the at least one manipulation.

### Claim 4 (depends on 3)

The method of claim 3, wherein:
the learning comprises at least a training phase wherein each of the data points used in the training phase further comprises a respective value of the classification;
the learning further comprises a fine-tuning phase following the training phase, wherein each of the data points used in the training phase is not labelled with a value of the classification; and
the data points used in the training phase comprise the first group, and the data points used in the fine-tuning phase comprise the second group.

### Claim 5 (depends on 4)

The method of claim 4, wherein the statistical models comprises one or more first statistical models and one or more second statistical models, wherein the one or more second statistical models are arranged to model the causal relationship between the manipulation vector and the feature vector; and:
a) when learning based on the first group of data points, the manipulation vector is set to a null value, and the parameters of the one or more first statistical models are learned whilst the parameters of the one or more second statistical models are fixed, whereas
b) when learning based on the second group of data points, the manipulation vector is either set to a known value representing the at least one manipulation if known or the manipulation vector is inferred if the at least one manipulation is not known, and the parameters of the at least one or more second statistical models are learned.

### Claim 6 (depends on 1)

The method of claim 1, wherein the statistical models comprises one or more first statistical models and one or more second statistical models, wherein the one or more second statistical models are arranged to model the causal relationship between the manipulation vector and the feature vector.

### Claim 7 (depends on 1)

The method of claim 1, wherein the one or more statistical models comprise one or more generative models mapping from the classification, latent vector and manipulation vector as inputs to the feature vector as an output; the learning comprising learning parameters of the one or more generative models which map the classification, latent vector and manipulation vector to the feature vector.

### Claim 8 (depends on 7)

The method of claim 7, wherein:
the statistical models comprises one or more first statistical models and one or more second statistical models, wherein the one or more second statistical models are arranged to model the causal relationship between the manipulation vector and the feature vector;
the one or more first statistical models comprise a first one or more of said generative models which take the latent vector and classification but not the manipulation vector as respective inputs; and
the one or more second statistical models comprise a second, separate one of said generative models which takes the manipulation vector as a respective input but not the latent vector nor the classification;
wherein each of the first and second generative models is configured to map its respective input to a respective output, the outputs of the first and second generative models being mapped to the feature vector.

### Claim 9 (depends on 8)

The method of claim 8, wherein:
the first generative models comprise a generative model taking the classification as a respective input but not the latent vector nor the manipulation vector, and a separate generative model which takes the latent vector as an input but not the classification nor the manipulation vector.

### Claim 10 (depends on 8)

The method of claim 8, wherein the first statistical models further comprise another of said generative models arranged as a merging generative model, mapping the outputs of the first and second generative networks to the feature vector via the merging generative model.

### Claim 11 (depends on 8)

The method of claims 8, wherein:
the observed data points comprise a first group of the data points which do not include the effect of at least one manipulation, and a second group of said data points which do include the effect of the at least one manipulation;
the learning comprises at least a training phase wherein each of the data points used in the training phase further comprises a respective value of the classification;
the learning further comprises a fine-tuning phase following the training phase, wherein each of the data points used in the training phase is not labelled with a value of the classification;
the data points used in the training phase comprise the first group, and the data points used in the fine-tuning phase comprise the second group;
the statistical models comprises one or more first statistical models and one or more second statistical models, wherein the one or more second statistical models are arranged to model the causal relationship between the manipulation vector and the feature vector; and:
a) when learning based on the first group of data points, the manipulation vector is set to a null value, and the parameters of the one or more first generative models and the merging generative model are learned, whilst the parameters of the second generative model are fixed, whereas
b) when learning based on the second group of data points, the manipulation vector is either set to a known value representing the at least one manipulation if known or the manipulation vector is inferred if the at least one manipulation is not known, and the parameters of at least the second generative model are learned.

### Claim 12 (depends on 1)

The method of claim 1, wherein the one or more statistical models comprise one or more inference models mapping from the classification, feature vector and manipulation vector as inputs to the latent vector as an output; the learning comprising learning parameters of the one or more inference models which map the classification, feature vector and manipulation vector to the latent vector.

### Claim 13 (depends on 12)

The method of claim 12, wherein:
the statistical models comprises one or more first statistical models and one or more second statistical models, wherein the one or more second statistical models are arranged to model the causal relationship between the manipulation vector and the feature vector; and
the one or more first statistical models comprises at least a first of said inference models mapping from the classification, feature vector and manipulation vector to the latent vector.

### Claim 14 (depends on 13)

The method of claim 13, wherein the one or more second statistical models comprise at least a second, separate one of said inference models mapping from the feature vector to the manipulation vector.

### Claim 15 (depends on 14)

The method of claim 14, wherein:
the observed data points comprise a first group of the data points which do not include the effect of at least one manipulation, and a second group of said data points which do include the effect of the at least one manipulation;
the learning comprises at least a training phase wherein each of the data points used in the training phase further comprises a respective value of the classification;
the learning further comprises a fine-tuning phase following the training phase, wherein each of the data points used in the training phase is not labelled with a value of the classification;
the data points used in the training phase comprise the first group, and the data points used in the fine-tuning phase comprise the second group;
the statistical models comprises one or more first statistical models and one or more second statistical models, wherein the one or more second statistical models are arranged to model the causal relationship between the manipulation vector and the feature vector; and:
a) when learning based on the first group of data points, the manipulation vector is set to a null value, and the parameters of the at least one first inference model are learned whilst the parameters of the second inference model are fixed, whereas
b) when learning based on the second group of data points, the manipulation vector is either set to a known value representing the at least one manipulation if known or the manipulation vector is inferred if the at least one manipulation is not known, and the parameters of the both the first and second inference models are learned.

### Claim 16 (depends on 1)

The method of claim 1, wherein the one or more statistical models further include:
a co-parent vector modelling a circumstance occurring within an environment of the ground truth having a similar effect to the ground truth, and/or
a parent vector modelling a parent cause of the classification.

### Claim 17 (depends on 1)

The method of claim 1, wherein each of any one, some or all of said statistical models is a neural network, in which said parameters are weights.

### Claim 18

(canceled)

### Claim 19

A computer program embodied on computer-readable storage, the program comprising code configured so as when run on one or more processors to perform operations of:
receiving a plurality of observed data points each comprising a respective vector of feature values, wherein for each observed data point, the respective feature values are values of a plurality of different features of a feature vector, and each observed data point represents a respective observation of a ground truth as observed in the form of the respective values of the feature vector; and
learning parameters of a machine-learning model based on the observed data points, wherein the machine-learning model comprises one or more statistical models arranged to model a causal relationship between the feature vector and a latent vector, a classification, and a manipulation vector, the manipulation vector representing an effect of potential manipulations occurring between the ground truth and the observation of the group truth as observed via said feature vector, wherein the learning comprises learning parameters of the one or more statistical models to map between the feature vector, latent vector, classification and manipulation vector.

### Claim 20

A computer system comprising:
storage comprising one or more memory units, and
processing apparatus comprising one or more processing units;
wherein the storage stores code arranged to run on the processing apparatus, the code being configured so as when thus run to perform operations of:
receiving a plurality of observed data points each comprising a respective vector of feature values, wherein for each observed data point, the respective feature values are values of a plurality of different features of a feature vector, and each observed data point represents a respective observation of a ground truth as observed in the form of the respective values of the feature vector; and
learning parameters of a machine-learning model based on the observed data points, wherein the machine-learning model comprises one or more statistical models arranged to model a causal relationship between the feature vector and a latent vector, a classification, and a manipulation vector, the manipulation vector representing an effect of potential manipulations occurring between the ground truth and the observation of the group truth as observed via said feature vector, wherein the learning comprises learning parameters of the one or more statistical models to map between the feature vector, latent vector, classification and manipulation vector.
