# US 2018/0024968 A1 · System and method for domain adaptation using marginalized stacked denoising autoencoders with domain prediction regularization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0024968 A1
- Document ID: US20180024968A1
- Status: Published application
- Kind: A1
- Application number: 15/216,805
- Filed: 2016-07-22
- Published: 2018-01-25
- Earliest claimed: 2016-07-22
- Assignees: Xerox Corporation
- Applicants: Xerox Corporation
- Inventors: Stéphane Clinchant; Gabriela Csurka; Boris Chidlovskii
- Web page: https://patentgrep.com/patent/US20180024968A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0499: Feedforward networks
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/096: Transfer learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for domain adaptation of samples includes receiving training samples from a plurality of domains, the plurality of domains including at least one source domain and a target domain, each training sample including values for a set of features. A domain predictor is learned on at least some of the training samples from the plurality of domains and respective domain labels. Domain adaptation is performed on the training samples using marginalized denoising autoencoding. This generates a domain adaptation transform layer (or layers) that transforms the training samples to a common adapted feature space. The domain adaptation employs the domain predictor to bias the domain adaptation towards one of the plurality of domains. Domain adapted training samples and their class labels can be used to train a classifier for prediction of class labels for unlabeled target samples that have been domain adapted with the domain adaptation transform layer(s).

## Claims

### Claim 1

A method for domain adaptation of samples comprising:
receiving training samples from a plurality of domains, the plurality of domains including at least one source domain and a target domain, each training sample including values for a set of features;
learning a domain predictor on at least some of the training samples from the plurality of domains and respective domain labels;
performing domain adaptation using marginalized denoising autoencoding operating on the training samples to generate at least one domain adaptation transform layer operative to transform the source and target training samples to a common adapted feature space, the domain adaptation employing the domain predictor to bias the domain adaptation of the training samples towards one of the plurality of domains; and
outputting at least one of:
the at least one domain adaptation transform layer, and
information generated therefrom.

### Claim 2 (depends on 1)

The method of claim 1, wherein at least one of the learning a domain predictor and the performing domain adaptation is performed with a processor.

### Claim 3 (depends on 1)

The method of claim 1, wherein the domain adaptation computes a mapping for mapping samples to the common adapted feature space as a function of a concatenation of the training samples, a selected dropout probability, and the domain predictor.

### Claim 4 (depends on 1)

The method of claim 1, wherein the at least one domain adaptation transform layer computes a mapping W in closed form as a function of:
(P+λ(1−p)XTRtcT)(I+λccT)−1Q−1,
where
Q
i
,
j
=
[
S
ij

q
i

q
j
,
if


i
≠
j
,
S
ij

q
i
,
if


i
=
j
,
]
,
Pi,j=Sijqj,
q=[1−p1, . . . , 1−pd]εRd, where d is the number of dimensions i,
S=XXT is the covariance matrix of the concatenation of the input training samples X,
pi is a feature i corruption probability,
λ is a fixed weight,
X is a replicated set of the input training samples;
Rt is a vector, indicating, for each training sample, a regularization objective; and
c is the domain predictor.

### Claim 5 (depends on 4)

The method of claim 4, where Rt includes a value of 1 for training samples from the source domain.

### Claim 6 (depends on 4)

The method of claim 4, wherein λ has a value of from 0.001 to 100.

### Claim 7 (depends on 1)

The method of claim 1 wherein the domain predictor includes at least one vector of values, each vector including a value for each of the features in the set of features.

### Claim 8 (depends on 1)

The method of claim 1, wherein the domain predictor is learned by minimizing a loss function, over the set X of source and target training samples, of the form:
(c,α)=∥D−Xc∥2+α∥c∥2
where α is a regularization term weight;
D is a binary vector indicating the domain for each sample in X.

### Claim 9 (depends on 8)

The method of claim 8, wherein α has a value of at least 0.01.

### Claim 10 (depends on 1)

The method of claim 1, wherein the domain adaptation operates on the training samples with different feature corruption probabilities for at least one of:
different features of the set of features, and
different domains of the plurality of domains.

### Claim 11 (depends on 1)

The method of claim 1, wherein the at least one domain adaptation transform layer comprises a plurality of domain adaptation transform layers stacked one over another.

### Claim 12 (depends on 11)

The method of claim 11, wherein in at least one of the stacked domain adaptation transform layers, a non-linear function is applied to the transformed training samples.

### Claim 13 (depends on 1)

The method of claim 1, further comprising performing supervised or semi-supervised learning on at least some of the training samples that have been transformed to the common adapted feature space, the training samples being labeled with labels from a set of class labels, to generate a classifier that outputs label predictions from the set of labels for the training samples and wherein the information comprises at least one of the trained classifier and a label prediction for a target sample generated with the trained classifier.

### Claim 14 (depends on 13)

The method of claim 13, further comprising generating a label prediction for an input sample in the target domain represented by values for the set of features by applying the classifier to the input sample transformed to the common adapted feature space using the at least one domain adaptation transform layer.

### Claim 15 (depends on 1)

The method of claim 1, wherein the training samples are representations of objects selected from text documents and images.

### Claim 16 (depends on 1)

A non-transitory storage medium storing instructions which when executed by a computer, perform the method of claim 1.

### Claim 17 (depends on 1)

A system comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory which execute the instructions.

### Claim 18

A system comprising:
a domain predictor for predicting a domain of a plurality of domains for a sample, the sample including values for a set of features, the plurality of domains including at least one source domain and a target domain;
a mapping component which receives a set of training samples from the plurality of domains, each training sample including values for the set of features, the mapping component performing domain adaptation using marginalized denoising autoencoding on the set of training samples to generate at least one domain adaptation transform layer operative to transform the training samples to a common adapted feature space, the domain adaptation incorporating the domain predictor to bias the domain adaptation of the source domain training samples towards one of the plurality of domains; and
a processor which implements the mapping component.

### Claim 19 (depends on 18)

The system of claim 18, further comprising:
a classifier learning component for learning a classifier with training samples transformed to the common adapted feature space, the training samples from each of the at least one source domain being labeled with class labels of a set of class labels.

### Claim 20

A classification method comprising:
receiving an input sample from a source domain, the input sample including values for a set of features;
classifying the input sample with a classifier trained on a set of domain adapted training samples, the domain adapted training samples each including values for the set of features; the domain adapted training samples having been generated from a set of training samples, from a plurality of domains including a source domain and a target domain, that have been transformed to an adapted feature space using a marginalized denoising autoencoding layer incorporating a domain predictor, whereby the transformation of source domain target samples is biased towards the target domain such that the domain adapted source domain target samples are more likely to be predicted to be target domain samples by the domain predictor; and
outputting a label prediction for the input sample based on the classification.