# US 2016/0253597 A1 · Content-aware domain adaptation for cross-domain classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0253597 A1
- Document ID: US20160253597A1
- Status: Published application
- Kind: A1
- Application number: 14/633,550
- Filed: 2015-02-27
- Published: 2016-09-01
- Earliest claimed: 2015-02-27
- Assignees: Xerox Corporation
- Applicants: Xerox Corporation
- Inventors: Himanshu Sharad Bhatt; Deepali Semwal; Shourya Roy
- Web page: https://patentgrep.com/patent/US20160253597A1

## CPC

- G06N 20/20: Ensemble learning
- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An adaptation method includes using a first classifier trained on projected representations of labeled objects from a first domain to predict pseudo-labels for unlabeled objects in a second domain, based on their projected representations. A classifier ensemble is iteratively learned. The ensemble includes a weighted combination of the first classifier and a second classifier. This includes training the second classifier on the original representations of the unlabeled objects for which a confidence for respective pseudo-labels exceeds a threshold. A classifier ensemble is constructed as a weighted combination of the first classifier and the second classifier. Pseudo-labels are predicted for the remaining original representations of the unlabeled objects with the classifier ensemble and weights of the first and second classifiers in the classifier ensemble are adjusted. As the iterations proceed, the unlabeled objects progressively receive pseudo-labels which can be used for retraining the second classifier.

## Claims

### Claim 1

An adaptation method comprising:
providing a first classifier trained on projected representations of objects from a first domain and respective labels, the projected representations having been generated by projecting original representations of the objects in the first domain into a shared feature space with a learned transformation;
providing a pool of original representations of unlabeled objects in a second domain;
projecting the original representations of the unlabeled objects with the learned transformation;
predicting pseudo-labels for ‘the projected representations of the unlabeled objects with the first classifier, each of the predicted pseudo-labels being associated with a confidence;
iteratively learning a classifier ensemble comprising a weighted combination of the first classifier and a second classifier, the learning including:
training the second classifier on the original representations of the unlabeled objects for which the confidence for respective pseudo-labels exceeds a threshold;
constructing a classifier ensemble as a weighted combination of the first classifier and the second classifier;
predicting pseudo-labels for remaining unlabeled objects with the classifier ensemble based on their original representations;
adjusting weights of the first and second classifiers in the classifier ensemble as a function of a learning rate; and
repeating the training, constructing, predicting, and adjusting;
wherein at least one of the predicting of pseudo-labels and iteratively learning the classifier ensemble is performed with a processor.

### Claim 2 (depends on 1)

The method of claim 1, wherein the shared representation is based on co-occurrence statistics.

### Claim 3 (depends on 1)

The method of claim 1, wherein the objects in the first and second domains are text documents and the original representations are based on word frequencies in the text documents.

### Claim 4 (depends on 1)

The method of claim 1, wherein the learned transformation is a matrix.

### Claim 5 (depends on 1)

The method of claim 1, wherein the weights of the first and second classifiers in the classifier ensemble are also adjusted as a function of a measure of similarity between the first and second domains.

### Claim 6 (depends on 5)

The method of claim 5, wherein the measure of similarity is a cosine similarity between feature-based representations of documents in the first and second domains.

### Claim 7 (depends on 1)

The method of claim 1, wherein the predicting pseudo-labels for the original representations of the unlabeled objects with the classifier ensemble comprises weighting a prediction of the first classifier with a first weight and weighting a prediction of the second classifier with a second weight and summing the weighted predictions.

### Claim 8 (depends on 1)

The method of claim 1, wherein the iterative leaning includes, for a first iteration, initializing the weights of the first and second classifiers.

### Claim 9 (depends on 1)

The method of claim 1, wherein the repeating of the training, constructing, predicting, and adjusting is performed until all of the unlabeled objects in the second domain have been assigned a label with at least a threshold confidence or until a predetermined number of iterations has been performed.

### Claim 10 (depends on 1)

The method of claim 1, further comprising outputting the second classifier and the learned weights.

### Claim 11 (depends on 1)

The method of claim 1, further comprising using the learned classifier ensemble to predict a label for a new unlabeled object in the second domain, based on its original representation.

### Claim 12 (depends on 1)

The method of claim 1, wherein in a subsequent iteration, the training of the second classifier is performed with the original representations of the unlabeled objects for which a confidence for the respective pseudo-labels predicted in a prior iteration exceeds a second threshold which is different from the threshold used for pseudo-labels predicted for the projected representations of the unlabeled objects with the first classifier.

### Claim 13 (depends on 1)

The method of claim 1 wherein the labels are opinion-related labels.

### Claim 14 (depends on 1)

The method of claim 1, further comprising learning the transformation with structural correspondence learning based on features extracted from objects in the first and second domains.

### Claim 15 (depends on 1)

A computer program product comprising a non-transitory recording medium storing instructions, which when executed on a computer, causes the computer to perform the method of claim 1.

### Claim 16 (depends on 1)

A system comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory for executing the instructions.

### Claim 17 (depends on 1)

A system for predicting labels for unlabeled objects in the second domain comprising:
memory which stores:
a classifier ensemble learned by the method of claim 1;
a prediction component for predicting the label of an unlabeled objects in the second domain with the learned classifier ensemble; and
a processor which implements the prediction component.

### Claim 18

An adaptation system comprising:
memory which stores:
a learned transformation;
a first classifier that has been trained on projected representations of objects from a first domain and respective labels, the projected representations having been generated by projecting original representations of the objects in the first domain with the learned transformation;
optionally, a representation generator which generates original representations of unlabeled objects in a second domain;
a transformation component which projects the original representations of the unlabeled objects with the learned transformation;
a prediction component which predicts pseudo-labels for unlabeled objects in a second domain with the first classifier based on the projected representations of the unlabeled objects;
an ensemble learning component which iteratively learns a classifier ensemble comprising a weighted combination of the first classifier and a second classifier, the learning including:
training the second classifier on the original representations of the unlabeled objects for which a confidence for the respective pseudo-labels exceeds a threshold confidence;
constructing a classifier ensemble as a weighted combination of the first classifier and the second classifier;
predicting pseudo-labels for remaining unlabeled objects with the classifier ensemble based on their original representations;
adjusting weights of the first and second classifiers in the classifier ensemble as a function of a learning rate; and
repeating the training, constructing, predicting, and adjusting; and
a processor which implements the transformation component, prediction component, and ensemble learning component.

### Claim 19 (depends on 18)

The system of claim 18 further comprising a similarity component which computes a similarity between the first and second domains, the ensemble learning component adjusting the weights of the first and second classifiers in the classifier ensemble as a function of the computed similarity.

### Claim 20

An adaptation method comprising:
learning a transformation based on features extracted from objects in first and second domains;
computing a similarity between the first and second domains;
projecting original representations of labeled objects in the first domain and unlabeled objects in the second domain with the learned projection;
training a first classifier on the projected representations of the objects from the first domain and respective labels;
predicting pseudo-labels for the projected representations of the unlabeled objects with the first classifier;
iteratively learning a classifier ensemble comprising a weighted combination of the first classifier and a second classifier, the learning including:
training the second classifier on the original representations of those of the unlabeled objects and respective pseudo-labels for which a confidence for the respective pseudo-labels exceeds a threshold confidence;
constructing a classifier ensemble as a weighted combination of the first classifier and the second classifier;
predicting pseudo-labels for the original representations of remaining unlabeled objects with the classifier ensemble;
adjusting weights of the first and second classifiers in the classifier ensemble as a function of the computed similarity; and
repeating the training, constructing, predicting, and adjusting,
wherein at least one of the learning of the transformation, computing of the similarity, projecting of the original representations, training of the first classifier, predicting of the pseudo-labels, and iteratively learning the classifier ensemble is performed with a processor.
