# US 12,217,484 B2 · Method of unsupervised domain adaptation in ordinal regression

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,484 B2
- Document ID: US12217484B2
- Status: Granted
- Kind: B2
- Application number: 17/737,114
- Filed: 2022-05-05
- Published: 2023-06-22
- Granted: 2025-02-04
- Earliest claimed: 2021-12-16
- Assignees: Naver Corporation
- Applicants: Naver Corporation
- Inventors: Boris Chidlovskii; Assem Sadek
- Web page: https://patentgrep.com/patent/US12217484B2

## CPC

- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 40/178: {estimating age from face image; using age information for improving recognition}
- G06V 10/7753: {Incorporation of unlabelled data, e.g. multiple instance learning [MIL]}
- G06V 10/7792: {the supervisor being an automated module, e.g. "intelligent oracle"}
- G06V 10/7796: {based on specific statistical tests}
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of jointly training of a transferable feature extractor network, an ordinal regressor network, and an order classifier network in an ordinal regression unsupervised domain adaption network by providing a source of labeled source images and unlabeled target images; outputting image representations from a transferable feature extractor network by performing a minimax optimization procedure on the source of labeled source images and unlabeled target images; training a domain discriminator network, using the image representations from the transferable feature extractor network, to distinguish between source images and target images; training an ordinal regressor network using a full set of source images from the transferable feature extractor network; and training an order classifier network using a full set of source images from said transferable feature extractor network.

## Claims

### Claim 1

An ordinal regression unsupervised domain adaption network for jointly training of a transferable feature extractor network, an ordinal regressor network, and an order classifier network, comprising:
a source of labeled source images and unlabeled target images;
a transferable feature extractor network, operatively connected to said source of labeled source images and unlabeled target images, to output image representations, said image representations being realized by a minimax optimization procedure;
a domain discriminator network operatively connected to said transferable feature extractor network;
an ordinal regressor network operatively connected to said transferable feature extractor network; and
an order classifier network operatively connected to said transferable feature extractor network and said domain discriminator network;
said domain discriminator network being trained, using said image representations from said transferable feature extractor network, to distinguish between source images and target images;
said ordinal regressor network being trained, using a full set of source images from said transferable feature extractor network;
said order classifier network being trained, using a pair of source images from said transferable feature extractor network.

### Claim 2 (depends on 1)

The ordinal regression unsupervised domain adaption network as claimed in claim 1, wherein said transferable feature extractor network is trained by maximizing a loss of said domain discriminator.

### Claim 3 (depends on 1)

The ordinal regression unsupervised domain adaption network as claimed in claim 1, wherein a total loss for training the ordinal regression universal domain adaptation network is given as:
ℒ
⁡
(
F
,
G
r
,
G
o
,
G
d
)
=
ℒ
or
(
F
,
G
r
)
+
γ
1
⁢
ℒ
ord
(
F
,
G
o
)
+
γ
2
⁢
ℒ
dom
(
F
,
G
d
)
,
γ1, γ2 are hyper-parameters controlling an importance of order and domain discrimination adversarial losses;
wherein
F
⋆
,
G
r
*
,
G
o
*
=
arg
min
F
,
G
r
,
G
o
max
G
d
ℒ
⁡
(
F
,
G
r
,
G
o
,
G
d
)
is solved by alternating between optimizing F, Gr, Go, and Gd until the total loss converges.

### Claim 4 (depends on 3)

The ordinal regression unsupervised domain adaption network as claimed in claim 3, wherein the loss for said ordinal regressor network is defined on labeled source images,
ℒ
or
(
F
,
G
o
)
=
𝔼
x
i
∈
D
s
⁢
L
coral
(
G
r
(
F
⁡
(
x
i
)
)
,
y
i
)
,
where Lcoral is Coral loss.

### Claim 5 (depends on 3)

The ordinal regression unsupervised domain adaption network as claimed in claim 3, wherein the loss for said order classifier network is defined on pairs of source images and order relationships thereof,
ℒ
ord
(
F
,
G
o
)
=
𝔼
x
i
,
x
j
~
D
s
⁢
L
ord
(
G
o
,
F
′
,
x
i
≺
x
j
)
,
where Lord is the cross entropy loss, and F′ compares the two image feature vectors.

### Claim 6 (depends on 3)

The ordinal regression unsupervised domain adaption network as claimed in claim 3, wherein the loss for said domain discriminator network is
ℒ
d
=
𝔼
x
s
∈
D
s
⁢
w
⁡
(
x
s
)
⁢
log
⁢
G
d
(
F
⁡
(
x
s
)
)
+
𝔼
x
t
∈
D
t
⁢
w
⁡
(
x
t
)
⁢
log
(
1
-
G
d
(
F
⁡
(
x
t
)
)
,
where weights w(xt) and w(xs) are obtained by target image weighting and source class weighting, w(xt)=p(xt∈Y), w(xs)=p(xs∈Y)=p(ys∈Y), ys=y(xs).

### Claim 7

A method of jointly training of a transferable feature extractor network, an ordinal regressor network, and an order classifier network in an ordinal regression unsupervised domain adaption network, comprising:
(a) providing a source of labeled source images and unlabeled target images;
(b) outputting image representations from a transferable feature extractor network by performing a minimax optimization procedure on the source of labeled source images and unlabeled target images;
(c) training a domain discriminator network, using the image representations from the transferable feature extractor network, to distinguish between source images and target images;
(d) training an ordinal regressor network using a full set of source images from the transferable feature extractor network; and
(e) training an order classifier network using a full set of source images from said transferable feature extractor network.

### Claim 8 (depends on 7)

The method as claimed in claim 7, further comprising:
(f) training the transferable feature extractor network by maximizing a loss of the domain discriminator.

### Claim 9 (depends on 7)

The method as claimed in claim 7, wherein a total loss for training the ordinal regression universal domain adaptation network is given as:
ℒ
⁡
(
F
,
G
r
,
G
o
,
G
d
)
=
ℒ
or
(
F
,
G
r
)
+
γ
1
⁢
ℒ
ord
(
F
,
G
o
)
+
γ
2
⁢
ℒ
dom
(
F
,
G
d
)
,
γ1, γ2 are hyper-parameters controlling an importance of order and domain discrimination adversarial losses;
wherein
F
⋆
,
G
r
*
,
G
o
*
=
arg
min
F
,
G
r
,
G
o
max
G
d
ℒ
⁡
(
F
,
G
r
,
G
o
,
G
d
)
is solved by alternating between optimizing F, Gr, Go, and Gd until the total loss converges.

### Claim 10 (depends on 9)

The method as claimed in claim 9, wherein the loss for the ordinal regressor network is defined on labeled source images,
ℒ
or
(
F
,
G
o
)
=
𝔼
x
i
∈
D
s
⁢
L
coral
(
G
r
(
F
⁡
(
x
i
)
)
,
y
i
)
,
where Lcoral is Coral loss.

### Claim 11 (depends on 9)

The method as claimed in claim 9, wherein the loss for the order classifier network is defined on pairs of source images and order relationships thereof,
ℒ
ord
(
F
,
G
o
)
=
𝔼
x
i
,
x
j
~
D
s
⁢
L
ord
(
G
o
,
F
′
,
x
i
≺
x
j
)
,
where Lord is the cross entropy loss, and F compares the two image feature vectors.

### Claim 12 (depends on 9)

The ordinal regression unsupervised domain adaption network as claimed in claim 9, wherein the loss for the domain discriminator network is
ℒ
d
=
𝔼
x
s
∈
D
s
⁢
w
⁡
(
x
s
)
⁢
log
⁢
G
d
(
F
⁡
(
x
s
)
)
+
𝔼
x
t
∈
D
t
⁢
w
⁡
(
x
t
)
⁢
log
(
1
-
G
d
(
F
⁡
(
x
t
)
)
,
where weights w(xt) and w(xs) are obtained by target image weighting and source class weighting, w(xt)=p(xt∈Y), w(xs)=p(xs∈Y)=p(ys∈Y), ys=y(xs).