# US 11,908,170 B2 · Multi-level transferable region-based domain adaptive object detection apparatus and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,170 B2
- Document ID: US11908170B2
- Status: Granted
- Kind: B2
- Application number: 17/540,757
- Filed: 2021-12-02
- Published: 2023-04-27
- Granted: 2024-02-20
- Earliest claimed: 2021-10-22
- Assignees: UIF (University Industry Foundation), Yonsei University
- Applicants: UIF (University Industry Foundation), Yonsei University
- Inventors: Hye Ran Byun; Mi Rae Do; Seog Kyu Jeon; Ki Beom Hong
- Web page: https://patentgrep.com/patent/US11908170B2

## CPC

- G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06F 18/213: Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F 18/253: {of extracted features}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/42: Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A multi-level transferable region-based domain adaptive object detection apparatus includes: an image-level feature part that extracts an image-level feature by providing unlabeled target data to a plurality of convolutional layers; an RPN-level feature part that determines region candidates by providing the image-level feature to a region proposal network (RPN) and performing RPN-level domain adaptation and transferable region filtering; and an instance-level feature part that extracts a region of interest (RoI) pooling feature by conducting dynamic instance sampling on the region candidates.

## Claims

### Claim 1

A multi-level transferable region-based domain adaptive object detection apparatus comprising:
an image-level feature part configured to extract an image-level feature by providing unlabeled target data to a plurality of convolutional layers;
an RPN-level feature part configured to determine region candidates by providing the image-level feature to a region proposal network (RPN) and performing RPN-level domain adaptation and transferable region filtering; and
an instance-level feature part configured to extract a region of interest (RoI) pooling feature by conducting dynamic instance sampling on the region candidates,
wherein the dynamic instance sampling includes dynamically determining a number of the region candidates in a target domain by multiplying a divergence between objectness scores of top N regions of the region candidates from a source domain and objectness scores of top N regions of the region candidates from the target domain by N, and
wherein the image-level feature part, the RPN-level feature part, and the instance-level feature part are each implemented via at least one processor.

### Claim 2 (depends on 1)

The multi-level transferable region-based domain adaptive object detection apparatus of claim 1, wherein the image-level feature part is further configured to create a transferable attention map by collecting multi-scale features of the unlabeled target data from each of the convolutional layers and fusing the multi-scale features.

### Claim 3 (depends on 2)

The multi-level transferable region-based domain adaptive object detection apparatus of claim 2, wherein the image-level feature part is further configured to create the image-level feature by multiplying the transferable attention map by a global feature.

### Claim 4 (depends on 1)

The multi-level transferable region-based domain adaptive object detection apparatus of claim 1, wherein the RPN-level feature part is further configured to generate an RPN-level domain feature by fusing a foreground feature outputted from the RPN and a domain-classified feature that has passed through a gradient reversal layer (GRL)-based domain classifier (Dbox).

### Claim 5 (depends on 4)

The multi-level transferable region-based domain adaptive object detection apparatus of claim 4, wherein the RPN-level feature part is further configured to determine the region candidates by receiving the RPN-level domain feature and filtering out background regions in the transferable region filtering process.

### Claim 6 (depends on 1)

The multi-level transferable region-based domain adaptive object detection apparatus of claim 1, wherein the instance-level feature part is further configured to adjust the number of region candidates based on a discrepancy between foreground probability and an image hardness score.

### Claim 7 (depends on 1)

The multi-level transferable region-based domain adaptive object detection apparatus of claim 1, further comprising an object detection part configured to detect an object from the unlabeled target data by providing the Rol pooling feature to deformable convolutional networks (DCN),
wherein the object detection part is implemented via at least one processor.

### Claim 8

A multi-level transferable region-based domain adaptive object detection method comprising:
extracting an image-level feature by providing unlabeled target data to a plurality of convolutional layers;
determining region candidates by providing the image-level feature to a region proposal network (RPN) and performing RPN-level domain adaptation and transferable region filtering; and
extracting a region of interest (Rol) pooling feature by conducting dynamic instance sampling on the region candidates,
wherein the dynamic instance sampling includes dynamically determining a number of the region candidates in a target domain by multiplying a divergence between objectness scores of top N regions of the region candidates from a source domain and objectness scores of top N regions of the region candidates from the target domain by N.

### Claim 9 (depends on 8)

The multi-level transferable region-based domain adaptive object detection method of claim 8, wherein the extracting of an image-level feature comprises creating a transferable attention map by collecting multi-scale features of the unlabeled target data from each of the convolutional layers and fusing the multi-scale features.

### Claim 10 (depends on 9)

The multi-level transferable region-based domain adaptive object detection method of claim 9, wherein the extracting of an image-level feature comprises creating the image-level feature by multiplying the transferable attention map by a global feature.

### Claim 11 (depends on 8)

The multi-level transferable region-based domain adaptive object detection method of claim 8, wherein the extracting of the Rol pooling feature comprises adjusting the number of region candidates based on a discrepancy between foreground probability and an image hardness score.

### Claim 12 (depends on 8)

The multi-level transferable region-based domain adaptive object detection method of claim 8, further comprising detecting an object from the unlabeled target data by providing the Rol pooling feature to deformable convolutional networks (DCN).

### Claim 13 (depends on 1)

The multi-level transferable region-based domain adaptive object detection apparatus of claim 1,
wherein a first value is obtained by the multiplying of the divergence between the objectness scores of the top N regions of the region candidates from the source domain and the objectness scores of the top N regions of the region candidates from the target domain by N, and
wherein the dynamic instance sampling further includes multiplying a domain probability of a domain classifier used to generate a transferable region using multi-scale features by N to obtain a second value and averaging the first value and the second value to obtain the number of the region candidates in the target domain.