# US 2024/0177456 A1 · Object detection method for detecting one or more objects using a plurality of deep convolution neural network layers and object detection apparatus using the same method and non-transitory storage medium thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0177456 A1
- Document ID: US20240177456A1
- Status: Published application
- Kind: A1
- Application number: 17/993,881
- Filed: 2022-11-24
- Published: 2024-05-30
- Earliest claimed: 2022-11-24
- Assignees: Industrial Technology Research Institute
- Applicants: Industrial Technology Research Institute
- Inventors: Peter Chondro
- Web page: https://patentgrep.com/patent/US20240177456A1

## CPC

- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/82: using neural networks
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

According to an exemplary embodiment, the disclosure provides an object detection method includes not limited to obtaining a set of a plurality of object annotated images in a source domain and have a first image style; obtaining a minority set of a plurality of object annotated images in a target domain and having a second image style; obtaining a majority set of a plurality of unannotated images which are in the target domain and having the second image style; performing an image style transfer to generate a converted set of object annotated images having the second image style; generating object annotation for the majority set of the plurality of unannotated images in the second image style to change from the majority set of a plurality of unannotated images into a majority set of a plurality of annotated images; and performing an active domain adaptation to generate an object detection model.

## Claims

### Claim 1

An object detection method for detecting one or more objects using a plurality of deep convolution neural layers, the method comprising:
obtaining a set of a plurality of object annotated images which are in a source domain and have a first image style;
obtaining a minority set of a plurality of object annotated images which are in a target domain and having a second image style;
obtaining a majority set of a plurality of unannotated images which are in the target domain and having the second image style;
performing an image style transfer from the source domain to the target domain by converting the plurality of object annotated images in the source domain from having the first image style into having the second image style to generate a converted set of object annotated images having the second image style;
generating object annotations for the majority set of the plurality of unannotated images in the second image style based on the converted set of object annotated images in the second image style and the minority set of the plurality of object annotated images in the second image style to change from the majority set of a plurality of unannotated images into a majority set of a plurality of annotated images; and
performing an active domain adaptation by adapting the minority set of a plurality of object annotated images, the converted set of object annotated images, and the majority set of the plurality of annotated images so as to generate an object detection model.

### Claim 2 (depends on 1)

The object detection method of claim 1 further comprising:
recognizing a classification of a detected object of a captured image by using the object detection model to indicate the detected object with an object annotation which indicates that the detected object has been classified.

### Claim 3 (depends on 2)

The object detection method of claim 2, wherein the object annotation is a two-dimensional (2D) label which is a 2D bounding box or a perspective-induced 2D bounding box or a 2D point to indicate that the object of the target image has been classified.

### Claim 4 (depends on 1)

The object detection method of claim 1, wherein performing the image style transfer comprising:
constructing a plurality of deep convolution neural layers which converting from the first image style of a 2D data array of the source domain into the second image style of another 2D data array of the target domain to generate the converted set of object annotated images having the second image style.

### Claim 5 (depends on 2)

The object detection method of claim 2, wherein performing the classification of the detected object of the captured image based on the object detection model comprising:
performing the classification of the detected object of the captured image with the least annotation density based on density estimators which are prioritized for gradient descent according to a scheduling policy.

### Claim 6 (depends on 4)

The object detection method of claim 4, wherein constructing the plurality of deep convolution neural layers further comprising:
encoding a 2D data array of a raw input space into a plurality of data arrays of a feature space and a first plurality of data arrays of a style space.

### Claim 7 (depends on 6)

The object detection method of claim 6, wherein constructing the plurality of deep convolution neural layers further comprising:
replacing the first plurality of data array of the style space with a second plurality of data arrays of the style space from the target domain and combining the second plurality of data arrays of the style space with the plurality of data arrays of the feature space.

### Claim 8 (depends on 6)

The object detection method of claim 6, wherein the set of the plurality of object annotated images which are in the source domain has a different quantity from the minority set of the plurality of object annotated images which are in the target domain plus the majority set of the plurality of unannotated images which are in the target domain.

### Claim 9 (depends on 6)

The object detection method of claim 6, wherein in response to the set of the plurality of object annotated images which are in the source domain has a different quantity from the minority set of the plurality of object annotated images which are in the target domain plus the majority set of the plurality of unannotated images which are in the target domain, the processor is further configured to duplicate either an image in the source domain or an image in the target domain so that the quality is no longer different.

### Claim 10 (depends on 4)

The object detection method of claim 4, wherein the performing the image style transfer from the source domain to the target domain comprising:
performing unsupervised learning by pairing a first image from the source domain without any annotation with a second image from the target domain without any annotation.

### Claim 11 (depends on 10)

The object detection method of claim 10 further comprising:
performing a source domain encoding the first image by extracting a first style feature and a first content feature from the first image; and
performing a target domain encoding the second image by extracting a second style feature and a second content feature from the second image.

### Claim 12 (depends on 11)

The object detection method of claim 11 further comprising:
performing a source domain decoding by receiving the first style feature, the first content feature, and the second style feature to generate a source target output image and a source to source output image; and
performing a target domain decoding by receiving the first content feature, the second content feature, and the second style feature to generate a target to target output image and a source to target output image.

### Claim 13 (depends on 11)

The object detection method of claim 11 further comprising:
calculating loss functions for the first style feature and the second style feature; and
performing a benchmark based on the calculated loss functions by iteratively adjusting weights of the plurality of deep convolution neural layers.

### Claim 14 (depends on 1)

The object detection method of claim 1 further comprising:
utilizing a pseudo-labeler which generates the object annotations by performing supervised trainings.

### Claim 15 (depends on 2)

The object detection method of claim 2 further comprising:
generating a model for a main detector which comprises a backbone architecture and an end layer and is generated in response to performing supervised training based on the active domain adaptation.

### Claim 16 (depends on 14)

The object detection method of claim 14, performing a data augmentation on the minority set of the plurality of object annotated images and the converted set of object annotated images so as to introduce a degree of generalization to pseudo-labeler during the supervised training.

### Claim 17 (depends on 15)

The object detection method of claim 15, further comprising:
performing a data augmentation on the minority set of the plurality of object annotated images, the converted set of object annotated images, and the majority set of the plurality of annotated images to increase data variations so as to introduce a degree of generalization to the main detector during the supervised training.

### Claim 18 (depends on 15)

The object detection method of claim 15, wherein the backbone architecture is configured to extract features at multi-scale resolutions, and end layer branch is configured to concatenate the multi-scale features to generate prediction regression for the object annotation.

### Claim 19

An object detection apparatus comprising:
a storage device; and
a processor coupled to the storage device and configured to:
obtain a set of a plurality of object annotated images which are in a source domain and have a first image style;
obtain a minority set of a plurality of object annotated images which are in a target domain and having a second image style;
obtain a majority set of a plurality of unannotated images which are in the target domain and having the second image style;
perform an image style transfer from the source domain to the target domain by converting the plurality of object annotated images in the source domain from having the first image style into having the second image style to generate a converted set of object annotated images having the second image style;
generate object annotation for the majority set of the plurality of unannotated images in the second image style based on the converted set of object annotated images in the second image style and the minority set of the plurality of object annotated images in the second image style to change from the majority set of a plurality of unannotated images into a majority set of a plurality of annotated images; and
perform an active domain adaptation by adapting the minority set of a plurality of object annotated images, the converted set of object annotated images, and the majority set of the plurality of annotated images so as to generate an object detection model.

### Claim 20

A non-transitory storage medium which stores computer readable program loaded into a processor of a computer to control the computer to perform functions comprising:
obtaining a set of a plurality of object annotated images which are in a source domain and have a first image style;
obtaining a minority set of a plurality of object annotated images which are in a target domain and having a second image style;
obtaining a majority set of a plurality of unannotated images which are in the target domain and having the second image style;
performing an image style transfer from the source domain to the target domain by converting the plurality of object annotated images in the source domain from having the first image style into having the second image style to generate a converted set of object annotated images having the second image style;
generating object annotation for the majority set of the plurality of unannotated images in the second image style based on the converted set of object annotated images in the second image style and the minority set of the plurality of object annotated images in the second image style to change from the majority set of a plurality of unannotated images into a majority set of a plurality of annotated images; and
performing an active domain adaptation by adapting the minority set of a plurality of object annotated images, the converted set of object annotated images, and the majority set of the plurality of annotated images so as to generate an object detection model.
