# US 2025/0095174 A1 · Learning Device, Learning Method And Test Device, Test Method Using The Same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095174 A1
- Document ID: US20250095174A1
- Status: Published application
- Kind: A1
- Application number: 18/658,739
- Filed: 2024-05-08
- Published: 2025-03-20
- Earliest claimed: 2023-09-20
- Assignees: Hyundai Motor Company; Kia Corporation
- Applicants: Hyundai Motor Company; Kia Corporation
- Inventors: Jin Ho Park; Jin Sol Kim; Jang Yoon Kim
- Web page: https://patentgrep.com/patent/US20250095174A1

## CPC

- G06T 7/50: Depth or shape recovery
- G06T 7/55: from multiple images
- G06T 7/248: {involving reference images or patches}
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A learning device, a learning method thereof, a test device using the same, and a test method using the same are provided. The learning device may obtain a target image and a source image, generate an estimated depth map based on the target image via a first network, generate pose change information corresponding to a pose change between the target image and the source image, generate a composite image corresponding to the target image, determine a first loss based on the composite image and the target image, and determine a second loss, and back-propagate the first loss and the second loss and update a parameter of the first network and a parameter of the second network.

## Claims

### Claim 1

A learning device comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, configure the learning device to:
obtain a target image and a source image;
generate, via a first network, an estimated depth map based on the target image;
generate, via a second network, pose change information corresponding to a pose change between the target image and the source image;
generate a composite image corresponding to the target image by using the estimated depth map, the pose change information, and the source image;
determine, based on the composite image and the target image, a first loss;
determine, based on a pseudo depth map corresponding to the target image and the estimated depth map, a second loss; and
back-propagate the first loss and the second loss, and update a parameter of the first network and a parameter of the second network.

### Claim 2 (depends on 1)

The learning device of claim 1, wherein the instructions, when executed by the one or more processors, configure the learning device to determine the second loss by:
determining, via the loss calculation device, the second loss further based on at least one of luminance information, contrast information, or structure information, of each of the pseudo depth map and the estimated depth map.

### Claim 3 (depends on 2)

The learning device of claim 2, wherein a first weight corresponding to the luminance information is smaller than a second weight corresponding to the contrast information and smaller than a third weight corresponding to the structure information.

### Claim 4 (depends on 1)

The learning device of claim 1, wherein the target image is generated by an image sensor at a first time, and
wherein the source image is generated by the image sensor at a second time within a threshold range around the first time.

### Claim 5 (depends on 4)

The learning device of claim 4, wherein the second network comprises a pose change information generation network, and
wherein the instructions, when executed by the one or more processors, configure the learning device to generate the pose change information by generating, via the second network, the pose change information based on a first pose at the first time and a second pose at the second time.

### Claim 6 (depends on 4)

The learning device of claim 4, wherein the instructions, when executed by the one or more processors, configure the learning device to:
obtain, based on the estimated depth map, first three-dimensional (3D) point cloud information at the first time;
convert, based on the pose change information, the first 3D point cloud information at the first time into second 3D point cloud information at the second time;
convert, based on an image sensor parameter corresponding to the image sensor, the second 3D point cloud information into two-dimensional (2D) image coordinates; and
generate the composite image by generating the composite image based on a pixel value of the source image corresponding to the 2D image coordinates.

### Claim 7 (depends on 1)

The learning device of claim 1, wherein the first network comprises an estimated depth map generation network, and
wherein the instructions, when executed by the one or more processors, further configure the learning device to:
generate, via a pseudo depth map generation network, the pseudo depth map based on the target image.

### Claim 8 (depends on 7)

The learning device of claim 7, wherein the pseudo depth map generation network comprises a parameter in a frozen state.

### Claim 9

A system comprising:
a test device comprising:
an acquisition device configured to obtain a target image for testing; and
a first network configured to generate, based on the target image, an estimated depth map for testing; and
a learning device configured to:
obtain the target image and a source image;
generate, via a second network, pose change information corresponding to a pose change between the target image and the source image;
generate a composite image corresponding to the target image by using the estimated depth map, the pose change information, and the source image;
determine, based on the composite image and the target image, a first loss;
determine, based on a pseudo depth map corresponding to the target image and the estimated depth map, a second loss; and
back-propagate the first loss and the second loss and update a parameter of the first network and a parameter of the second network,
wherein the test device is configured to perform testing based on the updated parameters.

### Claim 10 (depends on 9)

The system of claim 9, wherein the test device is configured to determine the second loss by:
determining the second loss further based on at least one of luminance information, contrast information, or structure information, of each of the pseudo depth map and the estimated depth map.

### Claim 11 (depends on 10)

The system of claim 10, wherein a first weight corresponding to the luminance information is smaller than a second weight corresponding to the contrast information and smaller than a third weight corresponding to the structure information.

### Claim 12 (depends on 9)

The system of claim 9, wherein the target image is generated by an image sensor at a first time, and
wherein the source image is generated by the image sensor at a second time within a threshold range around the first time.

### Claim 13 (depends on 12)

The system of claim 12, wherein the second network comprises a pose change information generation network, and wherein the test device is configured to generate the pose change information by generating, via the pose change information generation network, the pose change information based on a first pose at the first time and a second pose at the second time.

### Claim 14

A learning method comprising:
obtaining, by one or more processors, a target image and a source image;
generating, by the one or more processors, an estimated depth map based on the target image;
generating, by the one or more processors, pose change information corresponding to a pose change between the target image and the source image;
generating, by the one or more processors, a composite image corresponding to the target image by using the estimated depth map, the pose change information, and the source image;
determining, by the one or more processors and based on the composite image and the target image, a first loss;
determining, by the one or more processors and based on a pseudo depth map corresponding to the target image and the estimated depth map, a second loss; and
back-propagating, by the one or more processors, the first loss and the second loss and updating a parameter of a first network for generating the estimated depth map and a parameter of a second network for generating the pose change information.

### Claim 15 (depends on 14)

The learning method of claim 14, wherein the determining of the second loss comprises:
determining the second loss further based on at least one of luminance information, contrast information, or structure information, of each of the pseudo depth map and the estimated depth map.

### Claim 16 (depends on 15)

The learning method of claim 15, wherein a first weight corresponding to the luminance information is smaller than a second weight corresponding to the contrast information and smaller than a third weight corresponding to the structure information.

### Claim 17 (depends on 14)

The learning method of claim 14, wherein the target image is generated by an image sensor at a first time, and
wherein the source image is generated by the image sensor at a second time within a threshold range around the first time.

### Claim 18 (depends on 17)

The learning method of claim 17, wherein the generating of the pose change information comprises:
generating the pose change information based on a first pose at the first time and a second pose at the second time.

### Claim 19 (depends on 17)

The learning method of claim 17, wherein the generating of the composite image comprises:
obtaining first three-dimensional (3D) point cloud information at the first time;
converting, based on the pose change information, the first 3D point cloud information at the first time into second 3D point cloud information at the second time;
converting, based on an image sensor parameter corresponding to the image sensor, the second 3D point cloud information into two-dimensional (2D) image coordinates; and
generating the composite image based on a pixel value of the source image corresponding to the 2D image coordinates.

### Claim 20 (depends on 14)

The learning method of claim 14, further comprising:
generating, by a pre-trained pseudo depth map generation network, the pseudo depth map based on the target image before determining the second loss.
