# US 2025/0159129 A1 · Image provision device and image provision method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0159129 A1
- Document ID: US20250159129A1
- Status: Published application
- Kind: A1
- Application number: 18/939,534
- Filed: 2024-11-07
- Published: 2025-05-15
- Earliest claimed: 2023-11-09
- Assignees: SK hynix Inc.
- Applicants: SK hynix Inc.
- Inventors: Yuta SUZUKI
- Web page: https://patentgrep.com/patent/US20250159129A1

## CPC

- H04N 13/383: for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/82: using neural networks
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06V 40/175: {Static expression}
- H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
- H04N 13/178: Metadata, e.g. disparity information
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An image provision device and an image provision method are disclosed. The image provision device includes a first pre-processor configured to generate a first image by detecting a first face region from each frame of an original video, a second pre-processor configured to generate a second image by detecting a second face region from an original image, and a similar image generator configured to generate similar images respectively corresponding to a viewer's gaze positions based on the first image and the second image.

## Claims

### Claim 1

An image provision device comprising:
a first pre-processor configured to generate a first image by detecting a first face region from each frame of an original video;
a second pre-processor configured to generate a second image by detecting a second face region from an original image; and
a similar image generator configured to generate similar images respectively corresponding to a viewer's gaze positions based on the first image and the second image.

### Claim 2 (depends on 1)

The image provision device according to claim 1, wherein the similar image generator includes:
a first feature extractor configured to generate first feature information representing features of a face included in the first image;
a second feature extractor configured to generate second feature information representing features of a face included in the second image;
a head rotation component configured to generate correction feature information by converting the first feature information in response to head angle information including head angles respectively corresponding to the gaze positions of the viewer; and
an image restoration component configured to generate the similar images based on the correction feature information, the second feature information, and the second image.

### Claim 3 (depends on 2)

The image provision device according to claim 2, wherein each of the first feature information and the second feature information includes:
landmark information including three-dimensional coordinates for each landmark of the face; and
rotation information including rotation angles obtained when the face is rotated about each of a first axis, a second axis, and a third axis perpendicular to each other.

### Claim 4 (depends on 3)

The image provision device according to claim 3, wherein the head rotation component is configured to calculate the three-dimensional coordinates of the first feature information changed according to rotation allowing the rotation angle of the first feature information to be equal to the head angle.

### Claim 5 (depends on 3)

The image provision device according to claim 3, wherein the landmark includes at least one of a nose, mouth, eyes, ears, and a chin.

### Claim 6 (depends on 3)

The image provision device according to claim 3, wherein the rotation angle includes a pitch angle, a yaw angle, and a roll angle.

### Claim 7 (depends on 2)

The image provision device according to claim 2, wherein each of the similar images is an image in which the face of the second image has a facial expression included in the first image and the head angle.

### Claim 8 (depends on 2)

The image provision device according to claim 2, wherein each of the first feature extractor and the second feature extractor includes a U-net encoder or a self-attention Generative Adversarial Networks (GAN) encoder.

### Claim 9 (depends on 2)

The image provision device according to claim 2, wherein the image restoration component is a U-net decoder or a self-attention GAN decoder.

### Claim 10 (depends on 2)

The image provision device according to claim 2, wherein the image restoration component is configured to generate the similar images by referring to previously created similar images.

### Claim 11 (depends on 1)

The image provision device according to claim 1, further comprising an image display configured to:
select a similar image corresponding to a current gaze position of the viewer, from among the similar images; and
output the selected similar image on a screen.

### Claim 12 (depends on 11)

The image provision device according to claim 11, wherein the image display is configured to:
obtain a synthetic image by replacing a face included in the first face region of the original video with a face included in the selected similar image; and
output the synthetic image on the screen.

### Claim 13 (depends on 1)

The image provision device according to claim 1, wherein the first pre-processor is configured to generate the first image by cropping the first face region from each frame of the original video.

### Claim 14 (depends on 1)

The image provision device according to claim 1, wherein the second pre-processor is configured to generate the second image by cropping the second face region from the original image.

### Claim 15 (depends on 1)

The image provision device according to claim 1, wherein the first pre-processor is configured to generate the first image by adjusting the first face region to a predetermined reference size.

### Claim 16 (depends on 15)

The image provision device according to claim 15, wherein the second pre-processor is configured to:
adjust a position of the face included in the second face region to a predetermined reference position; and
adjust the adjusted second face region to the reference size to generate the second image.

### Claim 17

An image provision device comprising:
a similar image generator configured to generate similar images respectively corresponding to a viewer's gaze positions based on a first image generated using each frame of an original video and a second image generated using an original image; and
an image display configured to select a similar image corresponding to a current gaze position of the viewer, from among the similar images, and output the selected similar image on a screen.

### Claim 18 (depends on 17)

The image provision device according to claim 17, wherein the similar image generator includes:
a first feature extractor configured to generate first feature information representing features of a face included in the first image;
a second feature extractor configured to generate second feature information representing features of a face included in the second image;
a head rotation component configured to generate correction feature information by converting the first feature information in response to head angle information including head angles respectively corresponding to the gaze positions of the viewer; and
an image restoration component configured to generate the similar images based on the correction feature information, the second feature information, and the second image.

### Claim 19 (depends on 18)

The image provision device according to claim 18, wherein each of the first feature information and the second feature information includes:
landmark information including three-dimensional coordinates for each landmark of the face; and
rotation information including rotation angles obtained when the face is rotated about each of a first axis, a second axis, and a third axis perpendicular to each other.

### Claim 20 (depends on 19)

The image provision device according to claim 19, wherein the head rotation component is configured to:
calculate the three-dimensional coordinates of the first feature information changed according to rotation allowing the rotation angle of the first feature information to be equal to the head angle.

### Claim 21 (depends on 18)

The image provision device according to claim 18, wherein:
each of the similar images is an image in which the face of the second image has a facial expression included in the first image and the head angle.

### Claim 22

An image provision method comprising:
generating a first image using each frame of an original video;
generating a second image using an original image;
generating similar images respectively corresponding to a viewer's gaze positions based on the first image and the second image; and
selecting a similar image corresponding to a current gaze position of the viewer, from among the generated similar images to output the selected similar image on a screen.

### Claim 23 (depends on 22)

The image provision method according to claim 22, wherein the generating the first image includes extracting a next frame when a face region is not detected in any one frame.

### Claim 24 (depends on 22)

The image provision method according to claim 22, wherein the generating the second image includes receiving another original image when a face region is not detected in the original image.
