# US 12,225,180 B2 · Method and apparatus for generating stereoscopic display contents

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,225,180 B2
- Document ID: US12225180B2
- Status: Granted
- Kind: B2
- Application number: 17/973,086
- Filed: 2022-10-25
- Published: 2024-04-25
- Granted: 2025-02-11
- Earliest claimed: 2022-10-25
- Assignees: Orbbec 3D Technology International, Inc.
- Applicants: Orbbec 3D Technology International, Inc.
- Inventors: Xin Xie; Nan Xu; Xu Chen
- Web page: https://patentgrep.com/patent/US12225180B2

## CPC

- H04N 13/332: Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N 13/128: Adjusting depth or disparity
- H04N 13/207: using a single two-dimensional [2D] image sensor
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method of generating stereoscopic display contents includes obtaining, from a Red, Green, Blue plus Distance (RGB-D) image using a processor, a first Red, Green, and Blue (RGB) image and a depth image; determining, based on depth values in the depth image, a first disparity map in accordance with the RGB-D image; determining a second disparity map and a third disparity map by transforming the first disparity map using a disparity distribution ratio; and generating, by the processor, the pair of stereoscopic images comprising a second RGB image and a third RGB image, wherein the second RGB image is generated by shifting a first set of pixels in the first RGB image based on the second disparity map, and the third RGB image is generated by shifting a second set of pixels in the first RGB image based on the third disparity map.

## Claims

### Claim 1

A method of generating stereoscopic display contents, comprising:
obtaining, from a Red, Green, Blue plus Distance (RGB-D) image using a processor, a first Red, Green, and Blue (RGB) image and a depth image;
determining, based on depth values in the depth image, a first disparity map in accordance with the RGB-D image using at least one of a focal length f or an interpupillary distance b, wherein the first disparity map comprises a plurality of disparity values for the first RGB image to be transformed to a pair of stereoscopic images;
determining a left disparity map and a right disparity map by transforming the first disparity map using a disparity distribution ratio k, wherein the left disparity map is determined based on
d
L
(
x
,
y
)
=
k
*
d
⁡
(
x
,
y
)
=
k
*
(
f
*
b
z
⁡
(
x
,
y
)
)
,
and the right disparity map is determined based on
d
R
(
x
,
y
)
=
(
k
-
1
)
*
d
⁡
(
x
,
y
)
=
(
k
-
1
)
*
(
f
*
b
z
⁡
(
x
,
y
)
)
,
wherein z(x,y) represents the depth value of a pixel (x,y) in the RGB-D image, d(x, y) represents the disparity value in the first disparity map, dL(x, y) represents the disparity value in the left disparity map, dR(x, y) represents the disparity value in the right disparity map, and k is a constant value indicative of a position of an observation point between left and right eye; and
generating, by the processor, the pair of stereoscopic images from the first RGB image, the pair of stereoscopic images comprising a left eye image and a right eye image, wherein the left eye image is generated by shifting a first set of pixels in the first RGB image based on the left disparity map, and the right eye image is generated by shifting a second set of pixels in the first RGB image based on the disparity map.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating, based on the pair of stereoscopic images by the processor, a pair of adjusted display images resized to display requirements of an augmented reality (AR), virtual reality (VR), or mixed reality (MR) device, the pair of adjusted display images comprising a left eye display image derived from the left eye image and a right eye display image derived from the right eye image.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first disparity map is a two-dimensional (2D) matrix, wherein each element indicates a disparity value associated with a pixel in the first RGB image.

### Claim 4 (depends on 1)

The method of claim 1, wherein the RGB-D image is captured by an RGB-D sensor, and the depth value of a pixel in the depth image indicates a distance between the RGB-D sensor and a corresponding object captured in the first RGB image.

### Claim 5 (depends on 1)

The method of claim 1, wherein shifting the first set of pixels in the first RGB image based on the left disparity map is based on:
PixelL(x, y)=Pixel(x+dL,y)=(R(x+dL, y), G(x+dL, y), B(x+dL, y)); and shifting the second set of pixels in the first RGB image based on the right disparity map is based on:
PixelR(x, y)=Pixel(x+dR, y)=(R(x+dR, y), G(x+dR, y), B(x+dR, y)), wherein PixelL(x, y) is a pixel (x, y) in the left eye image, PixelR(x, y) is a pixel (x, y) in the right eye image, Pixel(x, y) is a pixel (x, y) in the first RGB image, and (R(x,y), G(x,y), B(x,y)) is a RGB color for the pixel (x, y).

### Claim 6 (depends on 1)

The method of claim 1, wherein determining, based on the depth values in the depth image, the first disparity map in accordance with the RGB-D image comprises: determining the first disparity map based on the focal length f, the interpupillary distance b, and the depth values in the depth image.

### Claim 7

An apparatus for generating stereoscopic display contents, comprising:
a non-transitory memory; and
a processor, wherein the non-transitory memory includes instructions executable by the processor to:
obtain, from a Red, Green, Blue plus Distance (RGB-D) image, a first Red, Green, and Blue (RGB) image and a depth image;
determine, based on depth values in the depth image, a first disparity map in accordance with the RGB-D image using at least one of a focal length f or an interpupillary distance b, wherein the first disparity map comprises a plurality of disparity values for the first RGB image to be transformed to a pair of stereoscopic images;
determine a left disparity map and a right disparity map by transforming the first disparity map using a disparity distribution ratio k, wherein the left disparity map is determined based on
d
L
(
x
,
y
)
=
k
*
d
⁡
(
x
,
y
)
=
k
*
(
f
*
b
z
⁡
(
x
,
y
)
)
,
and the right disparity map is determined based on
d
R
(
x
,
y
)
=
(
k
-
1
)
*
d
⁡
(
x
,
y
)
=
(
k
-
1
)
*
(
f
*
b
z
⁡
(
x
,
y
)
)
,
wherein z(x,y) is the depth value of a pixel (x,y) in the RGB-D image, d(x, y) is the disparity value in the first disparity map, dL(x, y) is the disparity value in the left disparity map, dR(x, y) is the disparity value in the right disparity map, and k is a constant value indicative of a position of an observation point between left and right eye; and
generate the pair of stereoscopic images from the first RGB image, the pair of stereoscopic images comprising a left eye image and a right eye image, wherein the left eye image is generated by shifting a first set of pixels in the first RGB image based on the left disparity map, and the right eye image is generated by shifting a second set of pixels in the first RGB image based on the right disparity map.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the instructions executable by the processor further comprise instructions to:
generate, based on the pair of stereoscopic images, a pair of adjusted display images resized to display requirements of an augmented reality (AR), virtual reality (VR), or mixed reality (MR) device.

### Claim 9 (depends on 7)

The apparatus of claim 7, wherein the first disparity map is a two-dimensional (2D) matrix, wherein each element indicates a disparity value.

### Claim 10 (depends on 7)

The apparatus of claim 7, wherein the RGB-D image is captured by an RGB-D sensor, the left eye image comprises a second RGB image, and the right eye image comprises a third RGB image.

### Claim 11 (depends on 10)

The apparatus of claim 10, where a pixel in the depth image indicates a distance between the RGB-D sensor and a corresponding object captured in the first RGB image.

### Claim 12 (depends on 7)

The apparatus of claim 7, wherein shifting the first set of pixels in the first RGB image based on the left disparity map is based on:
PixelL(x, y)=Pixel(x+dL,y)=(R(x+dL, y), G(x+dL, y), B(x+dL, y)); and shifting the second set of pixels in the first RGB image based on the right disparity map is based on:
PixelR(x, y)=Pixel(x+dR, y)=(R(x+dR, y), G(x+dR, y), B(x+dR, y)), wherein PixelL(x, y) is a pixel (x, y) in the left eye image, PixelR(x, y) is a pixel (x, y) in the right eye image, Pixel(x, y) is a pixel (x, y) in the first RGB image, and (R(x,y), G(x,y), B(x,y)) is a RGB color for the pixel (x, y).

### Claim 13

A non-transitory computer-readable storage medium configured to store computer programs for generating stereoscopic display contents, the computer programs comprising instructions executable by a processor to:
obtain, from a Red, Green, Blue plus Distance (RGB-D) image, a first Red, Green, and Blue (RGB) image and a depth image;
determine, based on depth values in the depth image, a first disparity map in accordance with the RGB-D image using at least one of a focal length f or an interpupillary distance b, wherein the first disparity map comprises a plurality of disparity values for the first RGB image to be transformed to a pair of stereoscopic images;
determine a left disparity map and a right disparity map by transforming the first disparity map using a disparity distribution ratio k, wherein the left disparity map is determined based on
d
L
(
x
,
y
)
=
k
*
d
⁡
(
x
,
y
)
=
k
*
(
f
*
b
z
⁡
(
x
,
y
)
)
,
and the right disparity map is determined based on
d
R
(
x
,
y
)
=
(
k
-
1
)
*
d
⁡
(
x
,
y
)
=
(
k
-
1
)
*
(
f
*
b
z
⁡
(
x
,
y
)
)
,
wherein z(x, y) is the depth value of a pixel (x,y) in the RGB-D image, d(x, y) is the disparity value in the first disparity map, dL(x, y) is the disparity value in the left disparity map, dR(x, y) is the disparity value in the right disparity map, and k is a constant value indicative of a position of an observation point between left and right eye; and
generate, by the processor, the pair of stereoscopic images comprising a left eye image and a right eye image, wherein the left eye image is generated by shifting a first set of pixels in the first RGB image based on the left disparity map, and the right eye image is generated by shifting a second set of pixels in the first RGB image based on the right disparity map.

### Claim 14 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the instructions executable by the processor further comprises instructions to:
generate, based on the pair of stereoscopic images by the processor, a pair of adjusted display images resized to display requirements of an augmented reality (AR), virtual reality (VR), or mixed reality (MR) device.

### Claim 15 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the first disparity map is a two-dimensional (2D) matrix, wherein each element indicates a disparity value.

### Claim 16 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein the RGB-D image is captured by an RGB-D sensor.