# US 12,581,047 B2 · Scaling object to be processed from VR image based on motion vector of user in target direction determined from six-degrees-of-freedom data of VR headset

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,581,047 B2
- Document ID: US12581047B2
- Status: Granted
- Kind: B2
- Application number: 18/348,284
- Filed: 2023-07-06
- Published: 2024-01-11
- Granted: 2026-03-17
- Earliest claimed: 2022-07-08
- Assignees: Beijing Zitiao Network Technology Co., Ltd.
- Applicants: Beijing Zitiao Network Technology Co., Ltd.
- Inventors: Lei Li; Xinyu Cui
- Web page: https://patentgrep.com/patent/US12581047B2

## CPC

- G06T 7/162: involving graph-based methods
- H04N 13/139: Format conversion, e.g. of frame-rate or size
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 7/223: using block-matching
- H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
- H04N 13/128: Adjusting depth or disparity
- H04N 13/344: with head-mounted left-right displays
- H04N 13/366: using viewer tracking
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 2203/04806: Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
- G06T 3/02: Affine transformations (for image registration G06T3/147; for image mosaicing G06T3/4038)
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 7/11: Region-based segmentation
- G06T 7/593: from stereo images
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- H04N 2013/0085: {Motion estimation from stereoscopic image signals}
- H04N 2013/0092: {Image segmentation from stereoscopic image signals}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present disclosure provides an image processing method and apparatus, a device, and a medium. The method is applied in a virtual reality headset and includes: obtaining six-degree-of-freedom data of the virtual reality headset and a virtual reality image corresponding to the six-degree-of-freedom data; determining a motion vector of a user in a target direction based on the six-degree-of-freedom data; determining a to-be-processed object from the virtual reality image; and scaling the to-be-processed object based on the motion vector. In the present disclosure, stereoscopic scaling of a VR image displayed by a VR device is implemented to achieve a stereoscopic scaling effect of the image, thereby satisfying use requirements for a user.

## Claims

### Claim 1

An image processing method, applied in a virtual reality headset and comprising:
obtaining six-degree-of-freedom data of the virtual reality headset and a virtual reality image corresponding to the six-degree-of-freedom data;
determining a motion vector of a user in a target direction based on the six-degree-of-freedom data;
performing a depth information extraction processing on the virtual reality image to obtain depth information;
performing a semantic segmentation processing on the virtual reality image to obtain a segmentation result;
determining a to-be-processed object from the virtual reality image based on the depth information and the segmentation result; and
scaling the to-be-processed object based on the motion vector.

### Claim 2 (depends on 1)

The method according to claim 1, wherein said scaling the to-be-processed object based on the motion vector comprises:
performing macroblock segmentation on the virtual reality image based on the to-be-processed object to obtain a plurality of first macroblocks;
determining, from the plurality of first macroblocks, one or more target macroblocks corresponding to the motion vector; and
scaling the one or more target macroblocks based on the motion vector.

### Claim 3 (depends on 2)

The method according to claim 2, wherein said determining, from the plurality of first macroblocks, the one or more target macroblocks corresponding to the motion vector comprises:
performing macroblock segmentation on a virtual reality image corresponding to the motion vector to obtain a plurality of second macroblocks;
determining one or more second macroblocks corresponding to the motion vector; and
determining the one or more target macroblocks from the plurality of first macroblocks based on the one or more second macroblocks.

### Claim 4 (depends on 2)

The method according to claim 2, wherein the to-be-processed object is composed of the one or more target macroblocks.

### Claim 5 (depends on 1)

The method according to claim 1, further comprising, prior to said scaling the to-be-processed object based on the motion vector:
adjusting a magnitude of the motion vector based on depth information of the to-be-processed object and a scaling coefficient of the to-be-processed object.

### Claim 6

An electronic device, comprising:
a processor;
a memory configured to store a computer program,
wherein the processor is configured to invoke and execute the computer program stored in the memory to:
obtain six-degree-of-freedom data of a virtual reality headset and a virtual reality image corresponding to the six-degree-of-freedom data;
determine a motion vector of a user in a target direction based on the six-degree-of-freedom data;
perform a depth information extraction processing on the virtual reality image to obtain depth information;
perform a semantic segmentation processing on the virtual reality image to obtain a segmentation result;
determine a to-be-processed object from the virtual reality image based on the depth information and the segmentation result;
scale the to-be-processed object based on the motion vector.

### Claim 7 (depends on 6)

The electronic device according to claim 6, wherein the processor is further configured to invoke and execute the computer program stored in the memory to:
perform macroblock segmentation on the virtual reality image based on the to-be-processed object to obtain a plurality of first macroblocks;
determine, from the plurality of first macroblocks, one or more target macroblocks corresponding to the motion vector; and
scale the one or more target macroblocks based on the motion vector.

### Claim 8 (depends on 7)

The electronic device according to claim 7, wherein the processor is further configured to invoke and execute the computer program stored in the memory to:
perform macroblock segmentation on a virtual reality image corresponding to the motion vector to obtain a plurality of second macroblocks;
determine one or more second macroblocks corresponding to the motion vector; and
determine the one or more target macroblocks from the plurality of first macroblocks based on the one or more second macroblocks.

### Claim 9 (depends on 7)

The electronic device according to claim 7, wherein the to-be-processed object is composed of the one or more target macroblocks.

### Claim 10 (depends on 6)

The electronic device according to claim 6, wherein the processor is further configured to invoke and execute the computer program stored in the memory to, prior to said scaling the to-be-processed object based on the motion vector;
adjust a magnitude of the motion vector based on the depth information and a scaling coefficient of the to-be-processed object.

### Claim 11

A non-transitory computer-readable storage medium, configured to store a computer program, wherein the computer program enables a computer to:
obtain six-degree-of-freedom data of a virtual reality headset and a virtual reality image corresponding to the six-degree-of-freedom data;
determine a motion vector of a user in a target direction based on the six-degree-of-freedom data;
perform a depth information extraction processing on the virtual reality image to obtain depth information;
perform a semantic segmentation processing on the virtual reality image to obtain a segmentation result;
determine a to-be-processed object from the virtual reality image based on the depth information and the segmentation result; and
scale the to-be-processed object based on the motion vector.

### Claim 12 (depends on 11)

The non-transitory computer-readable storage medium according to claim 11, wherein the computer program further enables the computer to:
perform macroblock segmentation on the virtual reality image based on the to-be-processed object to obtain a plurality of first macroblocks;
determine, from the plurality of first macroblocks, one or more target macroblocks corresponding to the motion vector; and
scale the one or more target macroblocks based on the motion vector.

### Claim 13 (depends on 12)

The non-transitory computer-readable storage medium according to claim 12, wherein the computer program further enables the computer to:
perform macroblock segmentation on a virtual reality image corresponding to the motion vector to obtain a plurality of second macroblocks;
determine one or more second macroblocks corresponding to the motion vector; and
determine the one or more target macroblocks from the plurality of first macroblocks based on the one or more second macroblocks.

### Claim 14 (depends on 12)

The non-transitory computer-readable storage medium according to claim 12, wherein the to-be-processed object is composed of the one or more target macroblocks.

### Claim 15 (depends on 11)

The non-transitory computer-readable storage medium according to claim 11, wherein the computer program further enables the computer to, prior to said scaling the to-be-processed object based on the motion vector:
adjust a magnitude of the motion vector based on the depth information and a scaling coefficient of the to-be-processed object.