# US 2016/0259403 A1 · Interactive Video Display Method, Device, and System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0259403 A1
- Document ID: US20160259403A1
- Status: Published application
- Kind: A1
- Application number: 15/060,976
- Filed: 2016-03-04
- Published: 2016-09-08
- Earliest claimed: 2015-03-04
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Kai Wang; Shiguo Lian; Zhaoxiang Liu
- Web page: https://patentgrep.com/patent/US20160259403A1

## CPC

- G06F 3/005: {Input arrangements through a video camera}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/1415: {with means for detecting differences between the image stored in the host and the images displayed on the displays}
- G06F 3/1462: {with means for detecting differences between the image stored in the host and the images displayed on the remote displays}
- G06F 3/147: using display panels
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 40/11: {Hand-related biometrics; Hand pose recognition}
- H04N 23/61: based on recognised objects
- H04N 23/63: by using electronic viewfinders
- H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet
- H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
- G06F 2203/012: Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
- G09G 2320/02: Improving the quality of display appearance
- G09G 2320/0261: in the context of movement of objects on the screen or movement of the observer relative to the screen
- G09G 2340/04: Changes in size, position or resolution of an image
- G09G 2340/0464: Positioning
- G09G 2370/022: Centralised management of display operation, e.g. in a server instead of locally
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A method, a device, and a system for interactive video display, relate to the field of multimedia technologies, and can resolve a problem in the prior art that browsing experience of a user is not real because a visible region is simply changed only according to a panning change of a location of the user. A specific solution is acquiring posture information of a user, where the posture information includes viewing angle information of the user and location information of the user, and adjusting, according to the posture information, an unadjusted visible region, in a video, for being displayed on a client in order to obtain an adjusted visible region currently for being displayed on the client. The embodiments of the present invention are used for performing interactive video display.

## Claims

### Claim 1

An interactive video display method, comprising:
acquiring posture information of a user, wherein the posture information comprises viewing angle information of the user and location information of the user; and
adjusting, according to the posture information, an unadjusted visible region, in a video, for being displayed on a client in order to obtain an adjusted visible region currently for being displayed on the client.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the location information of the user comprises a vertical distance between the user and a plane of the client, and wherein adjusting, according to the posture information, the unadjusted visible region, in the video, for being displayed on the client in order to obtain the adjusted visible region currently for being displayed on the client comprises:
acquiring at least one of a region size change value and region displacement of the unadjusted visible region according to the viewing angle information of the user and the vertical distance, wherein the region size change value comprises a width change value of the unadjusted visible region and a height change value of the unadjusted visible region, and wherein the region displacement comprises horizontal displacement of the unadjusted visible region and vertical displacement of the unadjusted visible region; and
adjusting the unadjusted visible region according to the at least one of the region size change value and the region displacement in order to obtain the adjusted visible region.

### Claim 3 (depends on 2)

The method according to claim 2, wherein adjusting the unadjusted visible region according to the at least one of the region size change value and the region displacement in order to obtain the adjusted visible region comprises:
adjusting an unadjusted region of a virtual window picture according to the region size change value and the region displacement in order to obtain an adjusted region of the virtual window picture when the vertical distance is greater than or equal to a preset distance threshold;
fully covering an unadjusted region of a dynamic image with an adjusted picture that is presented in the adjusted region of the virtual window picture to obtain the adjusted visible region when the vertical distance is greater than or equal to the preset distance threshold, wherein the video is the dynamic image on which the virtual window picture is superimposed, wherein the virtual window picture comprises a transparent region, a range that is in the dynamic image and that is covered by the transparent region of the virtual window picture is visible, and wherein the unadjusted visible region is obtained by superimposing the unadjusted region of the virtual window picture on the unadjusted region of the dynamic image; and
adjusting the unadjusted visible region according to the region displacement in order to obtain the adjusted visible region when the vertical distance is less than the preset distance threshold.

### Claim 4 (depends on 1)

The method according to claim 1, wherein viewing angle information of the user comprises face orientation information of the user, and wherein the face orientation information of the user comprises an angle by which the face orientation of the user rotates around a coronal axis and the angle by which the face orientation of the user rotates around a vertical axis.

### Claim 5 (depends on 4)

The method according to claim 4, wherein a region size change value of the unadjusted visible region is acquired using the following formulas:
Δ


w
=
z
u
*
(
tan

(
θ
fov
2
+
θ
u
y
)
+
tan

(
θ
fov
2
-
θ
u
y
)
)
-
2
*
z
u
*
tan

(
θ
fov
2
)
;
and
Δ


h
=
z
u
*
(
tan

(
θ
fov
2
+
θ
u
x
)
+
tan

(
θ
fov
2
-
θ
u
x
)
)
-
2
*
z
u
*
tan

(
θ
fov
2
)
,
wherein Δw represents a width change value of the unadjusted visible region, wherein Δh represents a height change value of the unadjusted visible region, wherein θux represents the angle by which the face orientation of the user rotates around the coronal axis, wherein θuy represents the angle by which the face orientation of the user rotates around the vertical axis, wherein θfov represents a maximum visual angle of human eyes, and wherein zu represents a vertical distance.

### Claim 6 (depends on 5)

The method according to claim 5, wherein a region displacement of the unadjusted visible region is acquired using the following formulas:
x
v
=
1
2
*
z
u
*
(
tan

(
θ
fov
2
+
θ
u
y
)
-
tan

(
θ
fov
2
-
θ
u
y
)
)
;
and
y
v
=
1
2
*
z
u
*
(
tan

(
θ
fov
2
+
θ
u
x
)
-
tan

(
θ
fov
2
-
θ
u
x
)
)
,
wherein xv represents a horizontal displacement of the unadjusted visible region, wherein yv represents a vertical displacement of the unadjusted visible region, wherein θux represents the angle by which the face orientation of the user rotates around the coronal axis, wherein θuy represents the angle by which the face orientation of the user rotates around the vertical axis, wherein θfov represents the maximum visual angle of human eyes, and wherein zu represents the vertical distance.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the posture information of the user comprises a change value of a vertical distance between the user and a plane of the client, and wherein adjusting, according to the posture information, the unadjusted visible region, in the video, for being displayed on the client in order to obtain the adjusted visible region currently for being displayed on the client comprises:
acquiring a region size change value of the unadjusted visible region according to the change value of the vertical distance, and wherein the region size change value comprises a width change value of the unadjusted visible region and a height change value of the unadjusted visible region; and
adjusting the unadjusted visible region according to the region size change value in order to obtain the adjusted visible region.

### Claim 8 (depends on 7)

The method according to claim 7, wherein a formula for acquiring the region size change value of the unadjusted visible region according to the change value of the vertical distance comprises
Δ


w
=
Δ


h
=
Δ


z
u
*
tan

(
θ
fov
2
)
*
2
,
wherein Δw represents the width change value of the unadjusted visible region, wherein Δh represents the height change value of the unadjusted visible region, wherein θfov represents a maximum visual angle of human eyes, and wherein Δzu represents the change value of the vertical distance.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the method is executed by the client, and wherein after adjusting, according to the posture information, the unadjusted visible region, in the video, for being displayed on the client in order to obtain the adjusted visible region currently for being displayed on the client, the method further comprises:
requesting, from a server, a bitstream file corresponding to a region in the video that covers the adjusted visible region; and
receiving the bitstream file.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the video is divided into n sub-regions, wherein n is a positive integer greater than 1, wherein adjacent sub-regions in the n sub-regions have an overlapped region and the adjacent sub-regions have non-overlapped regions, and wherein the region in the video that covers the adjusted visible region is a to-be-downloaded sub-region in the n sub-regions that has a region overlapped with the adjusted visible region.

### Claim 11 (depends on 10)

The method according to claim 10, wherein requesting, from the server, the bitstream file corresponding to the region in the video that covers the adjusted visible region comprises:
determining, according to a media presentation description that is of the video and that is obtained from the server, address information of the bitstream file corresponding to the to-be-downloaded sub-region, wherein the media presentation description carries a parameter that is used to indicate a sub-region, in the video, corresponding to the address information; and
requesting, from the server, according to the address information, the bitstream file corresponding to the to-be-downloaded sub-region.

### Claim 12 (depends on 1)

The method according to claim 1, wherein the method is executed by the client, and wherein after adjusting, according to the posture information, the unadjusted visible region, in the video, for being displayed on the client in order to obtain the adjusted visible region currently for being displayed on the client, the method further comprises:
predicting a next visible region according to the unadjusted visible region and the adjusted visible region;
requesting, from a server, a bitstream file corresponding to a region in the video that covers the next visible region; and
receiving the bitstream file.

### Claim 13 (depends on 1)

The method according to claim 1, wherein the method is executed by the client, wherein the video is shot by a camera device, wherein a camera of the camera device is rotatable, wherein the video comprises a buffer region, wherein the unadjusted visible region is within a region range of the buffer region, wherein the buffer region is within a region range of the video, and wherein after adjusting, according to the posture information, the unadjusted visible region, in the video, for being displayed on the client in order to obtain the adjusted visible region currently for being displayed on the client, the method further comprises:
instructing the camera device to rotate the camera in an adjustment direction when the adjusted visible region comprises a region outside the buffer region, wherein the adjustment direction is a direction from the unadjusted visible region to the adjusted visible region; and
redefining an adjusted buffer region such that the adjusted visible region is within a region range of the adjusted buffer region.

### Claim 14 (depends on 1)

The method according to claim 1, wherein the method is executed by a server, and wherein after adjusting, according to the posture information, an unadjusted visible region, in a video, for being displayed on a client in order to obtain an adjusted visible region currently for being displayed on the client, the method further comprises sending, to the client, a bitstream file corresponding to a region in the video that covers the adjusted visible region.

### Claim 15 (depends on 14)

The method according to claim 14, wherein the video is divided into n sub-regions, wherein n is a positive integer greater than 1, wherein adjacent sub-regions in the n sub-regions have an overlapped region and the adjacent sub-regions have non-overlapped regions, and wherein the region in the video that covers the adjusted visible region is a to-be-downloaded sub-region in the n sub-regions that has a region overlapped with the adjusted visible region.

### Claim 16 (depends on 15)

The method according to claim 15, wherein sending, to the client, the bitstream file corresponding to the region in the video that covers the adjusted visible region comprises:
determining, according to a generated media presentation description of the video, address information of the bitstream file corresponding to the to-be-downloaded sub-region, wherein the media presentation description carries a parameter that is used to indicate a sub-region, in the video, corresponding to the address information; and
sending, to the client, according to the address information, the bitstream file corresponding to the to-be-downloaded sub-region.

### Claim 17 (depends on 1)

The method according to claim 1, wherein the method is executed by a server, and wherein after adjusting, according to the posture information, the unadjusted visible region, in the video, for being displayed on the client in order to obtain the adjusted visible region currently for being displayed on the client, the method further comprises:
predicting a next visible region according to the unadjusted visible region and the adjusted visible region; and
sending, to the client, a bitstream file corresponding to a region in the video that covers the next visible region.

### Claim 18 (depends on 1)

The method according to claim 1, wherein the method is executed by a server, wherein the video is shot by a camera device, wherein a camera of the camera device is rotatable, wherein the video comprises a buffer region, wherein the unadjusted visible region is within a region range of the buffer region, wherein the buffer region is within a region range of the video, and wherein after adjusting, according to the posture information, the unadjusted visible region, in the video, for being displayed on the client in order to obtain the adjusted visible region currently for being displayed on the client, the method further comprises:
instructing the camera device to rotate the camera in an adjustment direction when the adjusted visible region comprises a region outside the buffer region, wherein the adjustment direction is a direction from the unadjusted visible region to the adjusted visible region; and
redefining an adjusted buffer region such that the adjusted visible region is within a region range of the adjusted buffer region.

### Claim 19 (depends on 11)

The method according to claim 11, wherein the media presentation description comprises a media presentation description having a viewing angle (VMPD), and wherein the VMPD comprises address information of the n sub-regions in each time segment obtained by performing compression coding at different bit rates.

### Claim 20

A computer device, comprising:
a memory; and
a processor coupled to the memory, wherein the computer processor is configured to:
acquire posture information of a user, and wherein the posture information comprises viewing angle information of the user and location information of the user; and
adjust, according to the posture information an unadjusted visible region, in a video, for being displayed on a client in order to obtain an adjusted visible region currently for being displayed on the client.