# US 10,769,754 B2 · Virtual reality cinema-immersive movie watching for headmounted displays > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,754 B2 - Document ID: US10769754B2 - Status: Granted - Kind: B2 - Application number: 16/660,548 - Filed: 2019-10-22 - Published: 2020-04-30 - Granted: 2020-09-08 - Earliest claimed: 2018-10-31 - Assignees: Dolby Laboratories Licensing Corporation - Applicants: Dolby Laboratories Licensing Corporation - Inventors: Alexandre Chapiro; Chaitanya Atluru; Chun Chi Wan; Haricharan Lakshman; William Rozzi; Shane Ruggieri; Ajit Ninan - Web page: https://patentgrep.com/patent/US10769754B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06T 3/18: Image warping, e.g. rearranging pixels individually - G02B 27/017: {Head mounted} - G02B 27/0172: {characterised by optical features} - G06F 3/012: {Head tracking input arrangements} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images - G06T 19/003: {Navigation within 3D models or images} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G02B 2027/0123: {comprising devices increasing the field of view} - G02B 2027/0134: {of stereoscopic type} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract Peripheral-vision expanded images are streamed to a video streaming client. The peripheral-vision expanded images are generated from source images in reference to view directions of the viewer at respective time points. View direction data is collected and received in real time while the viewer is viewing display images derived from the peripheral-vision expanded images. A second peripheral-vision expanded image is generated from a second source image in reference to a second view direction of the viewer at a second time point. The second peripheral-vision expanded image has a focal-vision image portion covering the second view direction of the viewer and a peripheral-vision image portion outside the focal-vision image portion. The second peripheral-vision expanded image is transmitted to the video streaming client. ## Claims ### Claim 1 A method for providing image data to support immersive media content consumption experiences, comprising: streaming to a video streaming client, one or more peripheral-vision expanded images, the one or more peripheral-vision expanded images to be used by the video streaming client to generate one or more display images for rendering to a viewer at one or more time points, the one or more peripheral-vision expanded images being generated from one or more source images in reference to one or more view directions of the viewer at the one or more time points; receiving at least a part of view direction data of the viewer collected in real time while the viewer is viewing the one or more display images, the view direction data being used to determine a second view direction of the viewer at a second time point subsequent to the one or more time points; generating a second peripheral-vision expanded image from a second source image in reference to the second view direction of the viewer at the second time point, the second peripheral-vision expanded image to be used by the video streaming client to generate a second display image for rendering to the viewer at the second time point, the second peripheral-vision expanded image having a focal-vision image portion covering the second view direction of the viewer and a peripheral-vision image portion outside the focal-vision image portion, the focal-vision image portion of the second peripheral-vision expanded image being generated from a first corresponding source image portion in the second source image to cover the viewer's focal vision as covered by the first corresponding source image portion, the peripheral-vision image portion of the second peripheral-vision expanded image being generated by spatially stretching a second corresponding source image portion in the second source image to cover the viewer's peripheral vision including one or more peripheral vision portions not covered by the second corresponding source image portion of the second source image; transmitting the second peripheral-vision expanded image to the video streaming client. ### Claim 2 (depends on 1) The method of claim 1, further comprising: generating image metadata to be transmitted to the video streaming client, wherein the image metadata is used by the video streaming client to generate peripheral-vision expanded video content. ### Claim 3 (depends on 1) The method of claim 1, wherein the second peripheral-vision expanded image represents one of: an immersive image, a panorama image, an augmented reality image, a virtual reality image, or a remote presence image. ### Claim 4 (depends on 1) The method of claim 1, wherein the one or more time points and the second time point collectively represent consecutive time points covering a time interval. ### Claim 5 (depends on 1) The method of claim 1, wherein the second focal-vision image portion in the second display image covers the viewer's foveal-vision up to a maximum linear angular value: between 2 and 4 degrees, between 4 and 6 degrees, between 6 and 8 degrees, or another angular value range. ### Claim 6 (depends on 1) The method of claim 1, wherein the one or more source images and the second source image are a part of two-dimensional (2D) video content designated to be presented on a rectangular image rendering surface with a relatively limited field of view. ### Claim 7 (depends on 1) The method of claim 1, wherein the one or more display images and the second display image are a part of peripheral-vision expanded video content designated to be presented on a target display of headmounted device of the viewer with a relatively large field of view. ### Claim 8 (depends on 1) The method of claim 1, wherein the one or more display images and the second display image are a part of peripheral-vision expanded video content designated to be presented on a target display of headmounted device of the viewer with a relatively large field of view. ### Claim 9 (depends on 1) The method of claim 1, wherein a sound source as visually depicted in the one or more display images and the second display image is moved from an original spatial position in the one or more source images and the second source image to a new spatial position in the one or more display images and the second display image; the method further comprising: causing an audio rendering system to render sounds designated to be emitted from the sound source from the new spatial position. ### Claim 10 (depends on 1) The method of claim 1, wherein the one or more display images and the second display image is rendered with a curved screen; and wherein the curved screen is a virtual display screen. ### Claim 11 (depends on 1) The method of claim 1, wherein the one or more display images and the second display image comprise three-dimensional (3D) display image portions. ### Claim 12 (depends on 11) The method of claim 11, wherein the one or more source images and the second source image represent two-dimensional (2D) images; and wherein the 3D display image portions are generated using depth data portions generated from applying 2D to 3D conversion operations on corresponding 2D source image portions in the one or more source images and the second source image. ### Claim 13 (depends on 1) The method of claim 1, wherein one or more depicted objects in the peripheral-vision image portion of the second peripheral-vision expanded image are seamlessly stretched as compared with one or more corresponding depicted objects in the second corresponding image portion of the second source image. ### Claim 14 (depends on 1) The method of claim 1, wherein the peripheral-vision image portion of the second peripheral-vision expanded image is of a relatively low resolution as compared with the focal-vision image portion of the second peripheral-vision expanded image. ### Claim 15 (depends on 1) An apparatus performing the method of claim 1. ### Claim 16 (depends on 1) A system performing the method of claim 1. ### Claim 17 (depends on 1) A non-transitory computer readable storage medium, storing software instructions, which when executed by one or more processors cause performance of the method recited in claim 1. ### Claim 18 (depends on 1) A computing device comprising one or more processors and one or more non-transitory computer-readable storage media, storing a set of instructions, which when executed by one or more processors cause performance of the method recited in method claim 1. ### Claim 19 A method for providing immersive media content consumption experiences, comprising: rendering, to a viewer, one or more display images at one or more time points, the one or more display images being generated from one or more peripheral-vision expanded images received from a video streaming server, the one or more peripheral-vision expanded images being generated by the video streaming server from one or more source images in reference to one or more view directions of the viewer at the one or more time points; collecting, in real time, view direction data of the viewer collected while the viewer is viewing the one or more display images, the view direction data being used to determine a second view direction of the viewer at a second time point subsequent to the one or more time points; sending at least a part of the view direction data to the video streaming server to cause the video streaming server to generate a second peripheral-vision expanded image from a second source image in reference to the second view direction of the viewer at the second time point, the second peripheral-vision expanded image to be used to generate a second display image for rendering to the viewer at the second time point, the second peripheral-vision expanded image having a focal-vision image portion covering the second view direction of the viewer and a peripheral-vision image portion outside the focal-vision image portion, the focal-vision image portion of the second peripheral-vision expanded image being generated from a first corresponding source image portion in the second source image to cover the viewer's focal vision as covered by the first corresponding source image portion, the peripheral-vision image portion of the second peripheral-vision expanded image being generated by spatially stretching a second corresponding source image portion in the second source image to cover the viewer's peripheral vision including one or more peripheral vision portions not covered by the second corresponding source image portion of the second source image; receiving the second peripheral-vision expanded image from the video streaming server; generating the second display image from the second peripheral-vision expanded image; rendering at the second time point, to the viewer, the second display image. ### Claim 20 (depends on 19) The method of claim 19, wherein the one or more display images and the second display image are rendered on a target display on a headmounted device of the viewer.