# US 2019/0317599 A1 · Virtual reality/augmented reality apparatus and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0317599 A1 - Document ID: US20190317599A1 - Status: Published application - Kind: A1 - Application number: 16/453,189 - Filed: 2019-06-26 - Published: 2019-10-17 - Earliest claimed: 2016-09-16 - Assignees: Intel Corporation - Applicants: Intel Corporation - Inventors: Tomas G. AKENINE-MOLLER; Robert M. TOTH; Ingo WALD; Aditya S. YANAMANDRA; Brent E. INSKO; Michael APODACA; Prasoonkumar SURTI - Web page: https://patentgrep.com/patent/US20190317599A1 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/012: {Head tracking input arrangements} - G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units} - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06T 3/18: Image warping, e.g. rearranging pixels individually - G06T 15/005: {General purpose rendering architectures} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 2200/24: involving graphical user interfaces [GUIs] - G09G 2370/04: Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract A virtual reality apparatus and method are described. For example, one embodiment of an apparatus comprises: a compute cluster comprising global illumination circuitry and/or logic to perform global illumination operations on graphics data in response to execution of a virtual reality application and to responsively generate a stream of samples; a filtering/compression module to perform filtering and/or compression operations on the stream of samples to generate filtered/compressed samples; a network interface to communicatively couple the compute cluster to a network, the filtered/compressed samples to be streamed over the network; a render node to receive the filtered/compressed samples streamed over the network, the render node comprising: decompression circuitry/logic to decompress the filtered/compressed samples to generate decompressed samples; a sample buffer to store the decompressed samples; and sample insertion circuitry/logic to asynchronously insert samples into a light field rendered by a light field rendering circuit/logic. ## Claims ### Claim 1 An apparatus comprising: an image rendering circuit to render image frames to be displayed in a virtual reality head mounted display (HMD); a first buffer to store a current image frame being rendered by the image rendering circuit; a second buffer to store a prior image frame, previously rendered by the image rendering circuit; a time warp module to detect that a first portion of the current image frame cannot be displayed on the HMD in a sufficiently short amount of time, the time warp module to responsively perform a time warp operation on a corresponding portion of the prior image frame to generate a warped image portion and combine the warped image portion with a second portion of the current image frame which can be displayed on the HMD in a sufficiently short amount of time to generate a final image frame to be displayed on the HMD. ### Claim 2 (depends on 1) The apparatus as in claim 1, wherein performing the time warp operation comprises reading sensor data from the HMD to determine a current or expected orientation of the user's head and/or gaze and responsively transforming the corresponding portion of the prior image frame to accommodate for the current or expected orientation of the user's head and/or gaze. ### Claim 3 (depends on 2) The apparatus as in claim 2, further comprising one or more user/eye tracking sensors on the HMD to detect and provide the sensor data to the time warp module. ### Claim 4 (depends on 1) The apparatus as in claim 1, wherein the image frames comprise left image frames to be displayed in a left display of the HMD and right image frames to be displayed in a right display of the HMD. ### Claim 5 (depends on 1) The apparatus as in claim 1, wherein the image rendering circuit comprises a vertex shader, geometry shader, and texture mapper. ### Claim 6 (depends on 1) The apparatus as in claim 1, wherein the image rendering circuit comprises a ray generation module, a ray traversal module, an intersection module, and a shading module. ### Claim 7 (depends on 1) The apparatus as in claim 1, wherein the HMD comprises: one or more displays to display image frames rendered by a graphics processing engine; a linear actuator to moveably couple one or more lenses at a distance in front of the one or more displays, the linear actuator to alter the distance between the one or more displays and the one or more lenses responsive to a control circuit; an eye tracking device integrated on the HMD and configured to detect a current direction/focus of a user's eyes while the user is wearing the HMD; a virtual reality application to receive a current direction/focus of a user's eyes from the eye tracking device and to responsively transmit a signal to the control circuit of the linear actuator to set the distance between the one or more displays and the one or more lenses at a desired accommodation distance. ### Claim 8 (depends on 7) The apparatus as in claim 7, wherein if the eye tracking device tracks both of the user's eyes, the vergence distance is computed and used as the desired accommodation distance. ### Claim 9 (depends on 7) The apparatus as in claim 7, wherein if only one eye is tracked, a ray is traced in the gaze direction into a virtual scene to determine a surface at which the user is looking, the surface having a distance which is then used as the desired accommodation distance.