# US 12,120,364 B2 · Low latency wireless virtual reality systems and methods > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,120,364 B2 - Document ID: US12120364B2 - Status: Granted - Kind: B2 - Application number: 18/094,161 - Filed: 2023-01-06 - Published: 2023-05-18 - Granted: 2024-10-15 - Earliest claimed: 2016-04-15 - Assignees: Advanced Micro Devices, Inc.; ATI Technologies ULC - Applicants: Advanced Micro Devices, Inc.; ATI Technologies ULC - Inventors: Lei Zhang; Gabor Sines; Khaled Mammou; David Glen; Layla A. Mah; Rajabali M. Koduri; Bruce Montag - Web page: https://patentgrep.com/patent/US12120364B2 ## CPC - H04N 21/2343: involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)} - H04L 65/70: Media network packetisation - H04L 65/762: {at the source (reformatting of additional data in video distribution servers H04N21/2355)} - H04L 67/131: Protocols for games, networked simulations or virtual reality - H04L 69/24: Negotiation of communication capabilities - H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - H04N 21/23605: {Creation or processing of packetized elementary streams [PES]} - H04N 21/2368: Multiplexing of audio and video streams - H04N 21/41407: {embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57)} - H04N 21/422: Input-only peripherals {, i.e. input devices connected to specially adapted client devices}, e.g. global positioning system [GPS] {(input devices also receiving signals from specially adapted client devices H04N21/4104)} - H04N 21/42202: {environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes} - H04N 21/43072: {of multiple content streams on the same device} - H04N 21/4341: {Demultiplexing of audio and video streams} - H04N 21/4343: {Extraction or processing of packetized elementary streams [PES]} - H04N 21/437: Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server {(flow control in data networks H04L47/10; streaming protocols, e.g. RTP or RTCP, H04L65/65; scheduling or organising the servicing of application requests in data packet switching networks H04L67/60)} - H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)} - H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)} - H04N 21/816: {involving special video data, e.g 3D video} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A device and method for processing Virtual Reality (VR) data is disclosed. The method comprises transmitting feedback information from the device to a server, wherein the feedback information is captured in the device, receiving data from the server to be presented on the device based on the feedback information, wherein the data includes video data and audio data where the video data is a frame of video data in a sequence of frames and the audio data is the corresponding audio data of the frame, decoding the video data and corresponding audio data of the frame, and controlling the presentation of the video data and corresponding audio data on the device such that the video data is synchronized with the corresponding audio data. ## Claims ### Claim 1 A method for processing Virtual Reality (VR) data in a device, the method comprising: transmitting feedback information from the device to a server, wherein the feedback information is captured in the device; receiving data from the server to be presented on the device based on the feedback information, wherein the data includes a frame of video data in a sequence of frames and corresponding audio data; decoding the frame of video data and corresponding audio data; and controlling the presentation of the frame of video data and corresponding audio data on the device such that the frame of video data is synchronized with the corresponding audio data. ### Claim 2 (depends on 1) The method of claim 1, wherein the feedback information is used to predict a viewpoint of a next frame of a sequence of frames of video data. ### Claim 3 (depends on 1) The method of claim 1, wherein the feedback information comprises any one or both of user position information and user orientation information. ### Claim 4 (depends on 1) The method of claim 1, further comprising capturing the feedback information using one or more sensors. ### Claim 5 (depends on 4) The method of claim 4, wherein the feedback information captured using the one or more sensors includes head or eye tracking information. ### Claim 6 (depends on 1) The method of claim 1, wherein the feedback information includes time stamp information indicating a time when the feedback information is generated. ### Claim 7 (depends on 1) The method of claim 1, wherein the feedback information is transmitted to the server using a wireless transmission protocol. ### Claim 8 (depends on 1) The method of claim 1, wherein the data is received from the server using a wireless transmission protocol. ### Claim 9 (depends on 1) The method of claim 1, wherein the feedback information is continuously transmitted to the server. ### Claim 10 (depends on 1) The method of claim 1, wherein the controlling the presentation of the frame of video data and corresponding audio on the device includes: displaying the frame of video data on a display; and playing the corresponding audio data on a speaker. ### Claim 11 A device for processing Virtual Reality (VR) data, the device comprising: memory configured to store data; a processor in communication with the memory; and a transceiver; wherein the processor and transceiver are configured to: transmit feedback information from the device to a server, wherein the feedback information is captured in the device; receive data from the server to be presented on the device based on the feedback information, wherein the data includes a frame of video data in a sequence of frames and corresponding audio data; decode the frame of video data and corresponding audio data; and control the presentation of the frame of video data and corresponding audio data on the device such that the frame of video data is synchronized with the corresponding audio data. ### Claim 12 (depends on 11) The device of claim 11, wherein the feedback information is used to predict a viewpoint of a next frame of a sequence of frames of video data. ### Claim 13 (depends on 11) The device of claim 11, wherein the feedback information comprises any one or both of user position information and user orientation information. ### Claim 14 (depends on 11) The device of claim 11, further comprising: one or more sensors configured to capture the feedback information. ### Claim 15 (depends on 14) The device of claim 14, wherein the feedback information captured using the one or more sensors includes head or eye tracking information. ### Claim 16 (depends on 11) The device of claim 11, wherein the feedback information includes time stamp information indicating a time when the feedback information is generated. ### Claim 17 (depends on 11) The device of claim 11, wherein the feedback information is transmitted to the server using a wireless transmission protocol. ### Claim 18 (depends on 11) The device of claim 11, wherein the data is received from the server using a wireless transmission protocol. ### Claim 19 (depends on 11) The device of claim 11, wherein the feedback information is continuously transmitted to the server. ### Claim 20 (depends on 11) The device of claim 11, further comprising: a display; and a speaker, wherein the processor is further configured to display the frame of video data on the display and play the corresponding audio data on the speaker when controlling the presentation of the audio data and video data on the device. ### Claim 21 A device for processing Virtual Reality (VR) data, the device comprising: memory configured to store data; a processor in communication with the memory; and a transceiver; wherein the processor and transceiver are configured to transmit feedback information from the device to a server, wherein the feedback information and includes time stamp information indicating a time when the feedback information is generated at the device; receive data from the server, the data including a frame of video data in a sequence of frames and corresponding audio data, to be presented on the device based on the feedback information; decode the frame of video data and corresponding audio data; and control the presentation of the frame of video data and corresponding audio data on the device such that the frame of video data is synchronized with the corresponding audio data. ### Claim 22 A device for processing Virtual Reality (VR) data, the device comprising: memory configured to store data; a processor in communication with the memory; and a transceiver; wherein the processor and transceiver are configured to transmit feedback information from the device to a server, wherein the feedback information includes time stamp information indicating a time when the feedback information is sensed at the device; receive data from the server, the data including a frame of video data in a sequence of frames and corresponding audio data, to be presented on the device based on the feedback information; decode the frame of video data and corresponding audio data; and control the presentation of the frame of video data and corresponding audio data on the device such that the frame of video data is synchronized with the corresponding audio data.