Back to results
US 12,702,921 B2Granted
Decoding regions of interest in computer game video first while concealing missing parts using multiple decoders
AssigneeSony Interactive Entertainment Inc.
InventorsManoj Srivastava, Eric Hsuming Chen, Mario Sarria, Hideyuki Mizusawa
Life of this patent
Estimating the term…
Abstract
Techniques are described for reducing latency in computer game network streaming by using plural encoders and decoders, with one encoder-decoder pair being used for regions of interest (ROI) in the video and being given priority in transmitting and rendering over background video that is processed by another encoder/decoder pair.
Claims20
An apparatus comprising:
- at least one processor assembly configured to:
- encode a first portion of a video using a first encoder of the at least one processor assembly, the first portion of the video including a region of interest (ROI) portion of the video;
- concurrently encode a second portion of the video using a second encoder of the at least one processor assembly, the second portion of the video including a non-ROI portion of the video;
- and
- schedule transmission such that the encoded first portion is sent over a network to at least one receiver before sending the encoded second portion, the encoded first portion being prioritized in transmission over the network based on a latency target thereby facilitating:
- responsive to the second portion being unavailable within the latency target and prior to presentation, update of a frame buffer to (i) obfuscate the second portion, (ii) reuse prior frame content, or (iii) reconstruct missing second portion content using a machine learning model;
- and
- responsive to the second portion becoming available, blending of the non-ROI portion with the ROI portion.
5 more elements
An apparatus comprising:
- at least one processor assembly including a frame buffer, the at least one processor assembly configured to:
- decode a first portion of a video using a first decoder of the at least one processor assembly into the frame buffer, the first portion of the video including a region of interest (ROI) portion of the video;
- concurrently decode a second portion of the video using a second decoder of the at least one processor assembly into the frame buffer, the second portion of the video including a non-ROI portion of the video;
- present the decoded first portion on at least one video display while the second portion is being decoded;
- responsive to the second portion being unavailable within a latency period and prior to presentation, update the frame buffer to (i) obfuscate the second portion, (ii) reuse prior frame content, or (iii) reconstruct missing second portion content using a machine learning model;
- and
- responsive to the second portion becoming available, blending the non-ROI portion with the ROI portion.
4 more elements
A method, comprising:
- transmitting a region of interest (ROI) portion of a video frame to a receiver before transmitting a portion of the frame outside the ROI portion;
- decoding the ROI portion of the video with a decoder of the receiver;
- presenting the ROI portion on a video display regardless of whether portions the portion of the frame outside the ROI portion is decoded for presentation;
- responsive to the portion of the frame outside the ROI portion being unavailable within a latency period and prior to presentation, updating a frame buffer to (i) obfuscate the portion of the frame outside the ROI portion, (ii) reuse prior frame content, or (iii) reconstruct missing portion of the frame outside the ROI portion using a machine learning model;
- and
- responsive to the portion of the frame outside the ROI portion becoming available, blending a non-ROI portion with the ROI portion.
3 more elements
Dependencies are inferred from the claim text.