# US 12,003,722 B2 · Encoding apparatus, encoding device, encoding method, computer program, and corresponding computer system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,003,722 B2 - Document ID: US12003722B2 - Status: Granted - Kind: B2 - Application number: 17/657,636 - Filed: 2022-04-01 - Published: 2022-07-14 - Granted: 2024-06-04 - Earliest claimed: 2022-04-01 - Assignees: Intel Corporation - Applicants: Intel Corporation - Inventors: Andrey Belogolovy; Evgeny Stupachenko; Niranjan Mylarappa Gowda - Web page: https://patentgrep.com/patent/US12003722B2 ## CPC - H04N 19/12: Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264 - A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client - H04N 19/103: Selection of coding mode or of prediction mode - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/164: Feedback from the receiver or from the transmission channel - H04N 19/172: the region being a picture, frame or field - H04N 19/177: the unit being a group of pictures [GOP] - H04N 19/192: the adaptation method, adaptation tool or adaptation type being iterative or recursive - H04N 19/423: characterised by memory arrangements (H04N19/433 takes precedence) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR ## Abstract Examples relate to an encoding apparatus, encoding device, encoding method, computer program and to a corresponding computer system. An encoding apparatus comprises processing circuitry configured to encode at least two video streams to perform video compression encoding using one or more encoders, wherein the one or more encoders are each configured to deter-mine an encoding context based on the video stream processed by the respective encoder, the encoding context comprising at least one or more reference frames determined by the respective encoder. The processing circuitry is configured to store at least one of the encoding contexts determined by the one or more encoders in a shared memory portion of the memory circuitry that is accessible to the one or more encoders. The processing circuitry is configured to proceed, using the one or more encoders, with encoding the at least two video streams based on an encoding context stored in the shared memory portion. ## Claims ### Claim 1 An encoding apparatus comprising processing circuitry and memory circuitry, the processing circuitry being configured to: encode frames of at least two video streams to perform video compression encoding using one or more encoders, wherein the one or more encoders are each configured to determine an encoding context based on the video stream processed by the respective encoder, the encoding context comprising at least one or more reference frames determined by the respective encoder; store at least one of the encoding contexts determined by the one or more encoders in a shared memory portion of the memory circuitry that is accessible to the one or more encoders; select one of the encoding contexts based on which one of previous encoded frames of the at least two video streams is transmitted to a recipient of an encoded video stream; and proceed, using the one or more encoders, with encoding the at least two video streams based on the selected encoding context stored in the shared memory portion. ### Claim 2 (depends on 1) The encoding apparatus according to claim 1, wherein the one or more encoders are two or more encoders. ### Claim 3 (depends on 1) The encoding apparatus according to claim 1, wherein the encoding context comprises the reference frame and encoding statistics. ### Claim 4 (depends on 1) The encoding apparatus according to claim 1, wherein the processing circuitry is configured to store each of the determined encoding contexts in the shared memory portion, and to update a pointer based on the selected encoding context. ### Claim 5 (depends on 4) The encoding apparatus according to claim 4, wherein the pointer is used by each of the one or more encoders to access the selected encoding context. ### Claim 6 (depends on 4) The encoding apparatus according to claim 4, wherein the processing circuitry is configured to free memory used for encoding contexts that have not been selected. ### Claim 7 (depends on 1) The encoding apparatus according to claim 1, wherein the processing circuitry is configured to store exclusively the selected encoding context in the shared memory portion. ### Claim 8 (depends on 1) The encoding apparatus according to claim 1, wherein each of the one or more encoders is configured to use the selected encoding context to proceed with encoding. ### Claim 9 (depends on 1) The encoding apparatus according to claim 1, wherein a first video stream of the at least two video streams is based on a predicted user behavior and a second video stream of the at least two video streams is based on an observed user behavior, wherein the encoding context is selected based on a difference between the predicted user behavior and the observed user behavior. ### Claim 10 (depends on 9) The encoding apparatus according to claim 9, wherein the user behavior relates to a behavior of the user in a cloud video game. ### Claim 11 (depends on 9) The encoding apparatus according to claim 9, wherein the second video stream is based on the observed user behavior up to a first point in time and the first video stream is based on the predicted user behavior up to a second point in time, with the second point in time being later than the first point in time, and with the selection being based on a difference between the predicted user behavior up to the second point in time and the observed user behavior up to the second point in time. ### Claim 12 (depends on 9) The encoding apparatus according to claim 9, wherein the processing circuitry is configured to obtain information on the observed user behavior from a video consumer device. ### Claim 13 (depends on 12) The encoding apparatus according to claim 12, wherein the processing circuitry is configured to generate the at least two video streams based on the information on the observed user behavior. ### Claim 14 (depends on 13) The encoding apparatus according to claim 13, wherein the processing circuitry is configured to determine the predicted user behavior based on the information on the observed user behavior, and to generate the first video stream based on the predicted user behavior. ### Claim 15 (depends on 13) The encoding apparatus according to claim 13, wherein the processing circuitry is configured to generate the at least two video streams using a video game engine, the first video stream being generated based on the predicted user behavior and the second video stream being generated based on the observed user behavior. ### Claim 16 (depends on 9) The encoding apparatus according to claim 9, wherein the processing circuitry is configured to generate an encoded output video stream based on encoded frames of the one or more encoders, with the encoded frames being selected based on the difference between the predicted user behavior and the observed user behavior, and to provide the output video stream to a consumer video device. ### Claim 17 (depends on 1) The encoding apparatus according to claim 1, wherein the number of video streams equals the number of encoders. ### Claim 18 (depends on 1) The encoding apparatus according to claim 1, wherein the number of video streams is greater than the number of encoders, with the one or more encoders being configured to perform encoding context switching on a frame-for-frame basis using the encoding contexts stored in the shared memory portion. ### Claim 19 (depends on 1) A system comprising the encoding apparatus according to claim 1 and a video consumer device, wherein the processing circuitry of the encoding apparatus is configured to provide an encoded output video stream that is based on encoded frames of the one or more encoders to the video consumer device. ### Claim 20 An encoding method comprising: encoding frames of at least two video streams to perform video compression encoding using one or more encoders, wherein the one or more encoders each determine an encoding context based on the video stream processed by the respective encoder, the encoding context comprising at least one or more reference frames determined by the respective encoder; storing at least one of the encoding contexts determined by the one or more encoders in a shared memory portion of memory that is accessible to the one or more encoders; selecting one of the encoding contexts based on which one of previously encoded frames of the at least two video streams is transmitted to a recipient of an encoded video stream; and proceeding, using the one or more encoders, with encoding the at least two video streams based on the selected encoding context stored in the shared memory portion. ### Claim 21 (depends on 20) A non-transitory machine-readable storage medium including program code, when executed, to cause a machine to perform the encoding method of claim 20.