# US 2022/0284665 A1 · Error concealment in split rendering using shading atlases

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284665 A1
- Document ID: US20220284665A1
- Status: Published application
- Kind: A1
- Application number: 17/190,758
- Filed: 2021-03-03
- Published: 2022-09-08
- Earliest claimed: 2021-03-03
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Joseph Daniel GARVEY; Bojan VRCELJ
- Web page: https://patentgrep.com/patent/US20220284665A1

## CPC

- A63F 13/5255: according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
- G06T 15/80: Shading
- A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
- G02B 27/0172: {characterised by optical features}
- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06T 11/40: Filling planar surfaces by adding surface attributes, e.g. adding colours or textures
- G06T 15/005: {General purpose rendering architectures}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

A client device of a split rendering graphics processing architecture, receives an encoded shading atlas for a current frame. The encoded shading atlas is characterized by slices. The client decodes the received encoded shading atlas and identifies unsuccessfully decoded slices. The client then determines whether each patch of the decoded shading atlas intersects an identified unsuccessfully decoded slice. The client determines, for each intersecting patch, whether a prior valid version of each intersecting patch is stored at the client device. The client error conceals each intersecting patch having a prior valid version stored at the client device with the corresponding prior valid version to create an error concealed atlas. The client renders the current frame using the error concealed atlas.

## Claims

### Claim 1

A method of graphics processing, comprising:
in a client device of a split rendering graphics processing architecture:
receiving an encoded shading atlas for a current frame, the encoded shading atlas characterized by slices;
decoding the received encoded shading atlas and identifying unsuccessfully decoded slices;
determining whether each patch of the decoded shading atlas intersects an identified unsuccessfully decoded slice;
determining, for each intersecting patch, whether a prior valid version of each intersecting patch is stored at the client device;
error concealing each intersecting patch having a prior valid version stored at the client device with the corresponding prior valid version to create an error concealed atlas; and
rendering, the current frame using the error concealed atlas.

### Claim 2 (depends on 1)

The method of claim 1:
further comprising accumulating, by the client device, a list comprising a patch identifier for each patch of the shading atlas and a location in the shading atlas for the patch corresponding to each patch identifier,
wherein determining whether each patch of the decoded shading atlas intersects an unsuccessfully decoded slice is based on the accumulated list.

### Claim 3 (depends on 1)

The method of claim 1:
further comprising, indicating each region of the error concealed atlas corresponding to an intersecting patch determined not to correspond to a prior valid version,
wherein rendering the current frame further comprises rendering using in-painting for each indicated region.

### Claim 4 (depends on 1)

The method of claim 1, wherein the prior valid version of the intersecting patch is the version of the intersecting patch used to render an immediately preceding frame.

### Claim 5 (depends on 1)

The method of claim 1, wherein the client device is a client device for an extended reality (XR) application, an augmented reality (AR) application, a virtual reality (VR) application, or a cloud gaming application.

### Claim 6 (depends on 5)

The method of claim 5, wherein the client device is a headset, a multimedia console, or a head mounted display (HMD).

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
generating a replacement slice for at least one identified unsuccessfully decoded slice based on each error concealed patch corresponding to a patch determined to intersect the at least one identified unsuccessfully decoded slice; and
decoding an encoded shading atlas for a subsequent frame based on the replacement slice.

### Claim 8

An apparatus for graphics processing, comprising:
memory storing instructions executable by at least one processor; and
at least one processor coupled to the memory and configured, as a client device of a split rendering graphics processing architecture, to execute the instructions to:
receive an encoded shading atlas for a current frame, the encoded shading atlas characterized by slices;
decode the received encoded shading atlas and identify unsuccessfully decoded slices;
determine whether each patch of the decoded shading atlas intersects an identified unsuccessfully decoded slice;
determine, for each intersecting patch, whether a prior valid version of each intersecting patch is stored at the client device;
error conceal each intersecting patch having a prior valid version stored at the client device with the corresponding prior valid version to create an error concealed atlas; and
render, the current frame using the error concealed atlas.

### Claim 9 (depends on 8)

The apparatus of claim 8:
wherein the at least one processor is further configured to execute the instructions to accumulate a list comprising a patch identifier for each patch of the shading atlas and a location in the shading atlas for the patch corresponding to each patch identifier,
wherein determining whether each patch of the decoded shading atlas intersects an unsuccessfully decoded slice is based on the accumulated list.

### Claim 10 (depends on 8)

The apparatus of claim 8:
wherein the at least one processor is further configured to execute the instructions to indicate each region of the error concealed atlas corresponding to an intersecting patch determined not to correspond to a prior valid version,
wherein rendering the current frame further comprises rendering using in-painting for each indicated region.

### Claim 11 (depends on 8)

The apparatus of claim 8, wherein the prior valid version of the intersecting patch is the version of the intersecting patch used to render an immediately preceding frame.

### Claim 12 (depends on 8)

The apparatus of claim 8, wherein the client device is a client device for an extended reality (XR) application, an augmented reality (AR) application, a virtual reality (VR) application, or a cloud gaming application.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the client device is a headset, a multimedia console, or a head mounted display (HMD).

### Claim 14 (depends on 8)

The apparatus of claim 8, wherein the at least one processor is further configured to execute the instructions to:
generate a replacement slice for at least one identified unsuccessfully decoded slice based on each error concealed patch corresponding to a patch determined to intersect the at least one identified unsuccessfully decoded slice; and
decode an encoded shading atlas for a subsequent frame based on the replacement slice.

### Claim 15

An apparatus for graphics processing in a split rendering architecture in a client device, comprising:
means for receiving an encoded shading atlas for a current frame, the encoded shading atlas characterized by slices;
means for decoding the received encoded shading atlas and identifying unsuccessfully decoded slices;
means for determining whether each patch of the decoded shading atlas intersects an identified unsuccessfully decoded slice;
means for determining, for each intersecting patch, whether a prior valid version of each intersecting patch is stored at the client device;
means for error concealing each intersecting patch having a prior valid version stored at the client device with the corresponding prior valid version to create an error concealed atlas; and
means for rendering, the current frame using the error concealed atlas.

### Claim 16 (depends on 15)

The apparatus of claim 15:
further comprising means for accumulating, by the client device, a list comprising a patch identifier for each patch of the shading atlas and a location in the shading atlas for the patch corresponding to each patch identifier,
wherein means for determining whether each patch of the decoded shading atlas intersects an unsuccessfully decoded slice is based on the accumulated list.

### Claim 17 (depends on 15)

The apparatus of claim 15:
further comprising, means for indicating each region of the error concealed atlas corresponding to an intersecting patch determined not to correspond to a prior valid version,
wherein means for rendering the current frame further comprises means for rendering using in-painting for each indicated region.

### Claim 18 (depends on 15)

The apparatus of claim 15, wherein the prior valid version of the intersecting patch is the version of the intersecting patch used to render an immediately preceding frame.

### Claim 19 (depends on 15)

The apparatus of claim 15, wherein the client device is a client device for an extended reality (XR) application, an augmented reality (AR) application, a virtual reality (VR) application, or a cloud gaming application.

### Claim 20 (depends on 19)

The apparatus of claim 19, wherein the client device is a headset, a multimedia console, or a head mounted display (HMD).

### Claim 21 (depends on 15)

The apparatus of claim 15, further comprising:
means for generating a replacement slice for at least one identified unsuccessfully decoded slice based on each error concealed patch corresponding to a patch determined to intersect the at least one identified unsuccessfully decoded slice; and
means for decoding an encoded shading atlas for a subsequent frame based on the replacement slice.

### Claim 22

A computer-readable medium storing computer executable code for graphics processing by a client device in a split rendering architecture, the code when executed by a processor causes the processor to:
receive an encoded shading atlas for a current frame, the encoded shading atlas characterized by slices;
decode the received encoded shading atlas and identify unsuccessfully decoded slices;
determine whether each patch of the decoded shading atlas intersects an identified unsuccessfully decoded slice;
determine, for each intersecting patch, whether a prior valid version of each intersecting patch is stored at the client device;
error conceal each intersecting patch having a prior valid version stored at the client device with the corresponding prior valid version to create an error concealed atlas; and
render, the current frame using the error concealed atlas.

### Claim 23 (depends on 22)

The computer-readable medium of claim 22:
wherein the code when executed by a processor further causes the processor to accumulate a list comprising a patch identifier for each patch of the shading atlas and a location in the shading atlas for the patch corresponding to each patch identifier,
wherein determining whether each patch of the decoded shading atlas intersects an unsuccessfully decoded slice is based on the accumulated list.

### Claim 24 (depends on 22)

The computer-readable medium of claim 22:
wherein the code when executed by a processor further causes the processor to indicate each region of the error concealed atlas corresponding to an intersecting patch determined not to correspond to a prior valid version;
wherein rendering the current frame further comprises rendering using in-painting for each indicated region.

### Claim 25 (depends on 22)

The computer-readable medium of claim 22, wherein the prior valid version of the intersecting patch is the version of the intersecting patch used to render an immediately preceding frame.

### Claim 26 (depends on 22)

The computer-readable medium of claim 22, wherein the client device is a client device for an extended reality (XR) application, an augmented reality (AR) application, a virtual reality (VR) application, or a cloud gaming application.

### Claim 27 (depends on 26)

The computer-readable medium of claim 26, wherein the client device is a headset, a multimedia console, or a head mounted display (HMD).

### Claim 28 (depends on 22)

The computer-readable medium of claim 22, wherein the code when executed by a processor further causes the processor to:
generate a replacement slice for at least one identified unsuccessfully decoded slice based on each error concealed patch corresponding to a patch determined to intersect the at least one identified unsuccessfully decoded slice; and
decode an encoded shading atlas for a subsequent frame based on the replacement slice.