# US 2025/0159265 A1 · Cloud Application Data Streaming Using Drawing Data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0159265 A1
- Document ID: US20250159265A1
- Status: Published application
- Kind: A1
- Application number: 18/722,274
- Filed: 2022-09-22
- Published: 2025-05-15
- Earliest claimed: 2021-12-20
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Andreas Huber; Iliyan Malchev
- Web page: https://patentgrep.com/patent/US20250159265A1

## CPC

- H04N 21/23412: {for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects}
- H04N 21/41407: {embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57)}
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/23: for interfacing with the game device, e.g. specific interfaces between game controller and console
- A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
- A63F 13/52: involving aspects of the displayed game scene
- A63F 13/53: involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
- H04N 21/2343: involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)}
- H04N 21/234372: {for performing aspect ratio conversion}
- H04N 21/236: Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream {(multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00)}
- H04N 21/4223: Cameras (H04N23/00 takes precedence)
- H04N 21/44012: {involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs}
- A63F 13/87: Communicating with other players during game play, e.g. by e-mail or chat
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR

## Abstract

A cloud-based application streaming system [100] is implemented to stream an application to a client device [116]. The system includes a server [102] running the application on an application instance [104] configured to render video frames and determine drawing data [106] based on the application. The server then encodes the video frames and provides the video frames and drawing data to the client device as an application data stream [114]. In response to receiving the application data stream, the client device is configured to display at least a portion of the application data stream based on the drawing data. Additionally, the client device is configured to receive inputs and provide an input data stream [128] representing these inputs to the server. In response to the input data stream, the server modifies the application based on the input data stream and provides a modified application data stream to the client device.

## Claims

### Claim 1

A method comprising:
initiating application streaming between a client device and a cloud-based application server;
in response to receiving an application data stream from the cloud-based application server, displaying, at the client device, at least a portion of the application data stream based on drawing data, wherein the application data stream includes the drawing data; and
transmitting an input data stream to the cloud-based application server, the input data stream comprising data representing at least one input received at the client device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving a modified application data stream; and
displaying at least a portion of the modified application data stream based on modified drawing data.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
receiving one or more inputs; and
encoding the one or more inputs, wherein the input data stream includes the encoded one or more inputs.

### Claim 4 (depends on 3)

The method of claim 3, wherein the one or more inputs include video captured by a camera of the client device.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
outputting at least a portion of the application data stream using one or more output devices.

### Claim 6 (depends on 5)

The method of claim 5, wherein the one or more output devices comprises a speaker of the client device.

### Claim 7 (depends on 1)

The method of claim 1, wherein the drawing data includes a scene graph for at least a portion of the application data stream.

### Claim 8 (depends on 1)

The method of claim 1, the method further comprising:
decoding at least a portion of the application data stream and displaying the decoded at least a portion of the application data stream according to the drawing data.

### Claim 9 (depends on 1)

The method of claim 1, wherein the drawing data comprises a value representing a difference between attributes of a graphical object and a reference graphical object.

### Claim 10

A method comprising:
initiating application streaming between a client device and a cloud-based application server;
transmitting, to the client device, an application data stream including an encoded video frame associated with an application running on the cloud-based application server and drawing data associated with the application; and
in response to receiving an input data stream from the client device representing an input received at the client device, modifying the application based on the input.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
transmitting, to the client device, a modified application data stream including an encoded modified video frame associated with the modified application and modified drawing data associated with the modified application.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
decoding at least a portion of the input data stream and providing the decoded at least a portion of the input data stream to the application as an input.

### Claim 13 (depends on 12)

The method of claim 12, wherein the at least a portion of the input data stream includes encoded video captured from a camera of the client device.

### Claim 14 (depends on 10)

The method of claim 10, further comprising:
determining the drawing data based on the application; and
storing the drawing data in a drawing data cache.

### Claim 15 (depends on 10)

The method of claim 10, wherein the drawing data comprises a scene graph for at least a portion of the application data stream.

### Claim 16 (depends on 10)

The method of claim 10, wherein the drawing data comprises a value representing a difference between attributes of a graphical object and a reference graphical object.

### Claim 17 (depends on 10)

The method claim 10, wherein the application comprises at least one of a video calling application, presentation application, meeting application, or streaming application.

### Claim 18 (depends on 1)

The method of claim 1, wherein the client device comprises a smartphone.

### Claim 19 (depends on 1)

The method of claim 1, wherein the client device comprises a browser.

### Claim 20 (depends on 1)

The method of claim 1, wherein at least a portion of the application data stream is displayable at a plurality of resolutions on the client device.

### Claim at position 21

21-22. (canceled)
