# US 2010/0306413 A1 · Methods for detecting and handling video and video-like content in remote display system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0306413 A1
- Document ID: US20100306413A1
- Status: Published application
- Kind: A1
- Application number: 12/472,312
- Filed: 2009-05-26
- Published: 2010-12-02
- Earliest claimed: 2009-05-26
- Assignees: Red Hat Israel, Ltd.
- Inventors: Yaniv Kamay
- Web page: https://patentgrep.com/patent/US20100306413A1

## CPC

- H04N 19/17: the unit being an image region, e.g. an object
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Method and apparatus for improving streaming data of graphics remoting applications are described herein. According to one embodiment, a stream of data objects is generated, each data object representing graphics data associated with an image to be rendered at a client over a network. For each data object, it is detected whether the associated image includes a first region and a second region having content with different types of contents based on a changing rate of display output regions. The first region is compressed using a first compression method and the second region is compressed using a second compression method different than the first compression method. The graphics data associated with the compressed first region and second region is transmitted to the client over the network to be rendered at the client. Other methods and apparatuses are also described.

## Claims

### Claim 1

A computer-implemented method, comprising:
for each data object of a stream representing graphics data associated with an image to be rendered at a client over a network, detecting, by a host, whether the image includes a first region and a second region having content with different types of contents based on a changing rate of display output regions;
compressing, by the host, the first region using a first compression method and compressing the second region using a second compression method different than the first compression method; and
transmitting, by the host, graphics data associated with the compressed first region and second region to the client over the network to be rendered at the client.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first compression method comprises a lossy data compression method and the second compression method comprises a lossless data compression method.

### Claim 3 (depends on 2)

The method of claim 2, wherein the first region comprises actively changed content across a sequence of images, and wherein the second region comprises relatively static content across the sequence of images.

### Claim 4 (depends on 1)

The method of claim 1, wherein detecting whether the image includes a first region and a second region comprises:
identifying a graphics command associated with an area in an image;
adding a new data item corresponding to the graphics command to a representation of a display; and
determining whether an addition of the new data item indicates that the corresponding area has content that changes at relatively high frequency.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining whether an addition of the new data item indicates that the corresponding area has content that changes at relatively high frequency comprises:
if the addition of the new data item resulted in removal of an old data item, determining a time interval between an addition of an old data item to the tree and the addition of the new data item to the tree; and
if the time interval is below a threshold, determining that content of the current area has changed, and determining whether a change of the content satisfies a frequency criterion.

### Claim 6 (depends on 5)

The method of claim 5, wherein the new data item is of the same size as the old data item.

### Claim 7 (depends on 5)

The method of claim 5, wherein the new data item has the same location in the tree as the old data item.

### Claim 8 (depends on 3)

The method of claim 3, wherein the first region includes streaming video content and the second region includes textual content.

### Claim 9 (depends on 2)

The method of claim 2, wherein data objects of the stream are generated from a desktop application executed within a virtual machine (VM) and accessed remotely by the client over the network.

### Claim 10 (depends on 1)

The method of claim 1, further comprising adding a time stamp to each region for use at the client, the client holding a plurality of video frames and scheduling display of the plurality of video frames using corresponding time stamps.

### Claim 11 (depends on 1)

The method of claim 1, further comprising adding a time stamp to each region for use at the client, the client synchronizing a video stream with an audio stream using corresponding time stamps.

### Claim 12

A computer readable storage medium including instructions that, when executed by a processing system, cause the processing system to perform a method comprising:
for each data object of a stream representing graphics data associated with an image to be rendered at a client over a network, detecting whether the image includes a first region and a second region having content with different types of contents based on a changing rate of display output regions;
compressing first region using a first compression method and compressing the second region using a second compression method different than the first compression method; and
transmitting graphics data associated with the compressed first region and second region to the client over the network to be rendered as at the client.

### Claim 13 (depends on 12)

The computer readable storage medium of claim 12, wherein:
the first compression method comprises a lossy data compression method and the second compression method comprises a lossless data compression method; and
the first region comprises actively changed content across a sequence of images, and the second region comprises relatively static content across the sequence of images.

### Claim 14 (depends on 12)

The computer readable storage medium of claim 12, wherein detecting whether the image includes a first region and a second region comprises:
identifying a graphics command associated with an area in an image; adding a new data item corresponding to the graphics command to a representation of a display; and
determining whether an addition of the new data item indicates that the corresponding area has content that changes at relatively high frequency.

### Claim 15 (depends on 12)

The computer readable storage medium of claim 12, further comprising adding a time stamp to each region for use at the client, the client holding a plurality of video frames and scheduling display of the plurality of video frames using corresponding time stamps.

### Claim 16 (depends on 1)

The computer readable storage medium claim 1, further comprising adding a time stamp to each region for use at the client, the client synchronizing a video stream with an audio stream using corresponding time stamps.

### Claim 17

A data processing system, comprising:
a processor;
a memory;
a virtual machine (VM) executed from the memory;
a guest operating system (OS) executed within the VM, the guest OS including
an application running therein to generate a stream of data objects, each data object representing graphics data associated with an image to be rendered at a client over a network, and
a remote display system, coupled to the VM, to detect whether the image includes a first region and a second region having content with different types of contents based on a changing rate of display output regions,
compress first region using a first compression method and compress the second region using a second compression method different than the first compression method, and
transmit graphics data associated with the compressed first region and second region to the client over the network to be rendered at the client.

### Claim 18 (depends on 17)

The system of claim 17, wherein:
the first compression method comprises a lossy data compression method and the second compression method comprises a lossless data compression method; and
the first region comprises actively changed content across a sequence of images, and the second region comprises relatively static content across the sequence of images.

### Claim 19 (depends on 17)

The system of claim 17, wherein the remote display system detects whether the image includes a first region and a second region by:
identifying a graphics command associated with an area in an image;
adding a new data item corresponding to the graphics command to a representation of a display; and
determining whether an addition of the new data item indicates that the corresponding area has content that changes at relatively high frequency.

### Claim 20 (depends on 19)

The system of claim 19, wherein the first region includes streaming video content and the second region includes textual content.

### Claim 21 (depends on 17)

The system of claim 17, wherein the remote display system is further to add a time stamp to each region for use at the client.

### Claim 22 (depends on 21)

The system of claim 21, wherein the client holds a plurality of video frames and schedules display of the plurality of video frames using corresponding time stamps.

### Claim 23 (depends on 21)

The system of claim 21, wherein the client synchronizes an audio stream with a clock using corresponding time stamps.
