# US 2004/0001638 A1 · Rate allocation for mixed content video

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2004/0001638 A1
- Document ID: US20040001638A1
- Status: Published application
- Kind: A1
- Application number: 10/186,481
- Filed: 2002-06-28
- Published: 2004-01-01
- Earliest claimed: 2002-06-28
- Assignees: Microsoft Corporation
- Inventors: Sanjeev Mehrotra
- Web page: https://patentgrep.com/patent/US20040001638A1

## CPC

- H04N 19/17: the unit being an image region, e.g. an object
- H04N 19/124: Quantisation
- H04N 19/152: by measuring the fullness of the transmission buffer
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/61: in combination with predictive coding
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present invention relates to regulating the quality and/or bitrate of content within mixed content video when the video is compressed subject to a bitrate constraint. For example, a screen capture encoder encodes palletized content within a frame of screen capture video. Subject to an overall bitrate constraint, the encoder then allocates bits for continuous tone content within the frame. By controlling the allocation of bits used to encode the continuous tone content, the encoder regulates bitrate for the continuous tone content. This in turn can allow the encoder to regulate spatial quality and/or overall temporal quality for the video. In one scenario, for screen capture video encoded to a (relatively) constant overall bitrate, the screen capture encoder reduces the bitrate (and quality) of the continuous tone content, instead spending bits to increase the overall frame rate of the video.

## Claims

### Claim 1

In a screen capture encoder, a method for encoding captured screen content, the method comprising:
encoding one or more components of the captured screen content using lossless compression;
allocating bits for continuous tone pixels of the captured screen content; and
encoding the continuous tone pixels, wherein the allocating regulates quality and bitrate for the continuous tone pixels.

### Claim 2 (depends on 1)

The method of claim 1 wherein the one or more components of the captured screen content encoded using lossless compression include palettized pixels.

### Claim 3 (depends on 2)

The method of claim 2 wherein the one or more components of the captured screen content encoded using lossless compression further include a segmentation map.

### Claim 4 (depends on 2)

The method of claim 2 wherein the one or more components of the captured screen content encoded using lossless compression further include motion information and frame header information.

### Claim 5 (depends on 1)

The method of claim 1 further comprising:
repeating the encoding the one or more components, the allocating, and the encoding the continuous tone pixels on a frame-by-frame basis for a sequence of plural frames.

### Claim 6 (depends on 1)

The method of claim 1 wherein the allocating comprises:
computing a preliminary bit allocation;
comparing the preliminary bit allocation with a number of available bits in view of the encoding using lossless compression; and
allocating the bits based at least in part on results of the comparing.

### Claim 7 (depends on 1)

The method of claim 1 wherein the allocating comprises:
computing a preliminary bit allocation;
computing a minimum quality bit allocation;
comparing the preliminary bit allocation and the minimum quality bit allocation; and
allocating the bits based at least in part on results of the comparing.

### Claim 8 (depends on 1)

The method of claim 1 wherein the allocating comprises:
estimating a number of bits needed to encode the captured screen content;
estimating a number of bits needed to encode the continuous tone pixels; and
computing the minimum of:
the difference between the estimated number of bits needed to encode the captured screen content and a number of bits spent in the encoding using lossless compression;
a number of available bits in view of the encoding using lossless compression; and
the estimated number of bits needed to encode the continuous tone pixels.

### Claim 9 (depends on 8)

The method of claim 8 wherein the number of available bits is further reduced according to a buffer filling factor.

### Claim 10 (depends on 1)

The method of claim 1 wherein the encoding the continuous tone pixels comprises using a combination of lossy compression and lossless compression.

### Claim 11 (depends on 1)

A computer-readable medium storing computer-executable instructions for causing a computer programmed thereby to perform a method of claim 1.

### Claim 12

A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform method of encoding a mixture of palettized screen capture content and continuous tone screen capture content, the method comprising:
encoding the palettized screen capture content;
allocating bits for encoding the continuous tone screen capture content; and
encoding the continuous tone screen capture content, wherein the allocating controls bitrate for the continuous tone screen capture content.

### Claim 13 (depends on 12)

The computer-readable medium of claim 12 wherein the encoding the palettized screen capture content uses lossless compression and wherein the encoding the continuous tone screen capture content uses lossy and lossless compression.

### Claim 14 (depends on 12)

The computer-readable medium of claim 12 wherein the method further comprises before the allocating, encoding a frame header and segmentation map for a frame that includes the palettized screen capture content and the continuous tone screen capture content.

### Claim 15 (depends on 12)

The computer-readable medium of claim 12 wherein the allocating depends at least in part upon preliminary and minimum quality bit allocations for the continuous tone screen capture content and a number of available bits.

### Claim 16

A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform a method of encoding a frame comprising palettized content and continuous tone content, the method comprising:
encoding the palettized content in the frame;
checking for a number of bits available after the encoding the palettized content;
based at least in part on the checking, allocating a second number of bits, wherein the second number of bits is less than or equal to the number of bits available after the encoding the palettized content; and
encoding the continuous tone content in the frame with the second number of bits.

### Claim 17 (depends on 16)

The computer-readable medium of claim 16 wherein the method further comprises:
repeating the encoding the palettized content, the checking, the allocating, and the encoding the continuous tone content on a frame-by-frame basis for a sequence of plural frames.

### Claim 18 (depends on 16)

The computer-readable medium of claim 16 wherein the allocating comprises:
computing a preliminary bit allocation;
comparing the preliminary bit allocation with the number of bits available after the encoding the palettized content; and
allocating the bits based at least in part on results of the comparing.

### Claim 19 (depends on 16)

The computer-readable medium of claim 16 wherein the allocating comprises:
computing a preliminary bit allocation;
computing a minimum quality bit allocation;
comparing the preliminary bit allocation and the minimum quality bit allocation; and
allocating the bits based at least in part on results of the comparing.

### Claim 20 (depends on 16)

The computer-readable medium of claim 16 wherein the encoding the palettized content uses lossless compression, and wherein the encoding the continuous tone content uses a combination of lossy compression and lossless compression.

### Claim 21

A screen capture encoder comprising:
means for encoding one or more components of captured screen content using lossless compression;
means for encoding continuous tone pixels of the captured screen content; and
means for controlling bitrate of the captured screen content by allocating resources for encoding the continuous tone pixels based at least in part on resources used for the lossless compression.

### Claim 22 (depends on 21)

The screen capture encoder of claim 21 further comprising means for performing motion estimation on the captured screen content.

### Claim 23 (depends on 21)

The screen capture encoder of claim 21 wherein the one or more components encoded using lossless compression include palettized pixels.

### Claim 24 (depends on 21)

The screen capture encoder of claim 21 wherein the one or more components encoded using lossless compression include a segmentation map.

### Claim 25 (depends on 24)

The screen capture encoder of claim 24 wherein the segmentation map indicates locations of palettized pixels and continuous tone pixels among intra pixels of a P frame.

### Claim 26 (depends on 21)

The screen capture encoder of claim 21 wherein the one or more components encoded using lossless compression include motion information and frame header information.

### Claim 27

An encoder comprising:
a palettized content coder for losslessly encoding palettized content;
a continuous tone content coder for encoding continuous tone content; and
a rate controller to regulate quality and bitrate for the continuous tone content following encoding of the palettized content.

### Claim 28 (depends on 27)

The encoder of claim 27 further comprising a motion estimator.

### Claim 29 (depends on 27)

The encoder of claim 27 further comprising a segmenter for segmenting the continuous tone content and palettized content.
