# US 2011/0299605 A1 · Method and apparatus for video resolution adaptation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2011/0299605 A1
- Document ID: US20110299605A1
- Status: Published application
- Kind: A1
- Application number: 12/895,754
- Filed: 2010-09-30
- Published: 2011-12-08
- Earliest claimed: 2010-06-04
- Assignees: Apple Inc.
- Inventors: Douglas Scott PRICE; Xiaosong ZHOU; Hsi-Jung WU; James Oliver NORMILE
- Web page: https://patentgrep.com/patent/US20110299605A1

## CPC

- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- H04N 19/40: using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system and method for gradually changing the resolution of a video signal to avoid a large spike in the video data transmitted between an encoder and a decoder. Upon detection of a change in the quality of source video, of the quality of the encoding process, or of the channel conditions, any of which may negatively impact the rate of frame transmission from encoder to decoder, or the quality of frames transmitted, a responsive change in the resolution of the video frame may be gradually implemented. To change the resolution by increasing the effective image size, each successive frame in a sequence of frames may contain additional pixel blocks in the expansion image area at the new resolution. In an embodiment, the decoder displays the video image at the original resolution until the resolution switch has been completed.

## Claims

### Claim 1

A video coding system, comprising:
a pre-processor to spatially scale frames of an input image signal to a programmable effective image size,
a coder to encode frames output from the pre-processor, and
a controller to provide parameters to the pre-processor defining the effective image size, wherein, when the effective image size is increased from a first size to a second size:
the pre-processor, over a plurality of frames, outputs composite frames formed from an effective image area having the input image signal scaled to fit the first size and an incrementally increasing expansion image area, each expansion image area having a portion of the input signal sized to fit the second size.

### Claim 2 (depends on 1)

The video coding system of claim 1, wherein the composite frames further comprise an incrementally decreasing null image area.

### Claim 3 (depends on 1)

The video coding system of claim 1, wherein the coder is an integrated circuit discrete from integrated circuit(s) of the pre-processor and controller, the coder operating on input frames of a fixed sized according to a locally-coded coding policy.

### Claim 4 (depends on 1)

The video coding system of claim 1, wherein the coder operates on dynamically-sized input frames as determined by the controller.

### Claim 5

A video coding method, comprising:
coding scaled frame data according to predictive coding techniques;
transmitting the coded frame data;
prior to the coding, spatially scaling frames of an input image signal according to a programmable effective image size, wherein the scaling comprises, when the effective image size is changed from a first size to a second, larger size, generating hybrid frames over a plurality of frames of the input image signal, each hybrid frame comprising:
an effective image area taken from the input image signal according to the first size, and
an incrementally increased expansion image area having image content taken from the input image signal and sized according to the second size; and
upon coding of a final hybrid frame among the plurality, transmitting an indicator of the second effective image size.

### Claim 6 (depends on 5)

The method of claim 5, wherein
the coding operates on scaled frame data of a predetermined size (M×N) and,
when the scaling operates according to an effective image size lower than M×N, the scaling generates scaled frame data at the M×N size which includes an effective image area at the effective image size and null image content over a remainder of the M×N size.

### Claim 7

A video encoding system comprising:
a pre-processor operable to create a plurality of frames from an input video signal; and
a coding engine operable to encode the plurality of frames;
wherein each frame has an effective image area at a first resolution and a null content area;
wherein for each successive frame in the plurality of frames, a block in the null content area is changed to a second resolution until all of the blocks in the null content area are at the second resolution, then changing the effective image area to the second resolution in a next frame.

### Claim 8 (depends on 7)

The system of claim 7 further comprising a controller operable to detect the first resolution and the second resolution, and to transmit a resolution instruction to a decoder.

### Claim 9 (depends on 8)

The system of claim 8 wherein the resolution instruction is sent to a decoder as out-of-band data on a communication channel.

### Claim 10 (depends on 7)

The system of claim 7 further comprising a controller operable to detect when a change in resolution is to be initiated.

### Claim 11 (depends on 10)

The system of claim 10 wherein said controller detects the change in resolution is to be initiated when the controller detects channel congestion at an output channel of the encoding system.

### Claim 12 (depends on 10)

The system of claim 10 wherein said controller detects the change in resolution is to be initiated when the controller detects a change in the quality of the encoded frames.

### Claim 13 (depends on 10)

The system of claim 10 wherein said controller detects the change in resolution is to be initiated when the controller detects a change in the input video signal.

### Claim 14 (depends on 7)

The system of claim 7 wherein changing the effective image area to the second resolution further comprises upsizing a plurality of blocks from the effective image area to the second resolution.

### Claim 15

A video decoding system comprising:
a decoding engine operable to decode a received video signal into a plurality of frames;
a post-processor operable to prepare the plurality of frames for display; and
a controller operable to receive a resolution instruction and to adjust the post-processor according to the instruction;
wherein the resolution instruction comprises resolution change information from a first resolution to a second resolution for a frame to be displayed;
wherein each successive frame in the plurality of frames contains an additional block at the second resolution.

### Claim 16 (depends on 15)

The system of claim 15 wherein the resolution information comprises an effective image area for the frame.

### Claim 17 (depends on 16)

The system of claim 16 wherein the post-processor prepares only the effective image area of the frame for display.

### Claim 18 (depends on 16)

The system of claim 16 where the post-processor prepares a frame by changing a plurality of blocks outside the effective image area to a constant.

### Claim 19 (depends on 15)

The system of claim 15 wherein the resolution instruction is received from an encoder as out-of-band data on a communication channel.

### Claim 20 (depends on 15)

The system of claim 15 further comprising a controller operable to detect an incremental resolution change between frames and to set the resolution instruction.

### Claim 21

A method of coding video comprising:
creating a plurality of frames from an input video signal, said creating including:
setting a plurality of pixel blocks in an effective image area of a frame in the plurality of frames to a first resolution;
for each successive frame in the plurality of frames, adding a pixel block to an area of the frame outside the effective image area at a second resolution; and
when all the pixel blocks outside the effective image area of a frame in the plurality of frames are at the second resolution, changing the pixel blocks of the effective image area to the second resolution;
coding the plurality of frames; and
transmitting the coded plurality of frames to a receiver on a communication channel.

### Claim 22 (depends on 21)

The method of claim 21 further comprising creating the plurality of frames upon a detection that a resolution change from the first resolution to the second resolution is to be initiated.

### Claim 23 (depends on 22)

The method of claim 22 wherein said detection further comprises detecting congestion on the communications channel.

### Claim 24 (depends on 22)

The method of claim 22 wherein said detection further comprises detecting a change in the quality of the encoded frames.

### Claim 25 (depends on 21)

The method of claim 21 further comprising transmitting a resolution instruction to the receiver.

### Claim 26 (depends on 21)

The method of claim 21 wherein changing the pixel blocks of the effective image area to the second resolution comprises upsizing a plurality of pixel blocks in the effective image area.

### Claim 27

A method of decoding video comprising:
decoding an encoded video signal from a received video signal;
receiving a resolution instruction concerning a change from a first resolution to a second resolution in the encoded video signal; and
preparing the plurality of frames for display in accordance with the resolution instruction;
wherein the encoded video signal comprises a plurality of frames, each successive frame in the plurality of frames having more pixel blocks at the second resolution than a previous frame.

### Claim 28 (depends on 27)

The method of claim 27 wherein the resolution information comprises an effective image area for the plurality of frames.

### Claim 29 (depends on 28)

The method of claim 28 wherein preparing a frame for display further comprises displaying only the effective image area of the frame.

### Claim 30 (depends on 28)

The method of claim 28 wherein preparing a frame for display further comprises changing the area of the frame outside the effective image area to a constant and displaying the frame.
