# US 2002/0118755 A1 · Video coding architecture and methods for using same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2002/0118755 A1
- Document ID: US20020118755A1
- Status: Published application
- Kind: A1
- Application number: 09/925,769
- Filed: 2001-08-09
- Published: 2002-08-29
- Earliest claimed: 2001-01-03
- Assignees: Nokia Corporation
- Inventors: Marta Karczewicz; Ragip Kurceren
- Web page: https://patentgrep.com/patent/US20020118755A1

## CPC

- H04N 21/8451: {using Advanced Video Coding [AVC]}
- H04N 19/51: Motion estimation or motion compensation
- H04N 19/58: Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one (H04N19/23 takes precedence)
- H04N 19/61: in combination with predictive coding
- H04N 19/89: involving methods or arrangements for detection of transmission errors at the decoder
- H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A decoder and method for using a new picture or frame type is provided. This type is referred to a an SP-picture. The temporal redundancies are not exploited in I-frames, compression efficiency of I-frame coding is significantly lower than the predictive coding. A method in accordance with an embodiment of the invention allows use of motion compensated predictive coding to exploit temporal redundancy in the sequence while still allowing identical reconstruction of the frame using different reference frames. Methods in accordance with embodiments of the invention using this new picture type provide for error resilience/recovery, bandwidth scalability, bitstream switching, processing scalability, random access and other functions. The SP-type picture provides for, among other functions, switching between different bitstreams, random access, fast forward and fast error-recovery by replacing I-pictures to increase the coding efficiency. SP-pictures have the property that identical SP-frames may be obtained even when they are predicted using different reference frames.

## Claims

### Claim 1

A picture data signal embodied in a carrier wave for the reconstruction of video frames, said picture data signal comprising:
data blocks encoded using spatial correlation among the pixels (intra blocks);
data blocks encoded using both spatial and temporal correlation (inter or copy blocks); and providing identical reconstruction of said video frames even when different reference frames are used.

### Claim 2 (depends on 1)

The picture data signal of claim 1 wherein the value of each pixel S(x,y) in the inter or copy coded block is decoded as a weighted sum of a plurality of basis functions whose weigh values are obtained by quantization and dequantization of a plurality of reconstruction image coefficients formed using motion compensation prediction of this block constructed using previously transmitted and decoded pictures, received motion vectors and received quantized prediction error coefficients.

### Claim 3

A picture data structure for the reconstruction of video frames, said picture data structure comprising:
data blocks encoded using spatial correlation among the pixels (intra blocks); and
data blocks encoded using both spatial and temporal correlation (inter or copy blocks), and providing identical reconstruction of frames even when different reference frames are used.

### Claim 4 (depends on 3)

The picture data structure of claim 3 wherein the value of each pixel S(x,y) in the inter or copy coded block is decoded as a weighted sum of a plurality of basis functions whose weigh values are obtained by quantization and dequantization of a plurality of reconstruction image coefficients formed using motion compensation prediction of this block constructed using previously transmitted and decoded pictures, received motion vectors and received quantized prediction error coefficients.

### Claim 5

A method of coding a switching picture S.sub.12 for switching from a first bitstream having a SP-picture S.sub.1 to a second bitstream having a second SP-picture S.sub.2, said method comprising the steps of:
copying the bitstream of the intra macroblocks in second SP-picture S.sub.2 to switching picture S.sub.12; and
encoding remaining macroblocks comprising the steps of:
forming a predicted frame for S.sub.12 by performing motion estimation with a plurality of reference pictures which are preceding S.sub.1 in said first bitstream;
calculating a set of transform coefficients for predicted image c.sub.pred by performing a forward transform;
quantizing the obtained transform coefficients to resulting in quantized coefficient levels; and
subtracting said quantized coefficient levels from a set of corresponding coefficient levels of said second SP-picture S.sub.2.

### Claim 6

A decoder for decoding encoded data wherein identical frames may be obtained even when they are predicted using different reference frames, said decoder comprising:
means for forming a prediction block P of a current block of data I using a plurality of motion vectors and a reference frame;
means for calculating a plurality of transform coefficients c.sub.pred for said prediction block;
means for calculating a plurality of quantized reconstruction coefficients I.sub.rec for said current block of data, wherein
I.sub.rec=(c.sub.pred×A(QP1)+I.sub.err×F(QP1,QP2)+f×220)//220 where F(QP1,QP2)=(220×A(QP1)+0.5×A(QP2))//A(QP2);
means for dequantizing said plurality of quantized reconstruction image coefficients, creating a plurality of dequantized coefficients d.sub.rec; and
means for inverse transforming said plurality of dequantized coefficients.

### Claim 7 (depends on 6)

The decoder of claim 6 wherein means for calculating a plurality of quantized reconstruction coefficients I.sub.rec for said current block of data is comprised of:
means for calculating reconstruction image coefficients c.sub.rec wherein
c.sub.rec=c.sub.pred+alpha(QP2)×I.sub.err,
wherein alpha(QP2) is a quantization parameter dependent on the method of quantization and used QP value; and
means for quantizing said reconstruction coefficients creating a plurality of quantized reconstructed image coefficients I.sub.rec.

### Claim 8 (depends on 6, 7)

The decoder of claim 6 or 7, further comprising means for normalizing said plurality of inverse transformed dequantized coefficients.

### Claim 9 (depends on 8)

The decoder of claim 8, further comprising means for filtering said plurality of normalized inverse transformed dequantized coefficients.

### Claim 10

A decoder for decoding a block of encoded data wherein identical frames may be obtained even when they are predicted using different reference frames, said decoder comprising:
frame memory for storing a reference frame;
demultiplexer for receiving and demultiplexing said encoded data into motion information and current frame information;
motion compensation predictor coupled to said demultiplexer and said frame memory for receiving said motion information and constructing a prediction of the current block based on said motion information and reference frame;
transformer coupled to said motion compensation predictor for creating a plurality of transform coefficients;
quantizer coupled to said transformer for quantizing said plurality of coefficients;
adder coupled to said quantizer and said demultiplexer for adding current frame information and said quantized plurality of coefficients to form a plurality of coefficients for a reconstructed frame;
inverse quantizer coupled to said adder; and
inverse transformer coupled to said inverse quantizer.

### Claim 11 (depends on 10)

The decoder of claim 10, wherein normalizer is coupled to said demultiplexer, said normalizer for normalizing current frame data.

### Claim 12 (depends on 11)

The decoder of claim 11, wherein adder coupled to said quantizer and said normalizer, and said transformer for adding normalized current frame information and said plurality of transform coefficients.

### Claim 13

A method for decoding a frame of video data, comprising the steps of:
forming a prediction of a current block of data using a plurality of motion vectors and a reference frame creating a predicted block;
calculating a plurality of transform coefficients c.sub.pred from said predicted block;
calculating a plurality of quantized reconstruction coefficients I.sub.rec for said current block of data using
I.sub.rec=(c.sub.pred×A(QP1)+I.sub.err×F(QP1,QP2)+f×220)//220, wherein F(QP1,QP2)=(220×A(QP1)+0.5×A(QP2))//A(QP2);
dequantizing said plurality of quantized reconstruction image coefficients, creating a plurality of dequantized coefficients d.sub.rec; and
inverse transforming said plurality of dequantized coefficients.

### Claim 14

A method for decoding a frame of video data, comprising the steps of:
forming a prediction of a current block of data using a plurality of motion vectors and a reference frame creating a predicted block;
calculating a plurality of transform coefficients c.sub.pred from said predicted block;
calculating reconstruction image coefficients c.sub.rec wherein
c.sub.rec=c.sub.pred+alpha(QP2)×I.sub.err, wherein alpha(QP2) is a quantization parameter dependent on the method of quantization and used QP value; and
quantizing said reconstruction coefficients creating a plurality of quantized reconstructed image coefficients I.sub.rec;
dequantizing said plurality of quantized reconstruction image coefficients, creating a plurality of dequantized coefficients d.sub.rec; and
inverse transforming said plurality of dequantized coefficients.

### Claim 15 (depends on 13, 14)

The methods of claim 13 or 14, further comprising the step of normalizing said plurality of inverse transformed dequantized coefficients.

### Claim 16 (depends on 15)

The method of claim 15, further comprising the step of filtering said plurality of normalized inverse transformed dequantized coefficients.

### Claim 17

An encoder for encoding a frame of video data, comprising the steps of:
means for forming a prediction of a current block of data using a plurality of motion vectors and a reference frame;
means for calculating a plurality of transform coefficients c.sub.orig for said current block of data corresponding to a set of basis functions;
means for calculating a plurality of transform coefficients c.sub.pred for said predicted block;
means for quantizing said predicted block coefficients using quantization parameter QP=QP1 creating a plurality of quantized prediction image coefficients;I.sub.pred.
means for calculating the prediction error coefficients using
c.sub.err=c.sub.orig−I.sub.pred×alpha(QP1)
wherein alpha(QP1)=(220+0.5×A(QP1))//A(QP1); and
means for quantizing said prediction error coefficients.

### Claim 18

A method for encoding a frame of video data, comprising the steps of:
forming a prediction of a current block of data using a plurality of motion vectors and a reference frame;
means for calculating a plurality of transform coefficients c.sub.orig for said current block of data corresponding to a set of basis functions;
calculating a plurality of transform coefficients c.sub.pred for said predicted block;
quantizing said predicted block coefficients using quantization parameter QP=QP1 creating a plurality of quantized prediction image coefficientsI.sub.pred;
calculating the prediction error coefficients using:
c.sub.err=c.sub.orig−I.sub.pred×alpha(QP1); wherein alpha(QP1)=(220+0.5×A(QP1))//A(QP1); and
quantizing said prediction error coefficients.

### Claim 19

A method for switching between a plurality of bitstreams in a data communication system, wherein said bitstreams correspond to a same data sequence but are encoded at different bitrates, said method comprising the steps of:
placing a first SP-picture within a first bitstream in a location at which switching from a second bitstream to said first bitstream is desired;
transmitting a second SP-picture wherein said first SP-picture and said second SP-picture are represented by different bitstreams, but wherein said first SP-picture reconstructed values and said second SP-picture reconstructed values are identical.

### Claim 20

A method for enabling access in a data stream, said method comprising the steps of:
placing a plurality of SP-pictures at fixed intervals within a first bitstream;
generating an I-picture and an SP-picture for each one of said plurality of SP-pictures in said first bitstream;
storing said I-picture in a second bitstream at a temporal location preceding said each one of said plurality of SP-pictures in said first bitstream; and
storing said SP-picture in said second bitstream at same temporal locations as each of said SP-pictures in said first bitstream.

### Claim 21 (depends on 20)

The method of claim 20, wherein said second bitstream comprises only SP-pictures predicted from each other, but at longer temporal periods.

### Claim 22 (depends on 20)

The method of claim 20, wherein said first bitstream comprises only SP-pictures predicted from each other, but at longer temporal periods.

### Claim 23

A method for providing Video Redundancy Coding (VRC), comprising the steps of:
dividing a sequence of pictures into a plurality of threads wherein all pictures are assigned to one of said plurality of threads in a round-robin fashion;
coding each of said plurality of threads independently;
creating a SP-picture, wherein all of said threads converge; and
starting a second plurality of threads from said SP-picture.

### Claim 24

A method for providing error control in a data stream between a sender and a client in a communication system, said method comprising:
creating a plurality of representations of a picture in the form of a plurality of SP-pictures predicted from different reference pictures;
signaling said sender information regarding lost pictures and a one of said plurality of representations received by said client; and
sending said client a SP-picture which is the next picture in said one of plurality of representations received by client.

### Claim 25

A method of reconstructing blocks of encoded data comprising the steps of:
forming a prediction P(x,y) of a current block using a plurality of received motion vectors and a reference frame;
performing a forward transform on P(x,y) to obtain a set of transform coefficients c.sub.pred for P(x,y);
quantizing said set of transform coefficients c.sub.pred, resulting in quantized prediction image coefficients I.sub.pred.
adding said received quantized coefficients for the prediction error I.sub.err to I.sub.pred resulting in quantized reconstruction image coefficients I.sub.rec using:
I.sub.rec=I.sub.pred+(beta(QP2)×I.sub.err+0.5×beta(QP1))//beta(QP1).
wherein beta(QP) is a parameter dependent on method of quantization and used OP value;
dequantizing I.sub.rec resulting in dequantized coefficients d.sub.rec; and
performing inverse transform for d.sub.rec.

### Claim 26 (depends on 25)

The method of claim 25, further comprising the step of normalizing said plurality of inverse transformed dequantized coefficients.

### Claim 27 (depends on 26)

The method of claim 26, further comprising the step of filtering said plurality of normalized inverse transformed dequantized coefficients.
