Back to results
US 6,151,362 AGranted
Joint rate control for stereoscopic video coding
AssigneesMotorola, Inc., General Instrument Corporation
InventorLimin Wang
Life of this patent
Estimating the term…
Abstract
A method and apparatus are provided for controlling and balancing a quality ratio between left and right channels of a stereoscopic video sequence. A complexity measure for each of the channels is first determined. Thereafter, a number of bits representing the video sequence is allocated between the channels according to each channel's complexity. The resultant joint rate control allows the bit rates for the two channels to vary according to the complexity measures of their pictures, while the aggregate bit rate is maintained at the channel capacity, which is a constant value.
Claims35
A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of:
- determining a complexity measure for each of said channels;
- wherein said channels comprise successive image frames of said stereoscopic video sequence, including a first channel comprising a base layer, and a second channel comprising an enhancement layer;
- allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- grouping a number of frames of the first and second channels into successive respective groups of pictures (GOPs);
- and
- forming successive joint groups of pictures (GOPs) from corresponding GOPs of the first and second channels;
- wherein:
- each joint GOP includes the same number of I, P and B pictures from the first channel, and P and B pictures from the second channel;
- and
- said allocating step allocates the same target number of bits to each joint GOP.
7 more elements
Apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising:
- a processor adapted to determine a complexity measure for each of said channels;
- wherein said channels comprise successive image frames of said stereoscopic video sequence, including a first channel comprising a base layer, and a second channel comprising an enhancement layer;
- means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- means for grousing a number of frames of the first and second channels into successive respective groups of pictures (GOPs);
- and
- means for forming successive joint groups of pictures (GOPs) from corresponding GOPs of the first and second channels;
- wherein each joint GOP includes the same number of I, P and B pictures from the first channel, and P and B pictures from the second channel;
- and
- said allocating means allocates the same target number of bits to each joint GOP.
6 more elements
A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of:
- determining a complexity measure for each of said channels;
- and
- allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- wherein:
- said channels carry successive image frames of said stereoscopic video sequence;
- the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel;
- and
- the complexity (C) of an image frame comprises the product of an average quantization parameter (Q) used for that frame and the number of bits (R) generated when that frame is quantized using the quantization parameter Q.
5 more elements
A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of:
- (a) determining a complexity measure for each of said channels;
- (b) allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- wherein:
- said channels carry successive image frames of said stereoscopic video sequence;
- the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel;
- the successive image frames of a first channel comprise I, P and B pictures;
- the successive image frames of a second channel comprise only P and B pictures;
- a complexity measure C.sub.I is maintained for I pictures;
- a complexity measure C.sub.1P is maintained for the first channel P pictures;
- a complexity measure C.sub.1B is maintained for the first channel B pictures;
- a complexity measure C.sub.2P is maintained for the second channel P pictures;
- and
- a complexity measure C.sub.2B is maintained for the second channel B pictures;
- (c) grouping a number of frames of each channel into a respective group of pictures (GoP) for the channel;
- and
- (d) forming a joint GoP from corresponding GoPs of said first and second channels;
- wherein said allocating step is responsive to a target number of bits (T.sub.m,n) determined as:
- ##EQU37## where:
- C.sub.m,n is the complexity measure for the picture type of frame (m,n),
- m is a channel index and n is a frame index,
- subscripts I, P and B denote, respectively, I, P and B pictures,
- subscripts 1 and 2 denote, respectively, the first and second channels,
- N.sub.I, N.sub.1P, N.sub.1B, N.sub.2P and N.sub.2B are, respectively, the number of remaining I pictures, first-channel P pictures, first-channel B pictures, second-channel P pictures and second-channel B pictures in the joint GoP,
- w denotes a quality balance weighting factor,
- K denotes a quantization parameter weighting factor, and
- T.sub.r is the remaining number of bits in the joint GoP, defined as T.sub.r =T.sub.r -R.sub.m',n', where.sub.-- R.sub.m',n' is the number of bits generated for a frame (m',n') coded just before frame (m,n).
23 more elements
A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of:
- determining a complexity measure for each of said channels;
- and
- allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- wherein:
- said channels carry successive image frames of said stereoscopic video sequence, each image frame being divided into a plurality of slices;
- the complexity measure of a channel is based on the complexity (C) of the image frame slices carried by the channel;
- and
- the complexity (C) of an image frame slice comprises the product of a quantization parameter (Q) used for that slice and the number of bits (R) generated when that slice is quantized using the quantization parameter Q.
5 more elements
A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of:
- (a) determining a complexity measure for each of said channels;
- and
- (b) allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- wherein:
- said channels carry successive image frames of said stereoscopic video sequence;
- the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel;
- and
- said allocating step is responsive to a target bit rate T.sub.m,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index;
- and
- (c) controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image;
- wherein:
- said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame;
- said target bit rate is maintained within said lower and upper bounds according to the parameters ##EQU40## the quantity 2N is the number of frames in a joint group of pictures (GOP), ##EQU41## is the number of bits transmitted for channel m during the nth frame period, ##EQU42## is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B.sup.d.sub.max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said video decoder;
- and
- all of the terms in the lower bound are determinable at the video encoder.
12 more elements
An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising:
- means for determining a complexity measure for each of said channels;
- and
- means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- wherein:
- said channels carry successive image frames of said stereoscopic video sequence;
- the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel;
- and
- the complexity (C) of an image frame comprises the product of an average quantization parameter (Q) used for that frame and the number of bits (R) generated when that frame is quantized using the quantization parameter Q.
5 more elements
An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising:
- means for determining a complexity measure for each of said channels;
- means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- wherein:
- said channels carry successive image frames of said stereoscopic video sequence;
- the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel;
- the successive image frames of a first channel comprise I, P and B pictures;
- the successive image frames of a second channel comprise only P and B pictures;
- a complexity measure C.sub.I is maintained for I pictures;
- a complexity measure C.sub.1P is maintained for the first channel P pictures;
- a complexity measure C.sub.1B is maintained for the first channel B pictures;
- a complexity measure C.sub.2P is maintained for the second channel P pictures;
- and
- a complexity measure C.sub.2B is maintained for the second channel B pictures;
- means for grouping a number of frames of each channel into a respective group of pictures (GoP) for the channel;
- and
- means for forming a joint GoP from corresponding GoPs of said first and second channels, said joint GoP comprising 2N frames;
- wherein said allocating means is responsive to a target number of bits (T.sub.m,n) determined as:
- ##EQU43## where:
- C.sub.m,n is the complexity measure for the picture type of frame (m,n),
- m is a channel index and n is a frame index,
- subscripts I, P and B denote, respectively, I, P and B pictures,
- subscripts 1 and 2 denote, respectively, the first and second channels,
- N.sub.I, N.sub.1P, N.sub.1B, N.sub.2P and N.sub.2B are, respectively, the number of remaining I pictures, first-channel P pictures, first-channel B pictures second-channel P pictures and second-channel B pictures in the joint GoP,
- w denotes a quality balance weighting factor,
- K denotes a quantization parameter weighting factor, and
- T.sub.r is the remaining number of bits in the joint GoP, defined as T.sub.r =T.sub.r -R.sub.m',n', where.sub.-- R.sub.m',n' is the number of bits generated for a frame (m',n') coded just before frame (m,n).
23 more elements
An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising:
- means for determining a complexity measure for each of said channels;
- and
- means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- wherein:
- said channels carry successive image frames of said stereoscopic video sequence, each image frame being divided into a plurality of slices;
- the complexity measure of a channel is based on the complexity (C) of the image frame slices carried by the channel;
- and
- the complexity (C) of an image frame slice comprises the product of a quantization parameter (Q) used for that slice and the number of bits (R) generated when that slice is quantized using the quantization parameter Q.
5 more elements
An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising:
- means for determining a complexity measure for each of said channels;
- and
- means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- wherein:
- said channels carry successive image frames of said stereoscopic video sequence;
- the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel;
- and
- said allocating means is responsive to a target bit rate T.sub.m,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index;
- and
- means for controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image;
- wherein:
- said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame;
- said target bit rate is maintained within said lower and upper bounds according to the parameters ##EQU44## the quantity 2N is the number of frames in a joint group of pictures (GOP), ##EQU45## is the number of bits transmitted for channel m during the nth frame period, ##EQU46## is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B.sup.d.sub.max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said video decoder;
- and
- all of the terms in the lower bound are determinable at the video encoder.
12 more elements
A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of:
- (a) determining a complexity measure for each of said channels;
- and
- (b) allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- wherein:
- said channels carry successive image frames of said stereoscopic video sequence;
- the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel;
- and
- said allocating step is responsive to a target bit rate T.sub.m,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index;
- and
- (c) controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image;
- wherein:
- said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame;
- and
- said target bit rate is maintained within said lower and upper bounds according to the parameters:
- ##EQU47## where the quantity R.sup.c.sub.m,n is the number of bits transmitted for channel m during the nth frame period, B.sup.c.sub.m,n is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B.sup.d.sub.max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said decoder buffer.
12 more elements
An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising:
- means for determining a complexity measure for each of said channels;
- and
- means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity;
- wherein:
- said channels carry successive image frames of said stereoscopic video sequence;
- the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel;
- and
- said allocating means is responsive to a target bit rate T.sub.m,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index;
- and
- means for controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image;
- wherein:
- said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame;
- and
- said target bit rate is maintained within said lower and upper bounds according to the parameters:
- ##EQU48## where the quantity ##EQU49## is the number of bits transmitted for channel m during the nth frame period, ##EQU50## is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B.sup.d.sub.max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said decoder buffer.
12 more elements
Dependencies are inferred from the claim text.