# US 2003/0223643 A1 · Efficiency FGST framework employing higher quality reference frames > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2003/0223643 A1 - Document ID: US20030223643A1 - Status: Published application - Kind: A1 - Application number: 10/156,410 - Filed: 2002-05-28 - Published: 2003-12-04 - Earliest claimed: 2002-05-28 - Assignees: Koninklijke Philips Electronics N.V. - Inventors: Mihaela Van Der Schaar - Web page: https://patentgrep.com/patent/US20030223643A1 ## CPC - H04N 19/36: Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability - H04N 19/31: in the temporal domain - H04N 19/34: Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS] - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method and apparatus for coding video includes constructing motion-compensated extended reference frames from at least portions of temporal residual image frames. The motion-compensated extended reference frames are used for bi-directionally and/or uni-directionally predicting additional levels of FGS motion-compensated residual image or temporal frames. ## Claims ### Claim 1 A method of coding video, comprising the steps of: constructing motion-compensated extended reference frames from at least portions of first level motion-compensated residual image frames; predicting a second level of motion-compensated residual image frames from the motion-compensated extended reference frames and the video; and fine granular scalable coding the second level motion-compensated residual image frames into a data-stream of temporal frames. ### Claim 2 (depends on 1) The method of coding video according to claim 1, further comprising the steps of: coding the video into a data-stream of base layer frames; computing residual image frames from the base layer frames; and fine granular scalable coding the residual image frames into a data-stream of SNR quality frames. ### Claim 3 (depends on 2) The method of coding video according to claim 2, further comprising the steps of: constructing motion-compensated extended base layer reference frames from the base layer frames and at least portions of the residual image frames; predicting the first level motion-compensated residual image frames from the motion-compensated extended base layer reference frames and the video; and fine granular scalable coding the first level motion-compensated residual image frames into the data-stream of temporal frames. ### Claim 4 (depends on 3) The method of coding video according to claim 3, further comprising the step of combining the data-stream of temporal frames with the data-stream of SNR quality frames to construct a single data-stream of the temporal and SNR quality frames. ### Claim 5 (depends on 2) The method of coding video according to claim 2, further comprising the step of combining the data-stream of temporal frames with the data-stream of SNR quality frames to construct a single data-stream of the temporal and SNR quality frames. ### Claim 6 A method of coding a video signal, comprising the steps of: constructing motion-compensated extended reference frames from at least portions of first level motion-compensated residual image frames; predicting a second level of motion-compensated residual image frames from the motion-compensated extended reference frames and the video; and fine granular scalable coding the second level motion-compensated residual image frames into a data-stream of temporal frames. ### Claim 7 (depends on 6) The method of coding a video signal according to claim 6, further comprising the steps of: coding the video into a data-stream of base layer frames; computing residual image frames from the base layer frames; and fine granular scalable coding the residual image frames into a data-stream of SNR quality frames. ### Claim 8 (depends on 7) The method of coding a video signal according to claim 7, further comprising the steps of: constructing motion-compensated extended base layer reference frames from the base layer frames and at least portions of the residual image frames; predicting the first level motion-compensated residual image frames from the motion-compensated extended base layer reference frames and the video; and fine granular scalable coding the first level motion-compensated residual image frames into the data-stream of temporal frames. ### Claim 9 (depends on 8) The method of coding a video signal according to claim 8, further comprising the step of combining the data-stream of temporal frames with the data-stream of SNR quality frames to construct a single data-stream of the temporal and SNR quality frames. ### Claim 10 (depends on 7) The method of coding a video signal according to claim 7, further comprising the step of combining the data-stream of temporal frames with the data-stream of SNR quality frames to construct a single data-stream of the temporal and SNR quality frames. ### Claim 11 An apparatus for coding video, comprising: means for constructing motion-compensated extended reference frames from at least portions of first level motion-compensated residual image frames; means for predicting a second level of motion-compensated residual image frames from the motion-compensated extended reference frames and the video; and means for fine granular scalable coding the second level motion-compensated residual image frames into a data-stream of temporal frames. ### Claim 12 (depends on 11) The apparatus according to claim 11, further comprising: means for coding the video into a data-stream of base layer frames; means for computing residual image frames from the base layer frames; and means for fine granular scalable coding the residual image frames into a data-stream of SNR quality frames. ### Claim 13 (depends on 12) The apparatus according to claim 12, further comprising: means for constructing motion-compensated extended base layer reference frames from the base layer frames and at least portions of the residual image frames; means for predicting the first level motion-compensated residual image frames from the motion-compensated extended base layer reference frames and the video; and means for fine granular scalable coding the first level motion-compensated residual image frames into the data-stream of temporal frames. ### Claim 14 (depends on 13) The apparatus according to claim 13, further comprising means for combining the data-stream of temporal frames with the data-stream of SNR quality frames to construct a single data-stream of the temporal and SNR quality frames. ### Claim 15 (depends on 12) The apparatus video according to claim 12, further comprising means for combining the data-stream of temporal frames with the data-stream of SNR quality frames to construct a single data-stream of the temporal and SNR quality frames. ### Claim 16 A memory medium for coding video, comprising: code for constructing motion-compensated extended reference frames from at least portions of first level motion-compensated residual image frames; code for predicting a second level of motion-compensated residual image frames from the motion-compensated extended reference frames and the video; and code for fine granular scalable coding the second level motion-compensated residual image frames into a data-stream of temporal frames. ### Claim 17 (depends on 16) The memory medium according to claim 16, further comprising: code for coding the video into a data-stream of base layer frames; code for computing residual image frames from the base layer frames; and code for fine granular scalable coding the residual image frames into a data-stream of SNR quality frames. ### Claim 18 (depends on 17) The memory medium according to claim 17, further comprising: code for constructing motion-compensated extended base layer reference frames from the base layer frames and at least portions of the residual image frames; code for predicting the first level motion-compensated residual image frames from the motion-compensated extended base layer reference frames and the video; and code for fine granular scalable coding the first level motion-compensated residual image frames into the data-stream of temporal frames. ### Claim 19 (depends on 18) The memory medium according to claim 18, further comprising code for combining the data-stream of temporal frames with the data-stream of SNR quality frames to construct a single data-stream of the temporal and SNR quality frames. ### Claim 20 (depends on 17) The memory medium according to claim 17, further comprising means for combining the data-stream of temporal frames with the data-stream of SNR quality frames to construct a single data-stream of the temporal and SNR quality frames. ### Claim 21 A method of decoding a base layer data-stream and an enhancement layer data-stream, which collectively represent a video, comprising the steps of: constructing motion-compensated extended references from at least portions of first level motion-compensated residual image frames; fine granular scalable decoding a second level temporal frame portion of the enhancement layer data-stream to construct second level of motion-compensated residual frames; combining the motion-compensated extended reference frames with the second level motion-compensated residual frames to construct second level temporal frames. ### Claim 22 (depends on 21) The method of decoding according to claim 21, further comprising the steps of: decoding the base layer data-stream to construct base layer frames; fine granular scalable decoding an SNR quality frame portion of the enhancement layer data-stream to construct SNR quality frames; constructing motion-compensated extended base layer reference frames from the base layer frames and at least portions of the SNR quality frames; fine granular scalable decoding a temporal frame portion of the enhancement layer data-stream to construct the first level motion-compensated residual frames; combining the motion-compensated extended base layer reference frames with the first level motion-compensated residual frames to construct first level temporal frames. ### Claim 23 (depends on 22) The method of decoding according to claim 22, further comprising the step of combining the base layer frames and the SNR quality frames into an enhanced video. ### Claim 24 (depends on 22) The method of decoding according to claim 22, further comprising the step of combining the base layer frames, the SNR quality frames, and the first and second level temporal frames into an enhanced video. ### Claim 25 An apparatus for decoding a base layer data-stream and an enhancement layer data-stream, which collectively represent a video, comprising: means for constructing motion-compensated extended references from at least portions of first level motion-compensated residual image frames; means for fine granular scalable decoding a second level temporal frame portion of the enhancement layer data-stream to construct second level of motion-compensated residual frames; means for combining the motion-compensated extended reference frames with the second level motion-compensated residual frames to construct second level temporal frames. ### Claim 26 (depends on 25) The apparatus according to claim 25, further comprising: means for decoding the base layer data-stream to construct base layer frames; means for fine granular scalable decoding an SNR quality frame portion of the enhancement layer data-stream to construct SNR quality frames; means for constructing motion-compensated extended base layer reference frames from the base layer frames and at least portions of the SNR quality frames; means for fine granular scalable decoding a temporal frame portion of the enhancement layer data-stream to construct the first level motion-compensated residual frames; means for combining the motion-compensated extended base layer reference frames with the first level motion-compensated residual frames to construct first level temporal frames. ### Claim 27 (depends on 26) The apparatus according to claim 26, further comprising means for combining the base layer frames and the SNR quality frames into an enhanced video. ### Claim 28 (depends on 26) The apparatus according to claim 26, further comprising means for combining the base layer frames, the SNR quality frames, and the first and second level temporal frames into an enhanced video. ### Claim 29 A memory medium for decoding a base layer data-stream and an enhancement layer data-stream, which collectively represent a video, comprising: code for constructing motion-compensated extended references from at least portions of first level motion-compensated residual image frames; code for fine granular scalable decoding a second level temporal frame portion of the enhancement layer data-stream to construct second level of motion-compensated residual frames; code for combining the motion-compensated extended reference frames with the second level motion-compensated residual frames to construct second level temporal frames. ### Claim 30 (depends on 29) The memory medium according to claim 29, further comprising: code for decoding the base layer data-stream to construct base layer frames; code for fine granular scalable decoding an SNR quality frame portion of the enhancement layer data-stream to construct SNR quality frames; code for constructing motion-compensated extended base layer reference frames from the base layer frames and at least portions of the SNR quality frames; code for fine granular scalable decoding a temporal frame portion of the enhancement layer data-stream to construct the first level motion-compensated residual frames; code for combining the motion-compensated extended base layer reference frames with the first level motion-compensated residual frames to construct first level temporal frames. ### Claim 31 (depends on 30) The memory medium according to claim 30, further comprising code for combining the base layer frames and the SNR quality frames into an enhanced video. ### Claim 32 (depends on 30) The memory medium according to claim 30, further comprising code for combining the base layer frames, the SNR quality frames, and the first and second level temporal frames into an enhanced video.