# US 8,542,735 B2 · Method and device for coding a scalable video stream, a data stream, and an associated decoding method and device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,542,735 B2 - Document ID: US8542735B2 - Status: Granted - Kind: B2 - Application number: 12/158,245 - Filed: 2006-12-19 - Published: 2009-05-14 - Granted: 2013-09-24 - Earliest claimed: 2005-12-20 - Assignees: Canon Kabushiki Kaisha - Applicants: Xavier Henocq; Fabrice Le Leannec - Inventors: Xavier Henocq; Fabrice Le Leannec - Web page: https://patentgrep.com/patent/US8542735B2 ## CPC - H04N 19/615: using motion compensated temporal filtering [MCTF] - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/187: the unit being a scalable video layer - H04N 19/31: in the temporal domain - H04N 19/33: in the spatial domain - H04N 19/61: in combination with predictive coding - H04N 19/63: using sub-band based transform, e.g. wavelets - H04N 19/89: involving methods or arrangements for detection of transmission errors at the decoder - H04N 19/13: Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC] - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A video sequence is coded in the form of a base level and an refinement level. A group of pictures of the base level comprises intra coded or mono-predicted end images (I,P) and bi-predicted images (B1, B2, B3), the refinement level comprises at least one image (E2, E4, . . . , E16) with no temporal correspondent in the base level. Use is made of predictive coding of said image with no temporal correspondent (E2, E4, . . . , E16) by making reference to a plurality of images, each image of said plurality being an image of the refinement level with no temporal correspondent in the base level (E4, E8, E12, E14) or an image of the refinement level (E1, E17) corresponding temporally to an end image (I1P). ## Claims ### Claim 1 A method of coding a video sequence in the form of a base level and a refinement level, wherein a group of pictures of the base level comprises intra coded or mono-predicted end images and bi-predicted images, and wherein the refinement level comprises at least one image with no temporal correspondent in the base level, the method comprising: predictive coding of said image with no temporal correspondent by making reference to a plurality of images, each image of said plurality of images being an image of the refinement level with no temporal correspondent in the base level or an image of the refinement level corresponding temporally to an end image; and determining as an anchoring image an image of the refinement level with no temporal correspondent in the base level and temporally neighboring an image of the base level that is bi-predicted with reference to the end images. ### Claim 2 (depends on 1) The coding method according to claim 1, including a step of coding at least one image of the refinement level temporally corresponding to an image of the base level by making reference to the image of the base level. ### Claim 3 (depends on 1, 2) The coding method according to claim 1 or 2, including a step of coding at least one image of the refinement level temporally corresponding to an image of the base level by making reference to at least one image with no temporal correspondent in the base level. ### Claim 4 (depends on 1, 2) The coding method according to claim 1 or 2, including a step of coding said image of the refinement level temporally corresponding to an end image by making reference to the end image. ### Claim 5 (depends on 1) The coding method according to claim 1, wherein the determining step comprises a step of selecting the image optimizing a rate-distortion criterion from the images of the refinement level with no temporal correspondent in the base level and temporally neighboring the image of the base level bi-predicted with reference to the end images. ### Claim 6 (depends on 1) The coding method according to claim 1, wherein the coding of the images of the refinement level, with no temporal correspondent in the base level and situated temporally between the anchoring image and an end image, is performed on the basis of the anchoring image and the image of the refinement level temporally corresponding to that end image. ### Claim 7 (depends on 1, 2) The coding method according to claim 1 or 2, wherein closed-loop coding is applied to the set formed by the images of the refinement level with no temporal correspondent in the base level and the images of the refinement level corresponding temporally to the end images. ### Claim 8 (depends on 7) The coding method according to claim 7, wherein the temporal prediction of a first image of the refinement level with no temporal correspondent in the base level, by making reference to a second image of the refinement level with no temporal correspondent in the base level and to an image of the refinement level corresponding temporally to an end image, comprises a step of calculating a weighted average of two differences calculated with respect to the two reference images used, the weighting taking into account temporal distances between the images. ### Claim 9 (depends on 1, 2) The coding method according to claim 1 or 2, wherein open-loop coding using motion compensated temporal filtering is applied to the set formed by the images of the refinement level with no temporal correspondent in the base level and the images of the refinement level corresponding temporally to the end images. ### Claim 10 (depends on 9) The coding method according to claim 9, wherein the temporal filtering with reference to an image of the refinement level with no temporal correspondent in the base level and to an image of the refinement level corresponding temporally to an end image is weighted by temporal distances separating the images. ### Claim 11 (depends on 1) A method of transmitting a video sequence over a transmission channel, comprising: estimating losses over the transmission channel; coding the video sequence using the coding method according to claim 1, in which a coding mode of a macroblock of an image having a temporal correspondent in the base level is determined according to the estimated losses; sending the coded sequence. ### Claim 12 A method of decoding a video sequence received in the form of a base level and a refinement level, a group of pictures of the base level comprising intra coded or mono-predicted end images and bi-predicted images, the refinement level comprising at least one image with no temporal correspondent in the base level, said method comprising: predictive decoding of said image with no temporal correspondent by making reference to a plurality of images, each image of said plurality being an image of the refinement level with no temporal correspondent in the base level or an image of the refinement level corresponding temporally to an end image, wherein the decoding of the images of the refinement level, with no temporal correspondent in the base level and situated temporally between an anchoring image and the end image, is performed on the basis of the anchoring image and the image of the refinement level temporally corresponding to that end image, the anchoring image being an image of the refinement level with no temporal correspondent in the base level and temporally neighboring an image of the base level that is bi-predicted with reference to the end images. ### Claim 13 (depends on 1, 2) A non-transitory partially or totally removable information storage means which can be read by a computer or a microprocessor storing instructions of a computer program, characterized in that it makes it possible to implement the method according to any one of claims 1, 2, and 11-12. ### Claim 14 (depends on 1, 2) A computer program product which is stored into a non-transitory programmable apparatus, characterized in that it comprises sequences of instructions for implementing the method according to any one of claims 1, 2, and 11-12, when that program is loaded and executed by the programmable apparatus. ### Claim 15 (depends on 1, 2) A non-transitory information storage means which can be read by a computer or a microprocessor storing instructions of a computer program, characterized in that it makes it possible to implement the method according to any one of claims 1, 2, and 11-12. ### Claim 16 (depends on 12) The decoding method according to claim 12, wherein closed-loop decoding is applied to the set formed by the images of the refinement level with no temporal correspondent in the base level and the images of the refinement level corresponding temporally to the end images. ### Claim 17 (depends on 16) The decoding method according to claim 16, wherein the temporal prediction of a first image of the refinement level with no temporal correspondent in the base level, by making reference to a second image of the refinement level with no temporal correspondent in the base level and to an image of the refinement level corresponding temporally to an end image, comprises a step of using a weighted average of two differences calculated with respect to the two reference images used, the weighting taking into account temporal distances between the images. ### Claim 18 (depends on 12) The decoding method according to claim 12, wherein open-loop decoding using motion compensated temporal filtering is applied to the set formed by the images of the refinement level with no temporal correspondent in the base level and the images of the refinement level corresponding temporally to the end images. ### Claim 19 (depends on 18) The decoding method according to claim 18, wherein the temporal filtering with reference to an image of the refinement level with no temporal correspondent in the base level and to an image of the refinement level corresponding temporally to an end image is weighted by temporal distances separating the images. ### Claim 20 A device for coding a video sequence in the form of a base level and an refinement level, a group of pictures of the base level comprising intra coded or mono-predicted end images and bi-predicted images, the refinement level comprising at least one image with no temporal correspondent in the base level, said device comprising: means for predictive coding of said image with no temporal correspondent by making reference to a plurality of images configured such that each image of said plurality is an image of the refinement level with no temporal correspondent in the base level or an image of the refinement level corresponding temporally to an end image; and means for determining as an anchoring image an image of the refinement level with no temporal correspondent in the base level and temporally neighboring an image of the base level that is bi-predicted with reference to the end images. ### Claim 21 (depends on 20) The coding device according to claim 20, including means for selecting the image optimizing a rate-distortion criterion from the images of the refinement level with no temporal correspondent in the base level and temporally neighboring the image of the base level bi-predicted with reference to the end images. ### Claim 22 (depends on 20, 21) The coding device according to claim 20 or 21, including means for closed-loop coding of the set formed by the images of the refinement level with no temporal correspondent in the base level and the images of the refinement level corresponding temporally to the end images. ### Claim 23 (depends on 22) The coding device according to claim 22, wherein means for temporal prediction of a first image of the refinement level with no temporal correspondent in the base level, by making reference to a second image of the refinement level with no temporal correspondent in the base level and to an image of the refinement level corresponding temporally to an end image, comprise means for calculating a weighted average of two differences calculated with respect to the two reference images used, the weighting taking into account temporal distances between the images. ### Claim 24 (depends on 20, 21) The coding device according to claim 20 or 21, including means for open-loop coding using motion compensated temporal filtering of the set formed by the images of the refinement level with no temporal correspondent in the base level and the images of the refinement level corresponding temporally to the end images. ### Claim 25 (depends on 24) The coding device according to claim 24, wherein the means for temporal filtering with reference to an image of the refinement level with no temporal correspondent in the base level and to an image of the refinement level corresponding temporally to an end image comprise means for weighting by the temporal distances separating the images. ### Claim 26 (depends on 20) A device for transmitting a video sequence over a transmission channel, characterized in that it comprises: means for estimating losses over the transmission channel; the device for coding the video sequence according to claim 20, with means for determining a coding mode of a macroblock of an image having a temporal correspondent in the base level according to the estimated losses; and means for sending the coded sequence. ### Claim 27 A device for decoding a video sequence received in the form of a base level and an refinement level, a group of pictures of the base level comprising intra coded or mono-predicted end images and bi-predicted images, the refinement level comprising at least one image with no temporal correspondent in the base level, said device comprising: means for predictive decoding of said image with no temporal correspondent by making reference to a plurality of images configured such that each image of said plurality is an image of the refinement level with no temporal correspondent in the base level or an image of the refinement level corresponding temporally to an end image, wherein the decoding of the images of the refinement level, with no temporal correspondent in the base level and situated temporally between an anchoring image and the end image, is performed on the basis of the anchoring image and the image of the refinement level temporally corresponding to that end image, the anchoring image being an image of the refinement level with no temporal correspondent in the base level and temporally neighboring an image of the base level that is bi-predicted with reference to the end images. ### Claim 28 A non-transitory computer-readable medium storing a data stream representing a video sequence in the form of a base level and an refinement level, a group of pictures of the base level comprising intra coded or mono-predicted end images and bi-predicted images, the refinement level comprising at least one image with no temporal correspondent in the base level, wherein said image with no temporal correspondent is predictively coded by making reference to a plurality of images, each image of said plurality being an image of the refinement level, with no temporal correspondent in the base level and temporally neighboring an image of the base level that is bi-predicted with reference to the end image, is determined as an anchoring image. ### Claim 29 A method of coding a video sequence in the form of a base level and a refinement level, a group of pictures of the base level comprising intra coded or mono-predicted end images and bi-predicted images, the refinement level comprising at least one B-hierarchical image with no temporal correspondent in the base level, said method comprising: predictive coding of said B-hierarchical image with no temporal correspondent by making reference to at least one image of the refinement level with no temporal correspondent in the base level; and determining as an anchoring image an image of the refinement level with no temporal correspondent in the base level and temporally neighboring an image of the base level that is bi-predicted with reference to the end images. ### Claim 30 (depends on 29) The coding method according to claim 29, wherein said B-hierarchical image is coded by making reference to said at least one image and to another image of the refinement level with no temporal correspondent in the base level. ### Claim 31 A method of decoding a video sequence received in the form of a base level and an refinement level, a group of pictures of the base level comprising intra coded or mono-predicted end images and bi-predicted images, the refinement level comprising at least one B-hierarchical image with no temporal correspondent in the base level, said method comprising: predictive decoding of said B-hierarchical image with no temporal correspondent by making reference to at least one image of the refinement level with no temporal correspondent in the base level, wherein the decoding of the images of the refinement level, with no temporal correspondent in the base level and situated temporally between an anchoring image and the end image, is performed on the basis of the anchoring image and the image of the refinement level temporally corresponding to that end image, the anchoring image being an image of the refinement level with no temporal correspondent in the base level and temporally neighboring an image of the base level that is bi-predicted with reference to the end images. ### Claim 32 (depends on 31) The decoding method according to claim 31, wherein said B-hierarchical image is decoded with reference to said at least one image and to another image of the refinement level with no temporal correspondent in the base layer. ### Claim 33 A device for coding a video sequence in the form of a base level and an refinement level, a group of pictures of the base level comprising intra coded or mono-predicted end images and bi-predicted images, the refinement level comprising at least one B-hierarchical image with no temporal correspondent in the base level, said device comprising: means for predictive coding of said B-hierarchical image with no temporal correspondent by making reference to at least one an image of the refinement level with no temporal correspondent in the base level; and means for determining as an anchoring image an image of the refinement level with no temporal correspondent in the base level and temporally neighboring an image of the base level that is bi-predicted with reference to the end images. ### Claim 34 A device for decoding a video sequence received in the form of a base level and an refinement level, a group of pictures of the base level comprising intra coded or mono-predicted end images and bi-predicted images, the refinement level comprising at least one B-hierarchical image with no temporal correspondent in the base level, said device comprising: means for predictive decoding of said B-hierarchical image with no temporal correspondent by making reference to at least one image of the refinement level with no temporal correspondent in the base level, wherein the decoding of the images of the refinement level, with no temporal correspondent in the base level and situated temporally between an anchoring image and the end image, is performed on the basis of the anchoring image and the image of the refinement level temporally corresponding to that end image, the anchoring image being an image of the refinement level with no temporal correspondent in the base level and temporally neighboring an image of the base level that is bi-predicted with reference to the end images. ### Claim 35 A non-transitory computer-readable medium storing a data stream representing a video sequence in the form of a base level and an refinement level, a group of pictures of the base level comprising intra coded or mono-predicted end images and bi-predicted images, the refinement level comprising at least one B-hierarchical image with no temporal correspondent in the base level, wherein said B-hierarchical image with no temporal correspondent is predictively coded by making reference to at least one image of the refinement level, with no temporal correspondent in the base level and temporally neighboring an image of the base level that is bi-predicted with reference to the end image, is determined as an anchoring image. ### Claim 36 (depends on 35) The non-transitory computer-readable medium storing a data stream according to claim 35, wherein said B-hierarchical image is predictively coded by making reference to said at least one image and to another image of the refinement level with on temporal correspondent in the base level.