# US 9,521,412 B2 · Method and device for determining residual data for encoding or decoding at least part of an image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,521,412 B2 - Document ID: US9521412B2 - Status: Granted - Kind: B2 - Application number: 14/134,362 - Filed: 2013-12-19 - Published: 2014-06-26 - Granted: 2016-12-13 - Earliest claimed: 2012-12-21 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Patrice Onno; Edouard Francois; Christophe Gisquet; Guillaume Laroche - Web page: https://patentgrep.com/patent/US9521412B2 ## CPC - H04N 19/86: involving reduction of coding artifacts, e.g. of blockiness - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635) - H04N 19/139: Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability - H04N 19/33: in the spatial domain - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method and device for determining residual data for encoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units each having associated residual data, the method comprising for a processing block of the enhancement layer determining, based on at least the number of elementary units of the base layer spatially corresponding, at least partially, to the processing block, whether or not to use residual data of one or more of said spatially corresponding elementary units of the base layer for prediction of the processing block. ## Claims ### Claim 1 A method of determining residual data for encoding or decoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units each having associated residual data, the method comprising: for a processing block of the enhancement layer, determining, based on at least the number of elementary units of the base layer spatially corresponding, at least partially, to the processing block, whether or not to use residual data of one or more of said spatially corresponding elementary units of the base layer for prediction of the processing block, wherein in the case where only one elementary unit spatially corresponds to the processing block, the residual data associated with the said one elementary unit is used for prediction of the corresponding processing block of the enhancement layer; otherwise, in the case where a plurality of elementary units spatially correspond at least partially to the processing block, either: 1) the residual data of none of the said plurality of elementary units is used for prediction of the corresponding processing block of the enhancement layer; or 2) the residual data of at least one of the plurality of elementary units is used, under a predefined condition, for prediction of the corresponding processing block of the enhancement layer, wherein the predefined condition defines that the motion vectors of the plurality of elementary units are consistent with one another. ### Claim 2 (depends on 1) The method according to claim 1 wherein the motion vectors of the plurality of elementary units are considered to be consistent with one another if the direction of each of the motion vectors fall within a predetermined angular tolerance. ### Claim 3 (depends on 1) The method according to claim 1 wherein the motion vectors of the plurality of elementary units are considered to be consistent with one another if the magnitude of each of the motion vectors falls within a predetermined range of magnitudes. ### Claim 4 (depends on 2) The method according to claim 2 wherein data representative of the tolerance is transmitted in the bitstream. ### Claim 5 (depends on 1) The method according to claim 1 wherein, the predefined condition is the application of a filtering process to the residual data of the plurality of elementary units. ### Claim 6 (depends on 5) The method according to claim 5 wherein the filtering process is applied to the residual data at the inner borders between the plurality of elementary units. ### Claim 7 (depends on 6) The method according to claim 6 wherein the filtering process comprises application of a deblocking filter. ### Claim 8 (depends on 6) The method according to claim 6 wherein the filtering strength of the filtering process is determined in dependence upon the difference between the motion vectors of the two elementary units delimited by the respective inner border. ### Claim 9 (depends on 8) The method according to claim 8 wherein the filtering strength is dependent on the difference in at least one of the angular directions and the magnitudes of the motion vectors. ### Claim 10 (depends on 1) The method according to claim 1 wherein the predefined condition is the application of only partial residual data for prediction of the processing block. ### Claim 11 (depends on 10) The method according to claim 10 wherein the partial residual data corresponds to the reconstruction of the low frequency component of the inverse DCT such that the residual data from the edges is not taken into account. ### Claim 12 (depends on 10) The method according to claim 10 wherein only the DC coefficients of the residual data are used for prediction of the processing block. ### Claim 13 (depends on 1) The method according to claim 1 wherein the state of predetermined flag signalled in the bitstream is indicative of whether one or more elementary units spatially correspond to the processing block. ### Claim 14 A device for determining residual data for encoding or decoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units each having associated residual data, the device comprising: a residual data processor for determining, based on at least the number of elementary units of the base layer spatially corresponding, at least partially, to a processing block of the enhancement layer, whether or not to use residual data of one or more of said spatially corresponding elementary units of the base layer for prediction of the processing block, wherein the residual data processor is configured to determine in the case where only one elementary unit spatially corresponds to the processing block, the residual data associated with the said one elementary unit is to be used for prediction of the corresponding processing block of the enhancement layer; otherwise, in the case where a plurality of elementary units spatially correspond at least partially to the processing block, either: 1) the residual data of none of the said plurality of elementary units is to be used for prediction of the corresponding processing block of the enhancement layer; or 2) the residual data of at least one of the plurality of elementary units is to be used, under a predefined condition, for prediction of the corresponding processing block of the enhancement layer, wherein the predefined condition defines that the motion vectors of the plurality of elementary units are consistent with one another. ### Claim 15 (depends on 14) The device according to claim 14 wherein the residual data processor is configured to consider motion vectors of the plurality of elementary units to be consistent with one another if the direction of each of the motion vectors fall within a predetermined angular tolerance. ### Claim 16 (depends on 1) A non-transitory computer readable storage medium storing instructions of a computer program for implementing a method, according to claim 1.