# US 2014/0192884 A1 · Method and device for processing prediction information for encoding or decoding at least part of an image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0192884 A1 - Document ID: US20140192884A1 - Status: Published application - Kind: A1 - Application number: 14/144,323 - Filed: 2013-12-30 - Published: 2014-07-10 - Earliest claimed: 2013-01-04 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: GUILLAUME LAROCHE; EDOUARD FRANÇOIS; CHRISTOPHE GISQUET; PATRICE ONNO - Web page: https://patentgrep.com/patent/US20140192884A1 ## CPC - H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567) - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method of processing prediction information for encoding or decoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units each having associated prediction data, the method comprising for a processing block of the enhancement layer: obtaining prediction data of at least a co-located elementary unit of the reference data, the prediction data defining first predictors; and generating, from said prediction data obtained, further predictors for prediction of the current processing block. ## Claims ### Claim 1 A method of processing prediction information for encoding or decoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units each having associated prediction data defining at least one predictor for prediction of the elementary unit, the method comprising for a processing block of the enhancement layer: obtaining prediction data of at least a co-located elementary unit of the reference data, the prediction data defining first predictors; and generating, from said prediction data obtained, further predictors for prediction of the current processing block, wherein the prediction data comprises motion data indicative of the location of the corresponding predictor in the video data; and in the case where the motion data is bidirectional comprising two motion data sets, the method comprising generating at least one set of unidirectional motion data from the bidirectional motion data for the further predictors. ### Claim 2 (depends on 1) A method according to claim 1, wherein the predictor is selected from the further predictors of the unidirectional data for prediction of the processing block. ### Claim 3 (depends on 1) A method according to claim 1, wherein if one of the two motion data sets is missing the further predictors are derived from the single motion data set to provide at least a second motion data set. ### Claim 4 (depends on 3) A method according to claim 3, wherein motion vectors of a reference frame in the single motion data set are scaled to the corresponding reference frame of the second motion data set. ### Claim 5 (depends on 3) A method according to claim 3, wherein if a frame reference of the single motion data set refers to the same reference frame as the second motion data set the frame reference of the second motion data set is changed to another frame reference of the second motion data set. ### Claim 6 (depends on 3) A method according to claim 3, wherein the corresponding motion vector of the reference frame defined in the second motion data set is modified in the case where the second motion data set defines only one reference frame. ### Claim 7 (depends on 1) A method according to claim 1, further comprising scaling motion vectors of the first predictors to obtain motion vectors of the further predictors. ### Claim 8 (depends on 1) A method according to claim 1, wherein prediction data is obtained from each sub-unit of the co-located elementary unit for generation of the second set of candidate predictors. ### Claim 9 (depends on 8) A method according to claim 8 wherein prediction data is obtained from one or more sub-units neighbouring the co-located elementary unit for generation of the further predictors. ### Claim 10 (depends on 1) A method according to claim 1, wherein prediction data is obtained from one or more sub-processing blocks of the enhancement layer neighbouring the said processing block for generation of the further predictors. ### Claim 11 (depends on 1) A method according to claim 1, wherein prediction data is obtained from a temporal co-located elementary unit for generation of the further predictors. ### Claim 12 (depends on 1) A method according to claim 1, wherein a predictor from the further predictors is selectable as for a predictor for use in AMVP and/or merge mode processes ### Claim 13 (depends on 1) A method according to claim 1, wherein at least one filtering process is applied to one or more predictors of the first predictors to obtain the further predictors. ### Claim 14 (depends on 1) A method according to claim 1, wherein a first predictor is used to generate at least two predictors of the further predictors by application of at least two different respective motion compensation processes. ### Claim 15 A method of processing prediction information for encoding or decoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units each having associated prediction data defining at least one predictor for prediction of the elementary unit, the method comprising for a processing block of the enhancement layer: obtaining prediction data of at least a co-located elementary unit of the reference data, the prediction data defining first predictors; and generating, from said prediction data obtained, further predictors for prediction of the current processing block. wherein the prediction data comprises motion data indicative of the location of the corresponding predictor in the video data and a motion vector of the prediction data is used to generate at least two predictors of the further predictors by application of at least two different respective motion compensation processes. ### Claim 16 (depends on 15) A method according to claim 15 wherein at least one of the motion compensation processes comprises predicting a temporal residual of the enhancement layer processing block from a temporal residual computed between a collocated elementary unit of the base layer and a reference elementary unit of the base layer determined in accordance with motion information of the enhancement layer. ### Claim 17 (depends on 16) A method according to claim 16 wherein the said at least one motion compensation process is defined respectively by at least one of the following expressions: PREDEL=MC1[REFEL,MVEL]+{UPS[RECBL−MC4[REFBL,MVEL/2]]} PREDEL=λMC1[REFEL,MVEL]+α{UPS[RECBL−βMC4[REFBL,MVEL/2]]} PREDEL=MC1[REFEL,MVEL]+{UPS[RECBL]−FILT1(MC2[UPS[REFBL],MVEL])} PREDEL=UPS[RECBL]+FILT1(MC3[REFEL−UPS[REFBL],MVEL]) PREDEL=MC1[REFEL,MVEL]+FILT1({UPS[RECBL−MC4[REFBL,MVEL/2]]}) PREDEL=FILT2(MC1[REFEL,MVEL])+{UPS[RECBL]−FILT1(MC2[UPS[REFBL],MVEL])} PREDEL=FILT2(UPS[RECBL])+FILT1(MC3[REFEL−UPS[REFBL],MVEL]) PREDEL=FILT2(MC1[REFEL,MVEL])+FILT1({UPS[RECBL−MC4[REFBL,MVEL/2]]} where: PREDEL corresponds to the prediction of the current processing block, RECBL is the co-located elementary unit from the reconstructed base layer image, corresponding to the current enhancement layer, MVEL is the motion vector used for the temporal prediction in the enhancement layer REFEL is the reference enhancement layer image, REFBL is the reference base layer image, UPS[x] is the upsampling operator for upsampling from the base layer to the enhancement layer MC1[x,y] is the enhancement layer operator performing the motion compensated prediction from image x using motion vector y MC2[x,y] is the base layer operator performing the motion compensated prediction from image x using the motion vector y ### Claim 18 (depends on 16) A method according to claim 16 wherein the said at least one motion compensation process is defined by PREDEL=λMC1[REFEL,MVEL]+α{UPS[RECBL−βMC4[REFBL,MVEL/2]]} wherein at least one of the parameters λ, α, β is varied to obtain a plurality of predictor candidates for the further predictors. ### Claim 19 (depends on 15) A method according to claim 15, wherein the plurality of predictors obtained by the plurality of motion compensation processes are averaged to obtain a candidate predictor. ### Claim 20 A method of processing prediction information for encoding or decoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units each having associated prediction data defining at least one predictor for prediction of the elementary unit, the method comprising for a processing block of the enhancement layer: obtaining prediction data of at least a co-located elementary unit of the reference data, the prediction data defining first predictors; and generating, from said prediction data obtained, further predictors for prediction of the current processing block. wherein at least one filtering process is applied to one or more predictors of the first predictors to obtain the further predictors ### Claim 21 (depends on 20) A method according to claim 20, wherein the filtering process comprises at least one filter selected from the group of deblocking filter, SAO filter, ALF filter, and Weiner filter ### Claim 22 (depends on 1) A method according to claim 1, further comprising determining the number of unique predictor candidates among the first predictors and further predictors, or among the further predictors. ### Claim 23 (depends on 1) A method according to claim 1, further comprising selecting, based on a rate-distortion criteria, a predictor from among the first predictors and the further predictors. ### Claim 24 (depends on 23) A method according to claim 23 further comprising signalling indicator data representative of the selected predictor. ### Claim 25 (depends on 24) A method according to claim 24 wherein the indicator data comprises an index coded by a unary max code wherein the maximum value is set to a fixed value ### Claim 26 (depends on 1) A method according to claim 1, further comprising selecting, based on a rate-distortion criteria, a predictor from among the further predictors. ### Claim 27 (depends on 26) A method according to claim 26, further comprising signalling indicator data representative of the selected predictor. ### Claim 28 (depends on 27) A method according to claim 27 wherein the indicator data comprises an index coded by a unary max code wherein the maximum value is set to a fixed value ### Claim 29 (depends on 1) A method according to claim 1, comprising identifying unique predictors of the first predictors. ### Claim 30 (depends on 1) A method according to claim 1, comprising identifying unique predictors of the further predictors. ### Claim 31 (depends on 1) A method according to claim 1, comprising identifying if the at least one co-located elementary unit of the base layer has a block residual. ### Claim 32 (depends on 1) A method according to claim 1, wherein predicted border processing blocks are compared with reconstructed border processing blocks and the predictor providing the minimum distortion between the predicted and the reconstructed border processing blocks is selected. ### Claim 33 A device for processing prediction information for encoding or decoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units each having associated prediction data defining at least one predictor for prediction of the elementary unit, the device comprising: a prediction data extractor for obtaining, for a processing block of the enhancement layer, prediction data of at least a co-located elementary unit of the reference data, the prediction data defining first predictors the prediction data comprising motion data indicative of the location of the corresponding predictor in the video data; and a predictor data generator for generating, from said prediction data obtained further predictors for prediction of the current processing block. wherein the predictor data generator is configured to generate at least one set of unidirectional motion data from bidirectional motion data for the further predictors in the case where the motion data is bidirectional comprising two motion data set. ### Claim 34 (depends on 33) A device according to claim 33, wherein the predictor is selected from the further predictors of the unidirectional data for prediction of the processing block. ### Claim 35 (depends on 33) A device according to claim 33, wherein if one of the two motion data sets is missing the predictor data generator is configured to derive the further predictors from the single motion data set to provide at least a second motion data set. ### Claim 36 (depends on 35) A device according to claim 35, comprising a scaler for scaling motion vectors of a reference frame in the single motion data set to the corresponding reference frame of the second motion data set. ### Claim 37 (depends on 35) A device according to claim 35 wherein if a frame reference of the single motion data set refers to the same reference frame as the second motion data set the frame reference of the second motion data set is changed to another frame reference of the second motion data set. ### Claim 38 (depends on 35) A device according to claim 35 wherein the corresponding motion vector of the reference frame defined in the second motion data set is modified in the case where the second motion data set defines only one reference frame. ### Claim 39 (depends on 33) A device according to claim 33, further comprising a motion vector scaler for scaling motion vectors of the first motion data set to obtain motion vectors of the further predictors. ### Claim 40 (depends on 33) A device according to claim 33, wherein the prediction data extractor is configured to obtain prediction data from each sub-unit of the co-located elementary unit for generation of the further predictors. ### Claim 41 (depends on 40) A device according to claim 40 wherein the prediction data extractor is configured to obtain prediction data from one or more sub-units neighbouring the co-located elementary unit for generation of the further predictors. ### Claim 42 (depends on 33) A device according to claim 33, wherein the prediction data extractor is configured to obtain prediction data from one or more sub-processing blocks of the enhancement layer neighbouring the said processing block for generation of the further predictors. ### Claim 43 (depends on 33) A device according to claim 33, wherein the prediction data extractor is configured to obtain prediction data from a temporal co-located elementary unit for generation of the further predictors. ### Claim 44 (depends on 33) A device according to claim 33, wherein a predictor from the further predictors is selectable as a predictor for use in AMVP or merge mode processes ### Claim 45 (depends on 33) A device according to claim 33, further comprising a filter for applying at least one filtering process is applied to one or more predictors of the first predictors to obtain the further predictors and/or the prediction data generator is configured to obtain a motion vector of the prediction data to generate at least two predictors of the further predictors by application of at least two different respective motion compensation processes. ### Claim 46 A device for processing prediction information for encoding or decoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units each having associated prediction data defining at least one predictor for prediction of the elementary unit, the device comprising: a prediction data extractor for obtaining, for a processing block of the enhancement layer, prediction data of at least a co-located elementary unit of the reference data, the prediction data defining first predictors; and a predictor data generator for generating, from said prediction data obtained, further predictors for prediction of the current processing block; wherein the prediction data comprises motion data indicative of the location of the corresponding predictor in the video data; and the prediction data generator is configured to obtain a motion vector of said prediction data to generate at least two predictors of the further predictors by application of at least two different respective motion compensation processes. ### Claim 47 (depends on 46) A device according to claim 46 wherein at least one of the motion compensation processes comprises predicting a temporal residual of the enhancement layer processing block from a temporal residual computed between a collocated elementary unit of the base layer and a reference elementary unit of the base layer determined in accordance with motion information of the enhancement layer. ### Claim 48 (depends on 47) A device according to claim 47 wherein the said at least one motion compensation process is defined respectively by at least one of the following expressions: PREDEL=MC1,[REFEL,MVEL]+{UPS[RECBL−MC4[REFBL,MVEL/2]]} PREDEL=λMC1[REFEL,MVEL]+α{UPS[RECBL−βMC4[REFBL,MVEL/2]]} PREDEL=MC1[REFEL,MVEL]+{UPS[RECBL]−FILT1(MC2[UPS[REFBL],MVEL])} PREDEL=UPS[RECBL]+FILT1(MC3[REFEL−UPS[REFBL],MVEL]) PREDEL=MC1[REFEL,MVEL]+FILT1({UPS[RECBL−MC4[REFBL,MVEL/2]]}) PREDEL=FILT2(MC1[REFEL,MVEL])+{UPS[RECBL]−FILT1(MC2[UPS[REFBL],MVEL])} PREDEL=FILT2(UPS[RECBL])+FILT1(MC3[REFEL−UPS[REFBL],MVEL]) PREDEL=FILT2(MC1[REFEL,MVEL])+FILT1({UPS[RECBL−MC4[REFBL,MVEL/2]]} where: PREDEL corresponds to the prediction of the current processing block, RECBL is the co-located elementary unit from the reconstructed base layer image, corresponding to the current enhancement layer, MVEL is the motion vector used for the temporal prediction in the enhancement layer REFEL is the reference enhancement layer image, REFBL is the reference base layer image, UPS[x] is the upsampling operator for upsampling from the base layer to the enhancement layer MC1[x,y] is the enhancement layer operator performing the motion compensated prediction from image x using motion vector y MC2[x,y] is the base layer operator performing the motion compensated prediction from image x using the motion vector y ### Claim 49 (depends on 47) A device according to claim 47 wherein the said at least one motion compensation process is defined by PREDEL=λMC1[REFEL,MVEL]+α{UPS[RECBL−βMC4[REFBL,MVEL/2]]} the device further comprising means to vary at least one of the parameters λ, α, β to obtain a plurality of predictor candidates for the further predictors. ### Claim 50 (depends on 46) A device according to claim 46, wherein the plurality of predictors obtained by the plurality of motion compensation processes are averaged to obtain a candidate predictor. ### Claim 51 A device for processing prediction information for encoding or decoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units each having associated prediction data defining at least one predictor for prediction of the elementary unit, the device comprising: a prediction data extractor for obtaining, for a processing block of the enhancement layer, prediction data of at least a co-located elementary unit of the reference data, the prediction data defining first predictors; a predictor data generator for generating, from said prediction data obtained, further predictors for prediction of the current processing block; and a filter for applying at least one filtering process to one or more predictors of the first set of predictors to obtain the further predictors. ### Claim 52 (depends on 51) A device according to claim 51, wherein the filtering process comprises at least one filter selected from the group of deblocking filter, SAO filter, ALF filter, and Weiner filter ### Claim 53 (depends on 33) A device according to claim 33, further comprising a processor for determining the number of unique predictor candidates in the first and further predictor candidates, or in the further predictor candidates ### Claim 54 (depends on 33) A device according to claim 33, further comprising a selector for selecting, based on a rate-distortion criteria, a predictor candidate from among the first set of predictors and the further predictors. ### Claim 55 (depends on 54) A device according to claim 54, further comprising signalling means for signalling indicator data representative of the selected predictor candidate. ### Claim 56 (depends on 55) A device according to claim 55, wherein the indicator data comprises an index coded by a unary max code wherein the maximum value is set to a fixed value. ### Claim 57 (depends on 33) A device according to claim 33, further comprising a selector for selecting, based on a rate-distortion criteria, a predictor candidate from among the further predictors. ### Claim 58 (depends on 57) A device according to claim 57, further comprising signalling means for signalling indicator data representative of the selected predictor candidate. ### Claim 59 (depends on 58) A device according to claim 58, wherein the indicator data comprises an index coded by a unary max code wherein the maximum value is set to a fixed value ### Claim 60 (depends on 33) A device according to claim 33, further comprising a unique predictor identifier for identifying unique predictors of the first set of predictors. ### Claim 61 (depends on 33) A device according to claim 33, further comprising a unique predictor identifier for identifying unique predictors of the further predictors. ### Claim 62 (depends on 33) A device according to claim 33, further comprising a block residual identifier for identifying if the at least one co-located elementary unit of the base layer has a block residual. ### Claim 63 (depends on 33) A device according to claim 33, further comprising a comparator for comparing predicted border processing blocks with reconstructed border processing blocks wherein the selector is configured to select the predictor providing the minimum distortion between the predicted and the reconstructed border processing blocks. ### Claim 64 A method of determining prediction information for encoding or decoding at least part of an image of an enhancement layer of video data, the video data including the enhancement layer and a base layer, the enhancement layer being composed of processing blocks and the base layer being composed of elementary units, the method comprising for each enhancement layer's processing block determining interlayer predictors from the collocated base layer's elementary units for processing the processing blocks, taking into account prediction constraints on the processing blocks. ### Claim 65 (depends on 64) A method according to claim 64, wherein there is one determined interlayer predictor per enhancement layer processing block, the determined interlayer predictor being associated with the portion of the processing block which is the closest to the bottom right corner of the processing block. ### Claim 66 (depends on 1) A non-transitory computer-readable medium having computer readable instructions stored thereon which when executed by a computer cause the computer to perform a method according to claim 1.