# US 12,010,334 B2 · Scalable video coding using base-layer hints for enhancement layer motion parameters > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,334 B2 - Document ID: US12010334B2 - Status: Granted - Kind: B2 - Application number: 16/850,588 - Filed: 2020-04-16 - Published: 2020-07-30 - Granted: 2024-06-11 - Earliest claimed: 2012-10-01 - Assignees: Ge Video Compression, LLC - Applicants: Ge Video Compression, LLC - Inventors: Tobias Hinz; Haricharan Lakshman; Jan Stegemann; Philipp Helle; Mischa Siekmann; Karsten Suehring; Detlev Marpe; Heiko Schwarz; Christian Bartnik; Ali Atef Ibrahim Khairat Abdelhamid; Heiner Kirchhoffer; Thomas Wiegand - Web page: https://patentgrep.com/patent/US12010334B2 ## CPC - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh - H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635) - H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference - H04N 19/157: Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter - H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/18: the unit being a set of transform coefficients - H04N 19/187: the unit being a scalable video layer - H04N 19/198: {including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value} - H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) - H04N 19/33: in the spatial domain - H04N 19/48: using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data (motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513) - H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109) - H04N 19/52: by predictive encoding - H04N 19/587: involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence - H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution - H04N 19/593: involving spatial prediction techniques - H04N 19/61: in combination with predictive coding - H04N 19/615: using motion compensated temporal filtering [MCTF] - H04N 19/82: involving filtering within a prediction loop - H04N 19/865: {with detection of the former encoding block subdivision in decompressed video} - H04N 19/11: among a plurality of spatial predictive coding modes - H04N 19/34: Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS] - H04N 19/36: Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability - H04N 19/463: by compressing encoding parameters before transmission - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Information available from coding/decoding the base layer, i.e. base-layer hints, is exploited to render the motion-compensated prediction of the enhancement layer more efficient by more efficiently coding the enhancement layer motion parameters. ## Claims ### Claim 1 A scalable video decoder comprising: a processor; and a predictive decoder configured to process, using the processor, information from a data stream to reconstruct an enhancement layer signal of the video based on block-wise predictive coding that includes: gathering, for a current block of a frame of the enhancement layer signal, a set of enhancement layer motion parameter candidates from already reconstructed neighboring blocks of the frame, the reconstructed neighboring blocks neighboring the current block, gathering a set of one or more base layer motion parameters of a block of a base layer signal, the block of the base layer signal being collocated to the current block of the frame of the enhancement layer signal, enlarging the set of enhancement layer motion parameter candidates with one or more base layer motion parameters of the set of one or more base layer motion parameters or a scaled version thereof to obtain an extended motion parameter candidate set, each of the set of enhancement layer motion parameter candidates and the one or more base layer motion parameters or a scaled version thereof in the extended motion parameter candidate set is a separate element of the extended motion parameter candidate set, wherein the extended motion parameter candidate set is ordered depending on the base layer motion parameters, extracting, from the data stream, a syntax element corresponding to the current block identifying one of the motion parameter candidates of the extended motion parameter candidate set to be selected for the current block, selecting the one of the motion parameter candidates of the extended motion parameter candidate set based on the syntax element, generating, by motion compensated prediction, a prediction of the current block of the enhancement layer signal using the selected one of the motion parameter candidates of the extended motion parameter candidate set, and reconstructing at least a portion of the enhancement layer signal based on the prediction of the current block and the residual signal. ### Claim 2 (depends on 1) The scalable video decoder according to claim 1, wherein the base layer motion parameters are scaled according to a spatial resolution ratio between the base layer signal and the enhancement layer signal to acquire the scaled version of the base layer motion parameters. ### Claim 3 (depends on 1) The scalable video decoder according to claim 1, the gathering, for the block of the frame of the enhancement layer signal, of the set of enhancement layer motion parameter candidates is performed from temporally and/or spatially neighboring already reconstructed blocks of the enhancement layer by copying and/or combining motion parameters of the neighboring already reconstructed blocks. ### Claim 4 (depends on 1) The scalable video decoder according to claim 1, wherein the block of the base layer signal, collocated to the block of the frame of the enhancement layer signal, is identified by checking as to which block of a temporally corresponding frame of the base layer signal overlays a position spatially corresponding a predetermined sample position of the block of the frame of the enhancement layer signal. ### Claim 5 (depends on 1) The scalable video decoder according to claim 1, wherein the set of one or more base layer motion parameters are checked to determine whether the set of one or more base layer motion parameters are obtained by merging with another block of the base layer signal. ### Claim 6 A scalable video encoder comprising: a processor; and an encoder configured to encode, using the processor, an enhancement layer signal of a video based on block-wise predictive coding that includes: gathering, for a current block of a frame of the enhancement layer signal, a set of enhancement layer motion parameter candidates from already reconstructed neighboring blocks of the frame, the reconstructed neighboring blocks neighboring the current block, gathering a set of one or more base layer motion parameters of a block of a base layer signal, the block of the base layer signal being collocated to the current block of the frame of the enhancement layer signal, enlarging the set of enhancement layer motion parameter candidates with one or more base layer motion parameters of the set of one or more base layer motion parameters or a scaled version thereof to obtain an extended motion parameter candidate set, each of the set of enhancement layer motion parameter candidates and the one or more base layer motion parameters or a scaled version thereof in the extended motion parameter candidate set is a separate element of the extended motion parameter candidate set, wherein the extended motion parameter candidate set is ordered depending on the base layer motion parameters, selecting one of the motion parameter candidates of the extended motion parameter candidate set based on a syntax element corresponding to the current block to be signaled in a data stream, the syntax element specifically indicating the one of the motion parameter candidates of the extended motion parameter candidate set to be selected for the current block, generating, by motion compensated prediction, a prediction of the current block of the enhancement layer signal using the selected one of the motion parameter candidates of the extended motion parameter candidate set, and obtaining a residual signal for the current block based on a difference of the current block and the prediction of the current block, wherein the residual signal and the syntax element are encoded into the data stream. ### Claim 7 (depends on 6) The scalable video encoder according to claim 6, wherein the base layer motion parameters are scaled according to a spatial resolution ratio between the base layer signal and the enhancement layer signal to acquire the scaled version of the base layer motion parameters. ### Claim 8 (depends on 6) The scalable video encoder according to claim 6, the gathering, for the block of the frame of the enhancement layer signal, of the set of enhancement layer motion parameter candidates is performed from temporally and/or spatially neighboring already reconstructed blocks of the enhancement layer by copying and/or combining motion parameters of the neighboring already reconstructed blocks. ### Claim 9 (depends on 6) The scalable video encoder according to claim 6, wherein the block of the base layer signal, collocated to the block of the frame of the enhancement layer signal, is identified by checking as to which block of a temporally corresponding frame of the base layer signal overlays a position spatially corresponding a predetermined sample position of the block of the frame of the enhancement layer signal. ### Claim 10 (depends on 6) The scalable video encoder according to claim 6, wherein the set of one or more base layer motion parameters are checked to determine whether the set of one or more base layer motion parameters are obtained by merging with another block of the base layer signal. ### Claim 11 A non-transitory computer-readable medium for storing data associated with a video, comprising: a data stream stored in the non-transitory computer-readable medium, the data stream comprising encoded information associated with an array of information samples representing a spatially sampled portion of the video, the encoded information including a residual signal and a syntax element, wherein the data stream is obtained based on a plurality of operations including: encoding an enhancement layer signal of a video based on block-wise predictive coding that includes: gathering, for a current block of a frame of the enhancement layer signal, a set of enhancement layer motion parameter candidates from already reconstructed neighboring blocks of the frame, the reconstructed neighboring blocks neighboring the current block, gathering a set of one or more base layer motion parameters of a block of a base layer signal, the block of the base layer signal being collocated to the current block of the frame of the enhancement layer signal, enlarging the set of enhancement layer motion parameter candidates with one or more base layer motion parameters of the set of one or more base layer motion parameters or a scaled version thereof to obtain an extended motion parameter candidate set, each of the set of enhancement layer motion parameter candidates and the one or more base layer motion parameters or a scaled version thereof in the extended motion parameter candidate set is a separate element of the extended motion parameter candidate set, wherein the extended motion parameter candidate set is ordered depending on the base layer motion parameters, selecting one of the motion parameter candidates of the extended motion parameter candidate set based on a syntax element corresponding to the current block to be signaled in a data stream, the syntax element specifically indicating the one of the motion parameter candidates of the extended motion parameter candidate set to be selected for the current block, generating, by motion compensated prediction, a prediction of the current block of the enhancement layer signal using the selected one of the motion parameter candidates of the extended motion parameter candidate set, and obtaining a residual signal for the current block based on a difference of the current block and the prediction of the current block, wherein the residual signal and the syntax element are encoded into the data stream. ### Claim 12 (depends on 11) The non-transitory computer-readable medium according to claim 11, wherein the base layer motion parameters are scaled according to a spatial resolution ratio between the base layer signal and the enhancement layer signal to acquire the scaled version of the base layer motion parameters.