# US 2021/0243465 A1 · Motion compensation for video encoding and decoding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0243465 A1 - Document ID: US20210243465A1 - Status: Published application - Kind: A1 - Application number: 17/051,825 - Filed: 2019-05-07 - Published: 2021-08-05 - Earliest claimed: 2018-05-09 - Assignees: InterDigital VC Holdings, Inc. - Applicants: InterDigital VC Holdings, Inc. - Inventors: YA Chen; Franck Galpin; Tangi Poirier - Web page: https://patentgrep.com/patent/US20210243465A1 ## CPC - H04N 19/139: Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability - H04N 19/513: Processing of motion vectors - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/149: by estimating the code amount by means of a model, e.g. mathematical model or statistical model - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 19/51: Motion estimation or motion compensation - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A video codec can involve processing video information based on a motion model involving a coding unit including a plurality of sub-blocks, such as an affine motion model, to produce motion compensation information, obtaining a local illumination compensation model, and encoding or decoding the video information based on the motion compensation information and the local illumination compensation model. ## Claims ### Claim at position 1 1-43. (canceled) ### Claim 44 A method comprising: obtaining local illumination compensation information associated with video information based on illumination changes in the video information and based on a plurality of motion vectors included in motion compensation information associated with the video information; and encoding the video information based on the local illumination compensation information and the motion compensation information; wherein the video information comprises a block including a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors comprises a group of motion vectors associated with one or more of a first subset of sub-blocks included in the first row of sub-blocks or a second subset of sub-blocks included in the first column of sub-blocks; and obtaining the local illumination compensation information is based on a set of reconstructed samples obtained based on the group of motion vectors. ### Claim 45 A method comprising: obtaining local illumination compensation information associated with video information based on illumination changes in the video information and based on a plurality of motion vectors included in motion compensation information associated with the video information; and decoding the video information based on the local illumination compensation information and the motion compensation information; wherein the video information comprises a block including a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors comprises a group of motion vectors associated with one or more of a first subset of sub-blocks included in the first row of sub-blocks or a second subset of sub-blocks included in the first column of sub-blocks; and obtaining the local illumination compensation information is based on a set of reconstructed samples obtained based on the group of motion vectors. ### Claim 46 Apparatus comprising: one or more processors configured to obtain local illumination compensation information associated with video information based on illumination changes in the video information and based on a plurality of motion vectors included in motion compensation information associated with the video information; and encode the video information based on the local illumination compensation information and the motion compensation information; wherein the video information comprises a block including a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors comprises a group of motion vectors associated with one or more of a first subset of sub-blocks included in the first row of sub-blocks or a second subset of sub-blocks included in the first column of sub-blocks; and the local illumination compensation information is obtained based on a set of reconstructed samples obtained based on the group of motion vectors. ### Claim 47 Apparatus comprising: one or more processors configured to obtain local illumination compensation information associated with video information based on illumination changes in the video information and based on a plurality of motion vectors included in motion compensation information associated with the video information; and decode the video information based on the local illumination compensation information and the motion compensation information; wherein the video information comprises a block including a first row of sub-blocks and a first column of sub-blocks, and the plurality of motion vectors comprises a group of motion vectors associated with one or more of a first subset of sub-blocks included in the first row of sub-blocks or a second subset of sub-blocks included in the first column of sub-blocks; and the local illumination compensation information is obtained based on a set of reconstructed samples obtained based on the group of motion vectors. ### Claim 48 (depends on 44) The method of claim 44, wherein the motion compensation information is obtained based on an affine motion model. ### Claim 49 (depends on 45) The method of claim 45, wherein the motion compensation information is obtained based on an affine motion model. ### Claim 50 (depends on 46) The apparatus of claim 46, wherein the motion compensation information is obtained based on an affine motion model. ### Claim 51 (depends on 47) The apparatus of claim 47, wherein the motion compensation information is obtained based on an affine motion model. ### Claim 52 (depends on 48) The method of claim 48, wherein obtaining the local illumination compensation information comprises determining at least one model parameter of a linear model of illumination changes in the video information based on the group of motion vectors. ### Claim 53 (depends on 49) The method of claim 49, wherein obtaining the local illumination compensation information comprises determining at least one model parameter of a linear model of illumination changes in the video information based on the group of motion vectors. ### Claim 54 (depends on 50) The apparatus of claim 50, wherein obtaining the local illumination compensation information comprises determining at least one model parameter of a linear model of illumination changes in the video information based on the group of motion vectors. ### Claim 55 (depends on 51) The apparatus of claim 51, wherein obtaining the local illumination compensation information comprises determining at least one model parameter of a linear model of illumination changes in the video information based on the group of motion vectors. ### Claim 56 (depends on 52) The method of claim 52, wherein the plurality of sub-blocks of the block of video information are partitioned into a plurality of groups of sub-blocks; and the at least one model parameter is determined for each of the plurality of groups of sub-blocks; and encoding or decoding based on the local illumination compensation information comprises processing video information associated with each group of sub-blocks using the respective at least one model parameter determined for each group. ### Claim 57 (depends on 53) The method of claim 53, wherein the plurality of sub-blocks of the block of video information are partitioned into a plurality of groups of sub-blocks; and the at least one model parameter is determined for each of the plurality of groups of sub-blocks; and encoding or decoding based on the local illumination compensation information comprises processing video information associated with each group of sub-blocks using the respective at least one model parameter determined for each group. ### Claim 58 (depends on 54) The apparatus of claim 54, wherein the plurality of sub-blocks of the block of video information are partitioned into a plurality of groups of sub-blocks; and the at least one model parameter is determined for each of the plurality of groups of sub-blocks; and encoding or decoding based on the local illumination compensation information comprises processing video information associated with each group of sub-blocks using the respective at least one model parameter determined for each group. ### Claim 59 (depends on 55) The apparatus of claim 55, wherein the plurality of sub-blocks of the block of video information are partitioned into a plurality of groups of sub-blocks; and the at least one model parameter is determined for each of the plurality of groups of sub-blocks; and encoding or decoding based on the local illumination compensation information comprises processing video information associated with each group of sub-blocks using the respective at least one model parameter determined for each group. ### Claim 60 (depends on 56) The method of claim 56, wherein the affine motion model comprises an affine merge motion model including determining a local illumination compensation flag based on at least one affine control point associated with a second block of video information neighboring the block of video information. ### Claim 61 (depends on 57) The method of claim 57, wherein the affine motion model comprises an affine merge motion model including determining a local illumination compensation flag based on at least one affine control point associated with a second block of video information neighboring the block of video information. ### Claim 62 (depends on 58) The apparatus of claim 58, wherein the affine motion model comprises an affine merge motion model including determining a local illumination compensation flag based on at least one affine control point associated with a second block of video information neighboring the block of video information. ### Claim 63 (depends on 59) The apparatus of claim 59, wherein the affine motion model comprises an affine merge motion model including determining a local illumination compensation flag based on at least one affine control point associated with a second block of video information neighboring the block of video information. ### Claim 64 (depends on 45) A non-transitory computer readable medium storing executable program instructions to cause a computer executing the instructions to perform a method according to claim 45. ### Claim 65 (depends on 47) A device comprising: an apparatus according to claim 47; and at least one of (i) an antenna configured to receive a signal, the signal including data representative of the video information, (ii) a band limiter configured to limit the received signal to a band of frequencies that includes the data representative of the video information, and (iii) a display configured to display an image from the video information. ### Claim 66 (depends on 63) The device of claim 63, wherein the device comprises one of a television, a television signal receiver, a set-top box, a gateway device, a mobile device, a cell phone, a tablet, a computer such as a laptop computer or a desktop computer, a server, or other electronic device.