# US 2021/0243450 A1 · Spatial block-level pixel activity extraction optimization leveraging motion vectors > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0243450 A1 - Document ID: US20210243450A1 - Status: Published application - Kind: A1 - Application number: 17/236,910 - Filed: 2021-04-21 - Published: 2021-08-05 - Earliest claimed: 2018-09-28 - Assignees: ATI Technologies ULC - Applicants: ATI Technologies ULC - Inventors: Mehdi Saeedi; Boris Ivanovic - Web page: https://patentgrep.com/patent/US20210243450A1 ## CPC - H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference - H04N 19/14: Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence) - H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147) - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/513: Processing of motion vectors - H04N 19/521: {for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems, apparatuses, and methods for implementing spatial block-level pixel activity extraction optimization leveraging motion vectors are disclosed. Control logic coupled to an encoder generates block-level pixel activity metrics for a new frame based on the previously calculated block-level pixel activity data from a reference frame. A cost is calculated for each block of a new frame with respect to a corresponding block of the reference frame. If the cost is less than a first threshold, then the control logic generates an estimate of a pixel activity metric for the block which is equal to a previously calculated pixel activity metric for a corresponding block of the reference frame. If the cost is greater than the first threshold but less than a second threshold, an estimate of the pixel activity metric is generated by extrapolating from the previously calculated pixel activity metric. ## Claims ### Claim 1 A system comprising: an interface configured to receive a new video frame of a video stream; control logic coupled to the interface, wherein the control logic is configured to generate estimates of block-level pixel activity for the new video frame based on: motion estimation data for the new video frame; and previously calculated block-level pixel activity from a reference video frame; an encoder configured to generate an encoded video frame based on the estimates, wherein the encoded video frame represents the new video frame. ### Claim 2 (depends on 1) The system as recited in claim 1, wherein for each block of the new video frame, the control logic is configured to: compare, to a first threshold, a cost of the block of the new video frame with respect to a corresponding block of the reference video frame; and generate an estimate of pixel activity for the block, wherein the estimate is equal to previously calculated pixel activity of a corresponding block in the reference video frame if the cost is less than the first threshold. ### Claim 3 (depends on 2) The system as recited in claim 2, wherein for each block of the new video frame, the control logic is further configured to: compare the cost of the block to a second threshold; responsive to determining that the cost is greater than or equal to the first threshold and less than the second threshold: map the cost to a correction factor using a transfer function; and generate an estimate of the pixel activity for the block by applying the correction factor to the previously calculated pixel activity of the corresponding block in the reference video frame. ### Claim 4 (depends on 3) The system as recited in claim 3, wherein the control logic is further configured to calculate the pixel activity for the block independently of the previously calculated pixel activity if the cost is greater than or equal to the second threshold. ### Claim 5 (depends on 1) The system as recited in claim 1, wherein the control logic is configured to classify the new video frame based on the estimates, and wherein the block-level pixel activity represents gradients, co-occurrence matrices, or any other metric where results of a mathematical operation on a first pixel value and a second pixel value at a defined relative displacement to the first pixel value are aggregated on blocks of the new video frame. ### Claim 6 (depends on 1) The system as recited in claim 1, wherein the previously calculated block-level pixel activity is at a first granularity, and wherein the encoder is configured to generate estimates for block-level pixel activity for the new video frame at a second granularity, wherein the first granularity is a finer granularity than the second granularity. ### Claim 7 (depends on 6) The system as recited in claim 6, wherein the encoder is configured to generate cumulative estimates for the block-level pixel activity for each block of the new video frame by summing pixel activities for a plurality of blocks from the reference video frame. ### Claim 8 A method comprising: receiving, by a server, a new video frame of a video stream; and generating, by control logic, estimates of block-level pixel activity for a new video frame of the video stream based on: motion estimation data for the new video frame; and previously calculated block-level pixel activity from a reference video frame; and generating, by an encoder, an encoded video frame based on the estimates, wherein the encoded video frame represents the new video frame. ### Claim 9 (depends on 8) The method as recited in claim 8, further comprising: comparing, to a first threshold, a cost of a block of the new video frame with respect to a corresponding block of the reference video frame; and generating an estimate of pixel activity for the block, wherein the estimate is equal to previously calculated pixel activity of a corresponding block in the reference video frame if the cost is less than the first threshold. ### Claim 10 (depends on 9) The method as recited in claim 9, further comprising: comparing the cost of the block to a second threshold; responsive to determining that the cost is greater than or equal to the first threshold and less than the second threshold: mapping the cost to a correction factor using a transfer function; and generating an estimate of the pixel activity for the block by applying the correction factor to the previously calculated pixel activity of the corresponding block in the reference video frame. ### Claim 11 (depends on 10) The method as recited in claim 10, further comprising calculating the pixel activity for the block independently of the previously calculated pixel activity if the cost is greater than or equal to the second threshold. ### Claim 12 (depends on 8) The method as recited in claim 8, further comprising classifying the new video frame based on the estimates, wherein the block-level pixel activity represents gradients, co-occurrence matrices, or any other metric where results of a mathematical operation on a first pixel value and a second pixel value at a defined relative displacement to the first pixel value are aggregated on blocks of the new video frame. ### Claim 13 (depends on 8) The method as recited in claim 8, wherein the previously calculated block-level pixel activity is at a first granularity, and wherein the method further comprising generating estimates for block-level pixel activity for the new video frame at a second granularity, wherein the first granularity is a finer granularity than the second granularity. ### Claim 14 (depends on 13) The method as recited in claim 13, further comprising generating cumulative estimates for the block-level pixel activity for each block of the new video frame by summing pixel activities for a plurality of blocks from the reference video frame. ### Claim 15 An apparatus comprising: a memory; an encoder coupled to the memory; and control logic coupled to the encoder, wherein the control logic is configured to generate estimates of block-level pixel activity for a new video frame based on: motion estimation data for the new video frame; and previously calculated block-level pixel activity from a reference video frame stored in the memory; wherein the encoder is configured to generate an encoded video frame based on the estimates, wherein the encoded video frame represents the new video frame. ### Claim 16 (depends on 15) The apparatus as recited in claim 15, wherein for each block of the new video frame, the control logic is configured to: compare, to a first threshold, a cost of the block of the new video frame with respect to a corresponding block of the reference video frame; and generate an estimate of pixel activity for the block, wherein the estimate is equal to previously calculated pixel activity of a corresponding block in the reference video frame if the cost is less than the first threshold. ### Claim 17 (depends on 16) The apparatus as recited in claim 16, wherein for each block of the new video frame, the control logic is further configured to: compare the cost of the block to a second threshold; responsive to determining that the cost is greater than or equal to the first threshold and less than the second threshold: map the cost to a correction factor using a transfer function; and generate an estimate of the pixel activity for the block by applying the correction factor to the previously calculated pixel activity of the corresponding block in the reference video frame. ### Claim 18 (depends on 17) The apparatus as recited in claim 17, wherein the control logic is further configured to calculate the pixel activity for the block independently of the previously calculated pixel activity if the cost is greater than or equal to the second threshold. ### Claim 19 (depends on 15) The apparatus as recited in claim 15, wherein the control logic is configured to classify the new video frame based on the estimates, and wherein the block-level pixel activity represents gradients, co-occurrence matrices, or any other metric where results of a mathematical operation on a first pixel value and a second pixel value at a defined relative displacement to the first pixel value are aggregated on blocks of the new video frame. ### Claim 20 (depends on 15) The apparatus as recited in claim 15, wherein the previously calculated block-level pixel activity is at a first granularity, and wherein encoder is configured to generate estimates for block-level pixel activity for the new video frame at a second granularity, wherein the first granularity is a finer granularity than the second granularity.