# US 9,883,190 B2 · Video encoding using variance for selecting an encoding mode > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,883,190 B2 - Document ID: US9883190B2 - Status: Granted - Kind: B2 - Application number: 14/935,513 - Filed: 2015-11-09 - Published: 2016-03-03 - Granted: 2018-01-30 - Earliest claimed: 2012-06-29 - Assignees: Google Inc. - Applicants: Google Inc. - Inventors: Markus Tapio Suvanto - Web page: https://patentgrep.com/patent/US9883190B2 ## CPC - H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction - H04N 19/10: using adaptive coding - H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh - H04N 19/139: Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability - H04N 19/14: Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence) - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/186: the unit being a colour or a chrominance component - H04N 19/86: involving reduction of coding artifacts, e.g. of blockiness - H04N 19/196: being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters (processing of motion vectors H04N19/513) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Video encoding using variance may include generating a first input variance based on pixel values from a first block from a first frame from an input video stream, generating a first encoded block by encoding the first block, generating a first reconstructed block by reconstructing the first encoded block, generating a first reconstruction variance based on pixel values of the first reconstructed block, comparing the first input variance and the first reconstruction variance to detect a visual artifact in the first reconstructed block, generating a second encoded block by encoding a second block from a second frame from the input video stream, wherein encoding the second block includes selecting an encoding mode based on the detection of the visual artifact in the first reconstructed block, including the second encoded block in an output bitstream, and storing or transmitting the output bitstream. ## Claims ### Claim 1 A method for encoding a video stream comprising: generating a first input variance based on pixel values from a first block from a first frame from an input video stream; generating a first encoded block by encoding the first block; generating a first reconstructed block by reconstructing the first encoded block; generating a first reconstruction variance based on pixel values of the first reconstructed block; comparing the first input variance and the first reconstruction variance to detect a visual artifact in the first reconstructed block; generating, by a processor in response to instructions stored on a non-transitory computer readable medium, a second encoded block by encoding a second block from a second frame from the input video stream, wherein encoding the second block includes selecting an encoding mode based on the detection of the visual artifact in the first reconstructed block; including the second encoded block in an output bitstream; and storing or transmitting the output bitstream. ### Claim 2 (depends on 1) The method of claim 1, wherein the encoding mode is an intra-prediction mode or an inter-prediction mode. ### Claim 3 (depends on 2) The method of claim 2, wherein the selecting the encoding mode is biased towards selection of the intra-prediction mode on a condition that the comparison of the first input variance and the first reconstruction variance indicates that one or more artifacts are detected in the reconstructed first block. ### Claim 4 (depends on 2) The method of claim 2, wherein encoding the second block includes: determining a first score associated with encoding the second block using the inter-prediction mode; determining a second score associated with encoding the second block using the intra-prediction mode; determining a penalty value based on the comparison; selecting one of the intra-prediction mode or the inter-prediction mode based on the first score, the second score, and the penalty value; and encoding the second block using the selected mode. ### Claim 5 (depends on 4) The method of claim 4, wherein the first score is determined based on a sum of absolute differences between the second block and prediction samples selected using the inter-prediction mode and the second score is determined based on a sum of absolute differences between the second block and prediction samples selected using the intra-prediction mode. ### Claim 6 (depends on 2) The method of claim 2, wherein comparing the first input variance and the first reconstruction variance includes determining a condition that the first input variance is less than the first reconstruction variance scaled by a first defined factor and encoding the second block includes biasing the selection of the encoding mode towards selection of the intra-prediction mode using a second defined factor in response to determining the condition. ### Claim 7 (depends on 6) The method of claim 6, wherein the condition is a first condition, the method further comprising: generating a second input variance based on pixel values from the second block; comparing the second input variance and the first reconstruction variance, wherein comparing the second input variance and the first reconstruction variance includes determining a second condition that the second input variance biased by a third defined factor is less than the first reconstruction variance scaled by the first defined factor; and wherein biasing the selection of the encoding mode is performed in response to both determining the first condition and determining the second condition. ### Claim 8 (depends on 6) The method of claim 6, wherein the condition is a first condition and the method further comprises: generating a second input variance based on pixel values from the second block; comparing the second input variance and a third defined factor including determining a second condition that the second input variance is less than the third defined factor; and wherein biasing the selection of the encoding mode is performed in response to both determining the first condition and determining the second condition. ### Claim 9 (depends on 6) The method of claim 6, wherein biasing the selection of the encoding mode is conditioned on a determination that the first block was encoded using an inter-prediction mode. ### Claim 10 (depends on 1) The method of claim 1, wherein the pixel values from the first block and the first reconstructed block respectively include a plurality of luminance values and a plurality of chrominance values, generating a first input variance includes generating an input variance for the luminance values from the first block and an input variance for the chrominance values from the first block, generating the first reconstruction variance includes generating a reconstruction variance for the luminance values from the reconstructed first block and a reconstruction variance for the chrominance values from the reconstructed first block, and the comparison includes separately comparing the variances of the luminance values and the variances of the chrominance values. ### Claim 11 (depends on 1) The method of claim 1, wherein the first block and the second block are located at spatially corresponding locations in the first frame and the second frame respectively. ### Claim 12 A method for encoding a video stream comprising: generating a first input variance based on pixel values from a first block from a first frame from an input video stream; generating a first encoded block by encoding the first block; generating a first reconstructed block by reconstructing the first encoded block; generating a first reconstruction variance based on pixel values of the first reconstructed block; comparing the first input variance and the first reconstruction variance to detect a visual artifact in the first reconstructed block; generating, by a processor in response to instructions stored on a non-transitory computer readable medium, a second encoded block by encoding a second block from a second frame from the input video stream, wherein encoding the second block includes selecting an encoding mode based on the detection of the visual artifact in the first reconstructed block, wherein the encoding mode is an intra-prediction mode or an inter-prediction mode; including the second encoded block in an output bitstream; and storing or transmitting the output bitstream. ### Claim 13 (depends on 12) The method of claim 12, wherein the selecting the encoding mode is biased towards selection of the intra-prediction mode on a condition that the comparison of the first input variance and the first reconstruction variance indicates that one or more artifacts are detected in the reconstructed first block. ### Claim 14 (depends on 12) The method of claim 12, wherein encoding the second block includes: determining a first score associated with encoding the second block using the inter-prediction mode; determining a second score associated with encoding the second block using the intra-prediction mode; determining a penalty value based on the comparison; selecting one of the intra-prediction mode or the inter-prediction mode based on the first score, the second score, and the penalty value; and encoding the second block using the selected mode. ### Claim 15 (depends on 14) The method of claim 14, wherein the first score is determined based on a sum of absolute differences between the second block and prediction samples selected using the inter-prediction mode and the second score is determined based on a sum of absolute differences between the second block and prediction samples selected using the intra-prediction mode. ### Claim 16 (depends on 12) The method of claim 12, wherein comparing the first input variance and the first reconstruction variance includes determining a condition that the first input variance is less than the first reconstruction variance scaled by a first defined factor and encoding the second block includes biasing the selection of the encoding mode towards selection of the intra-prediction mode using a second defined factor in response to determining the condition. ### Claim 17 (depends on 12) The method of claim 12, wherein the pixel values from the first block and the first reconstructed block respectively include a plurality of luminance values and a plurality of chrominance values, generating a first input variance includes generating an input variance for the luminance values from the first block and an input variance for the chrominance values from the first block, generating the first reconstruction variance includes generating a reconstruction variance for the luminance values from the reconstructed first block and a reconstruction variance for the chrominance values from the reconstructed first block, and the comparison includes separately comparing the variances of the luminance values and the variances of the chrominance values. ### Claim 18 (depends on 12) The method of claim 12, wherein the first block and the second block are located at spatially corresponding locations in the first frame and the second frame respectively. ### Claim 19 A method for encoding a video stream comprising: generating a first input variance based on pixel values from a first block from a first frame from an input video stream; generating a first encoded block by encoding the first block; generating a first reconstructed block by reconstructing the first encoded block; generating a first reconstruction variance based on pixel values of the first reconstructed block; comparing the first input variance and the first reconstruction variance to detect a visual artifact in the first reconstructed block; generating, by a processor in response to instructions stored on a non-transitory computer readable medium, a second encoded block by encoding a second block from a second frame from the input video stream, wherein encoding the second block includes selecting an intra-prediction mode or an inter-prediction mode based on the detection of the visual artifact in the first reconstructed block; including the second encoded block in an output bitstream; and storing or transmitting the output bitstream. ### Claim 20 (depends on 19) The method of claim 19, wherein encoding the second block includes: determining a first score associated with encoding the second block using the inter-prediction mode; determining a second score associated with encoding the second block using the intra-prediction mode; determining a penalty value based on the comparison; selecting one of the intra-prediction mode or the inter-prediction mode based on the first score, the second score, and the penalty value; and encoding the second block using the selected mode.