Video characterization for smart encoding based on perceptual quality optimization

A computer system for estimating bitrates in adaptive bitrate (ABR) applications that generate multiple encodings of the same video at different frame sizes and bitrates and that determine which encoding to transmit based on network conditions, the computer system comprising:

• a processor;
• and
• a memory with computer code instructions stored thereon, the memory operatively coupled to the processor such that, when executed by the processor, the computer code instructions cause the system to:
• estimate an encoding bitrate at a first frame size by iteratively predicting a target bitrate at which an encoder encoding a source video will produce a desired mean opinion score (MOS), the estimating comprising:
• generating an initial encoding of a source video at an encoding bitrate and a decoding of the initial encoding;
• computing a first metric and a second metric on the initial encoding, the first metric being computed based on a video-average gradient magnitude similarity deviation (GMSD) and the second metric being computed based on a log-normalized mean GMSD;
• applying a previously-derived model that predicts a mean opinion score (MOS) value for the initial encoding as a function of measurements of the computed first metric and second metric;
• updating the encoding bitrate to encode and decode the source video based on a relationship between the predicted MOS value and the desired MOS value;
• and
• iterating the estimating until the predicted MOS value is close to the desired MOS value, at which point the encoding bitrate converges to a predicted target bitrate at the first frame size and an encoding at the first frame size is determined;
• determine a bitrate estimate at a second frame size and an initial encoding at the second frame size using the predicted target bitrate at the first frame size and a fixed scaling factor;
• compute a dynamic scaling factor using the encoding at the first frame size and the initial encoding at the second frame size in a dynamic scaling process, the dynamic scaling process adjusting the scaling factor iteratively to determine a final scaling factor, the dynamic scaling process adjusting the scaling factor iteratively until average P-frame quantization parameter (QP) values of the encoding at the first frame size and the adjusted encoding at the second frame size converge;
• and
• compute a final bitrate estimate at the second frame size and a final encoding at the second frame size using the final scaling factor produced by the dynamic scaling algorithm.

A computer program product for estimating bitrates in adaptive bitrate (ABR) applications that generate multiple encodings of the same video at different frame sizes and bitrates and that determine which encoding to transmit based on network conditions, the computer program product comprising:

• one or more non-transitory computer-readable storage device with program instructions stored thereon, the program instructions, when loaded and executed by a processor, cause an apparatus associated with the processor to:
• estimate an encoding bitrate at a first frame size by iteratively predicting a target bitrate at which an encoder encoding a source video will produce a desired mean opinion score (MOS), the estimating comprising:
• generating an initial encoding of a source video at an encoding bitrate and a decoding of the initial encoding;
• computing a first metric and a second metric on the initial encoding, the first metric being computed based on a video-average gradient magnitude similarity deviation (GMSD) and the second metric being computed based on a log-normalized mean GMSD;
• applying a previously-derived model that predicts a mean opinion score (MOS) value for the initial encoding as a function of measurements of the computed first metric and second metric;
• updating the encoding bitrate to encode and decode the source video based on a relationship between the predicted MOS value and the desired MOS value;
• and
• iterating the estimating until the predicted MOS value is close to the desired MOS value, at which point the encoding bitrate converges to a predicted target bitrate at the first frame size and an encoding at the first frame size is determined;
• determine a bitrate estimate at a second frame size and an initial encoding at the second frame size using the predicted target bitrate at the first frame size and a fixed scaling factor;
• compute a dynamic scaling factor using the encoding at the first frame size and the initial encoding at the second frame size in a dynamic scaling process, the dynamic scaling process adjusting the scaling factor iteratively to determine a final scaling factor, the dynamic scaling process adjusting the scaling factor iteratively until average P-frame quantization parameter (QP) values of the encoding at the first frame size and the adjusted encoding at the second frame size converge;
• and
• compute a final bitrate estimate at the second frame size and a final encoding at the second frame size using the final scaling factor produced by the dynamic scaling algorithm.