Applying minimum and average distance constraint in video streaming
AssigneeDolby Laboratories Licensing Corporation
InventorsHarshad KADU, Guan-Ming SU
Life of this patent
Estimating the term…
Abstract
Input images are received as input to a multi-node system. The input images are divided into segments assigned to respective nodes of the multi-node system. Primary and secondary scenes are identified in the segments to ensure compliance with minimum and average distance constraints. Scene-level forward reshaping mappings are generated for the scenes by a respective node for an assigned segment. Forward reshaped images in the segment are generated by the node using the forward reshaping mappings and encoded into an output video signal, which enables a recipient device to generate reconstructed images and to render display images derived from the reconstructed images on an image display.
Claims11
A method comprising:
- receiving a sequence of input images in an input video signal and one or more input lists of scenes as input to a multi-node video encoding system having a plurality of video encoding nodes, the one or more input lists of scenes collectively identifying a plurality of primary scenes in the sequence of input images, wherein each of the plurality of primary scenes identified by the one or more input lists of scenes is generated by (a) merging manually identified scene cuts in an external list with automatically detected scene cuts by an automatic scene detector into combined scene cuts of neighboring scenes or by (b) merging neighboring scenes represented in the combined scene cuts into longer primary scenes, wherein neighboring scenes are merged until the length of the combined scene exceeds a threshold value representing a predetermined average distance to the next neighboring scene such that each combined scene fulfills an average distance constraint;
- dividing the sequence of input images into a plurality of non-overlapping segments, each segment in the plurality of non-overlapping segments being assigned to a corresponding node in the plurality of video encoding nodes, the corresponding node receiving input images within the respective segment as in-segment images for the segment and input images in one or two bumper sections of the respective segment, wherein each bumper section comprises a number of images that precede before the first image of the respective segment or succeed after the last image of the respective segment;
- generating a second list of scenes from the plurality of primary scenes in the sequence of input images, the second list of scenes identifying one or more of the plurality of primary scenes as in-segment scenes in the respective segment, the one or more in-segment scenes including zero or more primary scenes not on any segment boundary of the respective segment and one or more secondary scenes divided from one or more primary scenes on one or both segment boundaries of the respective segment, the secondary scenes comprising the images of the bumper section on the respective segment boundary, each secondary scene, exclusive of its bumper sections, having a length being greater than or equal to a predetermined minimum distance to the next neighboring scene, thereby each secondary scene fulfilling a minimum distance constraint, the predetermined minimum distance being smaller than the predetermined average distance, and the average length of the scenes in the second list of scenes, exclusive of their bumper sections, being greater than or equal to the predetermined average distance to the next neighboring scene, thereby the scenes in the second list of scenes fulfilling the average distance constraint;
- generating a scene-level forward reshaping mapping for each scene of the second list of scenes in the respective segment, the scene-level forward reshaping mapping capable of mapping input images to corresponding forward reshaped images having a Standard Dynamic Range—SDR—that is smaller than a High Dynamic Range—HDR—of the input images, each scene-level forward reshaping mapping being generated at least in part from reshaping statistics collected from input images in an overlap window determined for a respective scene of the second list of scenes in the respective segment inclusive of its bumper sections, the reshaping statistics comprising statistics from (i) luma codewords in an SDR version of each input image in the overlap window aggregated to a scene-level SDR histogram and (ii) luma codewords in each input image in the overlap window aggregated to a scene-level HDR histogram;
- applying, by the respective node, the generated scene-level forward reshaping mappings to the input images of the scenes of the second list of scenes for the respective segment exclusive of its bumper sections to generate forward reshaped images of the respective segment;
- encoding, by the respective node, the forward reshaped images in the respective segment into an output video signal encoded collectively by the plurality of video encoding nodes, the output video signal including an image metadata portion generated by the respective node to enable a recipient device to generate reconstructed images from the forward reshaped images in the respective segment and to render display images derived from the reconstructed images on an image display, wherein the same image metadata portion is used for all images of an encoded scene such that the image metadata portion fulfills the minimum distance constraint and the average distance constraint.
Dependencies are inferred from the claim text.