US 11,908,109 B1 · Enhanced video processor

Abstract

Various improvements to video processing are described, including highlight recovery by dividing an image into different frequency ranges and compressing the lower range before recombining, tone mapping to avoid hue shifts, dynamic tone mapping based on detected scene changes in the video, dynamic tone mapping using the shape of a tone mapping curve that is changed based on the histogram of each video frame, HDR Luminance Channel Repair of artifacts, changing the shape of the tone mapping curve based on a histogram of each video frame, upscaling Chroma by using Luma channel information motion interpolation using Neural Networks, motion compensated noise reduction using Neural Networks to filter random noise and film grain, grain/noise agnostic upscaling using Neural Networks, and hiding video frames by strategically dropping or repeating them in unnoticeable places to prevent a visible video stutter based on scene characteristics to accommodate clock variations.

Claims18

Dependent claims

A computer implemented method comprising:

receiving a video sequence having image frames containing pixels with a high dynamic range;
separating each image frame into low and high frequency components;
compressing the low frequency components using a non-linear brightness compression curve including tone mapping the low frequency components;
reducing brightness and saturation for pixels of the low frequency components which are outside a target gamut;
reconstructing the image frames of the video sequence by combining the high frequency components with the compressed and reduced brightness and saturation for pixels of the low frequency components;
and
rendering the reconstructed image frames of the video sequence during real time playback for display on a device having a lower dynamic range, while preserving highlight detail, wherein tone mapping the low frequency components comprises:
determining a tone mapping for each successive image frame;
smoothing the tone mapping to reduce visible flickering artifacts;
detecting a scene change;
and
discontinuing smoothing the tone mapping for a first image frame of the detected scene change.

9 more elements

A device comprising:

a processor;
and
a memory device coupled to the processor and having a program stored thereon for execution by the processor to perform operations comprising:
receiving a video sequence having image frames with a high dynamic range;
separating each image frame into low and high frequency components;
compressing the low frequency components using a non-linear brightness compression curve including tone mapping the low frequency components;
reducing brightness and saturation for pixels of the low frequency components which are outside a target gamut;
reconstructing the image frames of the video sequence by combining the high frequency components with the compressed and reduced brightness and saturation for pixels of the low frequency components;
and
rendering the reconstructed image frames of the video sequence during real time playback for display on a device having a lower dynamic range, while preserving highlight detail, wherein tone mapping the low frequency components comprises:
determining a tone mapping for each successive image frame;
smoothing the tone mapping to reduce visible flickering artifacts;
detecting a scene change;
and
discontinuing smoothing the tone mapping for a first image frame of the detected scene change.

12 more elements

Dependencies are inferred from the claim text.

Enhanced video processor

Life of this patent

Enhanced video processor

Life of this patent

Abstract

Claims18