Generative video artifact removal
AssigneeSima Labs, Inc.
InventorsSOPHIE RANIWALA, RUSHAAN MAHAJAN
Life of this patent
Estimating the term…
Abstract
Generative AI model-created video differs fundamentally from natural video, as it is generated through statistical algorithms rather than captured through a camera-based device. Such content has spatial and temporal (multi-frame) artifacts which are unique to generative video. Described herein is a post-processing method to improve the visual quality of GenAI-produced video content by reducing said artifacts before the video encoding stage. By addressing both global and local flicker (temporal artifacts) as well as unique high frequency noise (spatial artifacts) through frequency domain analysis, the performance of video codecs on processed GenAI videos is optimized, thereby improving compression rates (cost of streaming) and Quality of Experience (QoE).
Claims7
A computer implemented method for adaptive frame stabilization and noise removal in generative artificial intelligence produced video content, the method executable by at least one processor configured to execute computer program instructions for performing the method, the method comprising the steps of:
- applying an end-to-end trained temporal neural network to said generative artificial intelligence produced video content to detect and correct flicker patterns, wherein global flicker is classified as multiplicative or additive errors, and local flicker is detected as unintentional object jitter;
- applying said temporal neural network across the generative artificial intelligence produced video content, distinguishing between true object motion and flicker artifacts, and stabilizing brightness inconsistencies while preserving motion integrity;
- ensuring frame-to-frame coherence by reconstructing each frame of the generative artificial intelligence produced video content's visual context, and correcting unintended positional variations, thereafter producing a stabilized temporally corrected output;
- passing said stabilized temporally corrected output into a spatial neural network trained to enhance color consistency, suppress high-frequency noise, and refine luminance and chrominance channels through convolutional and cross-channel operations;
- applying frequency-adaptive discrete cosine transform (DCT)-based regularization to said passed temporally corrected output, selectively filtering non-essential noise while retaining perceptual detail;
- and
- generating a fully refined video output where temporal and spatial artifacts have been minimized, ensuring optimal video quality for encoding and playback.
A system for adaptive frame stabilization and noise removal in generative artificial intelligence produced video content, comprising;
- at least one processor;
- a non-transitory computer readable storage medium communicatively coupled to said at least one processor, said non-transitory computer readable storage medium configured to store modules, said at least one processor configured to execute said modules;
- and
- said modules comprising:
- a first module for,
- applying an end-to-end trained temporal neural network to said generative artificial intelligence produced video content to detect and correct flicker patterns, wherein global flicker is classified as multiplicative or additive errors, and local flicker is detected as unintentional object jitter;
- applying said temporal neural network across the generative artificial intelligence produced video content, distinguishing between true object motion and flicker artifacts, and stabilizing brightness inconsistencies while preserving motion integrity;
- ensuring frame-to-frame coherence by reconstructing each frame of the generative artificial intelligence produced video content's visual context and correcting unintended positional variations, thereafter producing a stabilized temporally corrected output;
- a second module for,
- passing said stabilized temporally corrected output into a spatial neural network trained to enhance color consistency, suppress high-frequency noise, and refine luminance and chrominance channels through convolutional and cross-channel operations;
- a third module for,
- applying frequency-adaptive discrete cosine transform (DCT)-based regularization to said passed temporally corrected output, selectively filtering non-essential noise while retaining perceptual detail;
- and
- thereby generating a fully refined video output where temporal and spatial artifacts have been minimized, ensuring optimal video quality for encoding and playback.
A computer implemented method for pre-processing generative AI-produced video content to correct visual artifacts prior to encoding, the method executable by at least one processor configured to execute computer program instructions for performing the method, the method comprising:
- receiving raw generative video content directly from a generative artificial intelligence model;
- detecting temporal inconsistencies and spatial artifacts unique to generative video in said raw generative video using at least one neural network;
- automatically applying corrections to stabilize said temporal inconsistencies and temporal artifacts, thereby ensuring positional coherence, consistent luminance, and inter-frame stability;
- configuring said corrections to improve encoding efficiency by reducing redundant noise, unnecessary motion vectors, and high-frequency inconsistencies;
- and
- finalizing the corrected generative video content for optimized encoding, thereby guaranteeing playback fidelity and efficient compression with existing codecs.
Dependencies are inferred from the claim text.