Context preservation for synthetic image augmentation using diffusion

Abstract

Approaches presented herein are directed to generating synthetic images with one or more augmentations realistically added to objects in the images, while ensuring the preservation and integrity of semantic or contextual information within the image. A synthetic augmentation system may identify and extract foreground image data (e.g., text), and a version of the image with the foreground image data removed can be processed by a generative diffusion model. One or more inputs can be provided to specify aspects such as a type or strength of augmentation to be performed. After an augmented image is generated using the generative diffusion model, the previously removed text can be blended back into the image. A synthetic augmentation system may use one or more blending weights for the text, such as a defect blending weight and a letter blending weight. The final result is a synthetic image with added realistic augmentations that preserves the semantic content.

Dependencies are inferred from the claim text.