US 12,367,586 B2 · Learning parameters for neural networks using a semantic discriminator and an object-level discriminator

Abstract

The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.

Claims20

Dependent claims

A non-transitory computer readable medium storing executable instructions which, when executed by a processing device, cause the processing device to perform operations comprising:

generating a predicted digital image from a semantic segmentation of a digital image utilizing a neural network;
generating, utilizing a semantic discriminator, a semantic image embedding from the predicted digital image and a panoptic condition combining a binary mask, a normalized semantic embedding, and an edge map;
combining the semantic image embedding with an image embedding of the predicted digital image;
generating a realism prediction, utilizing the semantic discriminator, from the combination of the semantic image embedding and the image embedding;
and
modifying parameters of the neural network based on the realism prediction.

3 more elements

A system comprising:

one or more memory devices comprising a digital image, a semantic segmentation, a neural network, an image-level semantic discriminator, and an object-level semantic discriminator;
and
one or more processors configured to cause the system to learn parameters for the neural network utilizing the image-level semantic discriminator and the object-level semantic discriminator by:
generating a predicted digital image from the digital image and the semantic segmentation utilizing the neural network;
generating, utilizing the object-level semantic discriminator, a semantic image embedding from the predicted digital image and a panoptic condition combining a binary mask, a normalized semantic embedding, and an edge map;
combining the semantic image embedding with an image embedding of the predicted digital image;
generating, utilizing the image-level semantic discriminator and the object-level semantic discriminator, a realism prediction from the combination of the semantic image embedding and the image embedding;
and
modifying parameters of the neural network based on the realism prediction.

6 more elements

Dependencies are inferred from the claim text.

Learning parameters for neural networks using a semantic discriminator and an object-level discriminator

Life of this patent

Learning parameters for neural networks using a semantic discriminator and an object-level discriminator

Life of this patent

Abstract

Claims20