US 2022/0067992 A1 · ARTIFICIAL INTELLIGENCE TECHNIQUES FOR PERFORMING IMAGE EDITING OPERATIONS INFERRED FROM NATURAL LANGUAGE REQUESTS

Abstract

This disclosure involves executing artificial intelligence models that infer image editing operations from natural language requests spoken by a user. Further, this disclosure performs the inferred image editing operations using inferred parameters for the image editing operations. Systems and methods may be provided that infer one or more image editing operations from a natural language request associated with a source image, locate areas of the source that are relevant to the one or more image editing operations to generate image masks, and performing the one or more image editing operations to generate a modified source image.

Claims20

Dependent claims

A system comprising:

one or more processors;
and
a non-transitory computer-readable medium communicatively coupled to the one or more processors and storing program code executable by the one or more processors, the program code implementing a natural language-based image editor comprising:
an operation classifier model configured to infer an image editing operation from a natural language request associated with a source image;
a grounding model configured to (a) locate an object or region of the source image that is inferred to correspond to the image editing operation and (b) generate an image mask for the object or region;
and
an operation modular network configured to generate a modified source image by performing the image editing operation.

4 more elements

A computer-implemented method comprising:

retrieving a source image and a natural language request;
inferring, using an operation classifier model, an image editing operation from the natural language request;
generating, using a grounding model, an image mask for an object or region of the source image that is inferred to correspond to the image editing operation;
performing, using a operation modular network, the image editing operation on the source image;
and
outputting a modified source image.

3 more elements

A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a processing apparatus to perform operations including:

retrieving a source image and a natural language request;
a step for editing the source image based on an image editing operation inferred from the natural language request;
and
outputting a modified source image.

1 more element

Dependencies are inferred from the claim text.

Artificial intelligence techniques for performing image editing operations inferred from natural language requests

Life of this patent

Artificial intelligence techniques for performing image editing operations inferred from natural language requests

Life of this patent

Abstract

Claims20