Removing and Replacing Objects in Images According to a Directed User Conversation

A system implemented by a computing device in a digital medium environment, the system including modules implemented at least partially in hardware of the computing device, the system comprising:

• a conversation module to direct a user conversation that includes an editing query for an image to be edited, the user conversation comprising multiple volleys of questions and responses and having a scope that is narrowed with each volley of the multiple volleys;
• a vision module to ascertain pixels of the image corresponding to an object to be removed or replaced indicated in the user conversation;
• a language module to make a determination whether the editing query includes a remove request or a replace request;
• responsive to the determination indicating the editing query includes a remove request:
• an image search module to obtain fill material recognized as similar to different pixels of the image than the pixels of the image;
• and
• a compositing module to:
• removing content from the pixels of the image;
• and
• add the fill material to the pixels of the image to form one or more composite images;
• responsive to the determination indicating the editing query includes a replace request:
• the image search module to obtain replacement material recognized as similar to the pixels of the image and corresponding to the replace request;
• and
• the compositing module to replace content of the pixels of the image with the replacement material to form the one or more composite images;
• a harmonizing module to harmonize the one or more composite images to form one or more harmonized images;
• and
• a display module to expose the one or more harmonized images in a user interface of the computing device.

In a digital medium environment to edit images, a method implemented by a computing device, the method comprising:

• a step for directing a user conversation that includes an editing query for an image to be edited and indicates an object to be removed or replaced, the user conversation including a question that repeats the object to implicitly confirm the object without explicitly asking for correction of the object;
• a step for ascertaining pixels of the image corresponding to the object;
• a step for displaying the image with an indication of a candidate object;
• a step for confirming that the candidate object matches the object;
• a step for determining whether the editing query includes a remove request or a replace request;
• responsive to the determining indicating the editing query includes a remove request:
• a step for removing content from the pixels of the image;
• and
• a step for adding fill material recognized as similar to different pixels of the image than the pixels of the image to the pixels of the image to form one or more composite images;
• responsive to the determining indicating the editing query includes a replace request, a step for replacing content of the pixels of the image with replacement material recognized as similar to the pixels of the image and corresponding to the replace request to form the one or more composite images;
• a step for harmonizing the one or more composite images to form one or more harmonized images by adjusting the replacement material or the fill material according to the different pixels;
• and
• a step for exposing the one or more harmonized images in a user interface of the computing device.