Depth-Aware Photo Editing

A computer-implemented method comprising:determining, by a computing device and based on segmentation data for an image, an object in the image;determining, by the computing device and based on a depth map for the image, depth information for the object;detecting, via a user interaction with a graphical user interface, an indication to insert a virtual object at a location in a vicinity of the object;determining a location and a pose for the virtual object in a three-dimensional (3D) coordinate system comprising a two-dimensional (2D) coordinate system of the image and the depth map;

• andinserting, by the computing device, the virtual object at the location in the vicinity of the object, wherein the inserting comprises:partially occluding the virtual object by the object based on the depth information, andmasking a portion of the virtual object that overlaps with the object such that the virtual object appears to be behind the object.

A computer-implemented method comprising:receiving, at a computing device, image data for an image;determining, by the computing device and based on segmentation data for the image, an object in the image;determining, by the computing device and based on a depth map for the image, depth information for a surface of the object;determining, by the computing device and based on the image data, a location and a three-dimensional (3D) pose for a virtual object, wherein the determining of the location and 3D pose is based at least in part on identifying, from the segmentation data and the depth information, a real-world surface within the image data where insertion of the virtual object is visually possible;

• andinserting, by the computing device and based on the determined location and 3D pose, the virtual object into the image.

A computer-implemented method comprising:receiving, by a computing device, image data for an image;determining, based on segmentation data for the image, an object in the image;determining, based on a depth map for the image, depth information for the object;automatically scanning, by the computing device, the image to identify a portion of the image for insertion of a virtual object, wherein the identifying of the portion is based on analyzing segmentation masks for objects in the image and at least one background area outside the segmentation masks for objects to find an area where the virtual object can fit;automatically determining, by the computing device, a location and a pose for the virtual object in a three-dimensional (3D) coordinate system comprising a 2D coordinate system of the image and the depth map;rendering a version of the virtual object based on the determined location and pose;

• andinserting the rendered version of the virtual object at the identified portion of the image.