Automatic background replacement for single-image and multi-view captures

A mobile computing device comprising:

• a processor configured to determine a segmentation of an object depicted in a first visual representation that includes one or more images of the object captured via one or more cameras at a mobile computing device, the object being positioned within a three-dimensional environment, the object not being positioned in front of a two-dimensional patterned or monochromatic surface, the segmentation including for each image a first respective image portion that includes the object, the segmentation further including for each image a second respective image portion that includes a respective ground area located beneath the object, the segmentation further including a third respective image portion that includes a background area located above the second respective portion and behind the object, wherein the processor is further configured to select a target background image based on a match between a first one or more ground plane vectors estimated for the target background image and a second one or more ground plane vectors estimated for the first visual representation;
• a memory module configured to store a second visual representation based on the first visual representation via the processor at the mobile computing device, the second visual representation including the first respective image portion, the second visual representation including for each image a respective target background image portion that replaces the third respective image portion, the respective target background image portion being selected from one of the selected target background images based on an area of the third respective image portion relative to the respective image;
• and
• a storage device configured to store the second visual representation on.

One or more non-transitory computer readable media having instructions stored thereon for performing a method, the method comprising:

• determining via a processor at a mobile computing device a segmentation of an object depicted in a first visual representation that includes one or more images of the object captured via one or more cameras at a mobile computing device, the object being positioned within a three-dimensional environment, the object not being positioned in front of a two-dimensional patterned or monochromatic surface, the segmentation including for each image a first respective image portion that includes the object, the segmentation further including for each image a second respective image portion that includes a respective ground area located beneath the object, the segmentation further including a third respective image portion that includes a background area located above the second respective portion and behind the object;
• selecting a target background image based on a match between a first one or more ground plane vectors estimated for the target background image and a second one or more ground plane vectors estimated for the first visual representation;
• constructing a second visual representation based on the first visual representation via the processor at the mobile computing device, the second visual representation including the first respective image portion, the second visual representation including for each image a respective target background image portion that replaces the third respective image portion, the respective target background image portion being selected from one of the selected target background images based on an area of the third respective image portion relative to the respective image;
• and
• storing the second visual representation on a storage device.