Overlaying 3d augmented reality content on real-world objects using image segmentation

A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a processor, cause the processor to:

• determine a position and a scale of a physical object depicted in an image from a viewpoint;
• identify a symmetry-based mismatch between a three-dimensional (3D) model of the physical object and a digital outline of the physical object at the viewpoint, the symmetry-based mismatch based at least in part on symmetrical characteristics of the physical object;
• cause the 3D model of the physical object to match the position and the scale of the physical object based on the symmetry-based mismatch;
• overlay a 3D virtual object at a first location on an exterior portion of the physical object in augmented reality based on a predetermined locational relation between the 3D virtual object and the 3D model of the physical object;
• and
• generate a composite view of the 3D virtual object overlaid on the exterior portion of the physical object, wherein the 3D virtual object is partially obstructed from view by the physical object at the viewpoint, wherein the 3D virtual object is an exterior object and a portion of the 3D virtual object is omitted from the composite view based on the portion of the 3D virtual object being partially obstructed by the physical object at the viewpoint.

A computing apparatus comprising:

• a processor;
• and
• a memory storing instructions that, when executed by the processor, cause the processor to:
• determine a position and a scale of a physical object depicted in an image from a viewpoint;
• identify a symmetry-based mismatch between a three-dimensional (3D) model of the physical object and a digital outline of the physical object at the viewpoint, the symmetry-based mismatch based at least in part on symmetrical characteristics of the physical object;
• cause the 3D model of the physical object to match the position and the scale of the physical object based on the symmetry-based mismatch;
• overlay a 3D virtual object at a first location on an exterior portion of the physical object in augmented reality based on a predetermined locational relation between the 3D virtual object and the 3D model of the physical object;
• and
• generate a composite view of the 3D virtual object overlaid on the exterior portion of the physical object, wherein the 3D virtual object is partially obstructed from view by the physical object at the viewpoint, wherein the 3D virtual object is an exterior object and a portion of the 3D virtual object is omitted from the composite view based on the portion of the 3D virtual object being partially obstructed by the physical object at the viewpoint.