Skeleton-based effects and background replacement

A method comprising:

• on a mobile device including a processor, a memory, a camera, a plurality of sensors, a microphone and a touchscreen display, receiving via an input interface on the mobile device a request to generate a multi-view interactive digital media representation of an object;
• receiving a live video stream including a plurality of frames from the camera on the mobile device as the mobile device moves along a trajectory wherein an orientation of the camera varies along the trajectory such that the object in the live video stream is captured from a plurality of camera views;
• receiving a selection of an effect which modifies the object in the multi-view interactive digital media representation of the object;
• for one or more frames among the plurality of frames, generating an initial skeleton indicating locations on the object in the one or more frames wherein the effect is anchored to one or more of the locations such that the effect appears attached to the object;
• for the one or more frames, generating an initial segmentation which includes a boundary between the object and an original background;
• for the one or more frames, based upon the initial skeleton and the initial segmentation, generating a preview of the effect wherein the preview includes one or more augmented frames with the effect modifying the object;
• and
• while the live video stream is being recorded for the multi-view interactive digital media representation and prior to generation of the multi-view interactive digital media representation, outputting the preview of the effect to the touchscreen display.