Live augmented reality using tracking

A method comprising:

• receiving a request to capture a plurality of images used to generate a multi-view interactive digital media representation of a vehicle appearing in the plurality of images;
• receiving first live images, comprising the vehicle, captured from a camera on a mobile device, the first live images being output to a display of the mobile device, the first live images comprising first 2-D pixel data captured by the camera;
• receiving first sensor data indicating a first orientation of the camera associated with a first image of the first live images;
• generating a first synthetic image comprising:
• 1) a location selector rendered into the first 2-D pixel data associated with the first image, the location selector being a movable first virtual object in the first synthetic image such that selection of the location selector causes a pixel location in the first image to be selected and 2) the vehicle;
• receiving, via a touch screen of the display and via the location selector, a selection of the pixel location in the first 2-D pixel data from the first image;
• determining a first pixel location in the first 2-D pixel data from the first image of a first tracking point, the first tracking point being within the first 2-D pixel data associated with the vehicle and the first tracking point being proximate to the pixel location selected via the location selector;
• generating a second synthetic image comprising 1) a second virtual object rendered into the first 2-D pixel data from the first image, the second virtual object being positioned in the first 2-D pixel data from the first live image relative to first pixel location of the first tracking point;
• outputting the second synthetic image to the display;
• after the first image is captured, receiving second live images captured by the camera, the second live images comprising second 2-D pixel data, the second live images comprising the vehicle from a plurality of different views;
• receiving second sensor data, associated with the second live images, indicating second orientations of the camera;
• receiving second live image data comprising second 2-D pixel data from the camera;
• based upon the first sensor data, the second sensor data, the first 2-D pixel data and the second 2-D pixel data, determining, as the view of the vehicle in the second live images changes, second pixel locations of the first tracking point in the second 2-D pixel data of the second live images on an image by image basis, the second pixel locations being determined using spatial intensity information or optical flows derived from the second 2-D pixel data;
• generating a third synthetic images comprising the second virtual object rendered into the second 2-D pixel data at third pixel locations positioned relative to the second pixel locations of the first tracking point;
• outputting the third synthetic images to the display, each of the third synthetic images showing one of the different views of the vehicle as captured by the camera and the second virtual object.