Techniques for recommending and presenting products in an augmented reality scene

A computer-implemented method performed by one or more server computers, the method comprising:

• receiving and storing information from a mobile computing device executing an augmented reality application, the information including an image of a real-world physical space captured by an image sensor of the mobile computing device, and a data representation of a virtual space corresponding to the real-world physical space, wherein the data representation of the virtual space includes information indicating a position of a virtual camera, within the virtual space, as determined by the mobile computing device using as input obtained from sensors of the mobile computing device, RGB images, depth images, IMU (inertial measurement unit) sampling data, and inferred scene geometry data;
• causing the image of the real-world space to be presented in a user interface at a client computing device, the user interface having controls to facilitate overlaying a rendered 3D model of a product on top of the image of the real-world physical space, wherein positioning and orientation of the rendered 3D model of the product overlaid on the image of the real-world physical space generates information indicating the position and orientation of the product relative within the virtual space, and the data representation of the virtual space including information indicating the position of the virtual camera is used to render the 3D model of the product in the user interface at the client computing device;
• storing the information for rendering images of the product along with the information indicating the position and orientation of the product relative to the model of virtual space, thereby enabling the stored information to be communicated to the mobile computing device later in time;
• and
• communicating to the mobile computing device executing the augmented reality application information for rendering images of the product along with the information indicating the position of the product relative to the model of virtual space, thereby enabling a 3D representation of the product to be rendered as part of a live augmented reality view of the real-world physical space.