Method for detecting objects and localizing a mobile computing device within an augmented reality experience

A method comprising:

• at a mobile device, during a first time period:
• capturing a first series of frames;
• and
• capturing a first set of motion data representing motion of the mobile device;
• detecting a first set of relative positions of a first constellation of objects based on the first series of frames and the first set of motion data;
• calculating a classification vector of each object in the first constellation of objects based on the first series of frames to generate a first set of classification vectors;
• calculating a transform that aligns the first constellation of objects with a localization map that defines a set of objects, the set of objects characterized by:
• a second set of relative positions that match the first set of relative positions of the first constellation of objects;
• and
• a second set of classification vectors that match the first set of classification vectors of the first constellation of objects;
• returning, to the mobile device:
• a set of augmented reality assets defined by the localization map;
• and
• the transform;
• and
• at the mobile device:
• calculating a position of the mobile device relative to the localization map based on the transform;
• and
• rendering the set of augmented reality assets based on positions of the set of augmented reality assets in the localization map and based on the position of the mobile device in the localization map.

A method comprising:

• at a mobile device, during a first time period:
• capturing a first series of frames;
• and
• capturing a first set of motion data representing motion of the mobile device;
• detecting a first set of relative positions of a constellation of objects based on the first series of frames and the first set of motion data;
• generating a first set of object signatures of the constellation of objects by, for each object in the constellation of objects, calculating a confidence score of the object for each object class in a set of object classes;
• calculating a transform that aligns the constellation of objects with a localization map that defines a set of objects, the set of objects characterized by:
• a second set of relative positions that match the first set of relative positions of the constellation of objects;
• and
• a second set of object signatures that match the first set of object signatures of the constellation of objects;
• accessing a set of augmented reality assets defined by the localization map and the transform;
• calculating a position of the mobile device relative to the localization map based on the transform and the first set of motion data;
• and
• rendering the set of augmented reality assets based on positions of the set of augmented reality assets in the localization map and based on the position of the mobile device in the localization map.