Method for visual pose determination of unknown objects in a mixed reality context

A recognition device within a mixed reality (MR) system, comprising:

• a camera;
• a first wideband module capable of communicating with an object of interest to obtain characteristics of the object of interest, thereby obtaining reference images of the object of interest, wherein:
• the object of interest constitutes a real-world environment;
• and
• the recognition device does not pre-store the characteristics of the object of interest;
• a tracking module configured to calculate a visual pose of the object of interest by matching the reference images of the object of interest to image features extracted from a segment of video flux provided by the camera;
• and
• a rendering module configured to compute a visual representation of the object of interest based on the calculated visual pose of the object of interest and the reference images of the object of interest.

A method for visual pose determination of an object of interest in a mixed reality (MR) system, comprising:

• advertising a presence of the object of interest to a recognition device;
• transmitting data between a first wideband module of the recognition device and a second wideband module of the object of interest via wideband radio, wherein:
• the object of interest constitutes a real-world environment and a visual representation of the object of interest is capable of being superimposed into a virtual (VR) environment;
• the transmitted data includes at least one characteristic of the object of interest, which is not pre-stored in the recognition device, and is sent from the second wideband module to the first wideband module after the presence of the object of interest is advertised;
• and
• the at least one characteristic of the object of interest is one of a group consisting of a 3D model of the object of interest, a series of photos of the object of interest under different perspectives, and an identifier of the object of interest;
• performing a coarse pose estimation of the object of interest;
• performing a fine pose estimation of the object of interest based on the coarse pose estimation of the object of interest;
• and
• reprojecting by the recognition device the visual representation of the object of interest into the VR environment.