Augmented reality or virtual reality system with active localisation of tools, use and associated process

A process for the calculation of a virtual trajectory with an augmented/virtual reality, AR/VR, simulation system operable by a user, wherein said AR/VR system comprises:

• an object comprising a plurality of optical markers disposed thereon;
• a physical tool suitable for use by the user, said physical tool configured for traveling a real trajectory in a space when used by the user;
• an AR/VR viewer;
• a first optical device configured for acquiring optical information;
• a processing unit communicatively connected at least to the physical tool, the first optical device and the AR/VR viewer;
• wherein:
• the physical tool comprises a second optical device configured for acquiring optical information;
• the physical tool comprises a terminal end;
• the plurality of optical markers of the object comprise first optical markers encoded with optical information acquirable by the first optical device, and second optical markers encoded with optical information which is acquirable by the second optical device comprised in the physical tool, wherein the second optical markers define a region of interaction between the physical tool and the object;
• the second optical device is arranged at the terminal end of the physical tool such that, for any operative position of the physical tool and the object for which the second optical markers are located within the field of view of the second optical device, said second optical markers are not occluded by the physical tool;
• and wherein the process comprises the following steps:
• acquiring optical information encoded on the first optical markers of the object using the first optical device;
• acquiring optical information encoded on the second optical markers of the object using the second optical device comprised in the physical tool;
• receiving, by the processing unit, the optical information acquired by the first optical device and the second optical device;
• processing, by the processing unit, said received optical information, based on identifying the first optical markers being observed from the first optical device, and the second optical markers being observed by the second optical device,
• calculating, by the processing unit, a virtual trajectory of a simulated tool based on the processed optical information, wherein the virtual trajectory comprises information related to the real trajectory travelled by the physical tool in space when used by the user, and
• calculating, by the processing unit and based on the processed optical information, a simulated interaction result between the physical tool and the object on the region of interaction defined by the second optical markers.