Real time sensor and method for synchronizing real time sensor data streams

An augmented reality system comprising:

• a computer network;
• a central computer coupled to the computer network;
• and
• a plurality of independent three-dimensional (3D) sensing apparatuses, each coupled to the computer network, wherein each 3D sensing apparatus includes:
• (i) stereoscopic image sensor pair including first video sensor spaced apart from a second video sensor configured to capture stereoscopic pairs of 2D images, the stereoscopic image sensor pair having a first field-of-view (FOV), the first and second video sensors being responsive to a trigger input, in common, that actuates the start of a stereoscopic image capture sequence capture one 3D image frame, and stereoscopic image sensor pair issuing a VSync signal at the completion of each captured 3D image frame;
• (ii) a time-of-flight (TOF) sensor having an actuation input that triggers the TOF sensor to start a TOF-data gathering cycle to collect TOF data, the actuation input of the TOF sensor being tied to the trigger input of the stereoscopic image sensor pair, the TOF data gathered during one 3D image frame being termed a TOF frame, the TOF sensor being aimed in the direction of the first FOV;
• (iii) at least one asynchronous sensor characterized by generating a continuous stream of asynchronous sensing data and lacking any input for aligning its asynchronous sensing data to any external clock source;
• (iv) a first memory space and a second memory space, both first memory space and second memory space being coupled to selectively store asynchronous sensing data from the at least one asynchronous sensor, the first memory space and second memory space being configured to alternately store asynchronous sensing data from the at least one asynchronous sensor in accordance with the VSync signal to define discrete asynchronous data groups each corresponding to a different 3D image frame, in accordance with the VSync signal;
• and
• (v) a data processing unit configured to provide a time stamp, in common, to each 3D image frame, TOF frame, and asynchronous data group in response to the VSync signal, and to collect the 3D image frames, TOF frames, and asynchronous data groups into data frame bundles in accordance with their common time stamps, and to output the collected data frame bundles to the central computer over the computer network.