Computer vision and speech algorithm design service

A system comprising:

• a memory embodied with executable instructions;
• and
• a processor programmed for:
• performing a simulation using ground truth data and configuration data, the configuration data including one or more of:
• scene configuration data, hardware configuration data, and hardware location data;
• based on the performed simulation, outputting synthetic imagery and simulated inertial measurement unit (IMU) data;
• testing a plurality of computer vision and speech algorithms using the synthetic imagery and the simulated IMU data;
• calculating a virtual position of a synthetic hardware device based on the testing;
• comparing the calculated virtual position with the ground truth data;
• producing evaluation results based on the comparison;
• and
• marking one or more of the plurality of computer vision and speech algorithms that did not produce the evaluation results within a variance threshold, the marking comprising marking the one or more of the plurality of computer vision and speech algorithms for one or more of:
• additional testing, simulations, and as faulty.

A method comprising:

• performing a simulation using ground truth data and configuration data, the configuration data including one or more of:
• scene configuration data, hardware configuration data, and hardware location data;
• based on the performed simulation, outputting synthetic imagery and simulated inertial measurement unit (IMU) data;
• testing a plurality of computer vision and speech algorithms using the synthetic imagery and the simulated IMU data;
• calculating a virtual position of a synthetic hardware device based on the testing;
• comparing the calculated virtual position with the ground truth data;
• producing evaluation results based on the comparison;
• and
• marking one or more of the plurality of computer vision and speech algorithms that did not produce the evaluation results within a variance threshold, the marking comprising marking the one or more of the plurality of computer vision and speech algorithms for one or more of:
• additional testing, simulations, and as faulty.

A computer storage device storing computer-executable instructions which on execution by a computer cause the computer to perform operations comprising:

• performing a simulation using ground truth data and configuration data, the configuration data including one or more of:
• scene configuration data, hardware configuration data, and hardware location data;
• based on the performed simulation, outputting synthetic imagery and simulated inertial measurement unit (IMU) data;
• testing a plurality of computer vision and speech algorithms using the synthetic imagery and the simulated IMU data;
• calculating a virtual position of a synthetic hardware device based on the testing;
• comparing the calculated virtual position with the ground truth data;
• producing evaluation results based on the comparison;
• and
• marking one or more of the plurality of computer vision and speech algorithms that did not produce the evaluation results within a variance threshold, the marking comprising marking the one or more of the plurality of computer vision and speech algorithms for one or more of:
• additional testing, simulations, and as faulty.