System and method for generating large simulation data sets for testing an autonomous driver

A system for training an autonomous system, comprising a target sensor, comprising:

• at least one hardware processor adapted to execute a code for:
• producing synthetic data simulating a possible signal captured from a common physical scene by the target sensor in an identified position relative to a plurality of sensors, where the synthetic data is produced using a plurality of real signals, the plurality of real signals are captured simultaneously from the common physical scene by the plurality of sensors;
• and
• training the autonomous system using the synthetic data;
• wherein producing the synthetic data using the plurality of real signals comprises:
• using a machine learning model to compute a plurality of computed depth maps based on the plurality of real signals, the plurality of real signals are captured simultaneously from the common physical scene, each of the plurality of real signals are captured by one of the plurality of sensors, each of the plurality of computed depth maps qualifies one of the plurality of real signals;
• applying a point of view transformation to the plurality of real signals and the plurality of computed depth maps, to produce synthetic data simulating the possible signal captured from the common physical scene by the target sensor in an identified position relative to the plurality of sensors;
• and
• providing the synthetic data to at least one testing engine to train the autonomous system.