Network for detecting edge cases for use in training autonomous vehicle control systems

A multi-sensor system for predicting a presence of two or more predefined features of interest in an environment comprising first and second regions which at least partially overlap with each other, the system comprising:

• a first sensor processor configured to receive, from a first sensor, first sensor data corresponding to a first plurality of sensed features detected in the first region by the first sensor, and to perform a first compression on the first sensor data to form encoded first sensor data that is smaller in size than the first sensor data, wherein the first sensor comprises a camera;
• a second sensor processor configured to receive, from a second sensor, second sensor data corresponding to a second plurality of sensed features detected in the second region by the second sensor, wherein the first and second pluralities of sensed features at least partially overlap with each other, and to perform a second compression on the second sensor data to form encoded second sensor data that is smaller in size than the second sensor data;
• a fusion processor configured to fuse the encoded first sensor data and the encoded second sensor data to form fused sensor data, to perform a third compression on the fused sensor data to form encoded fused sensor data, and to identify first and second predefined features of interest within the encoded fused sensor data;
• a network configured to exchange at least the encoded first sensor data and the encoded second sensor data between the first sensor processor, the second sensor processor, and the fusion processor, wherein
• at least one machine learning technique is used to train the multi-sensor system to minimize a cost function comprising (i) a first parameter that reduces a size of the at least one of the encoded first sensor data, the encoded second sensor data, and the encoded fused sensor data, and (ii) a second parameter that reduces a prediction error associated with at least one of said first and second identified predefined features of interest;
• said training causes the multi-sensor system to perform one or more sensor-configuration actions selected to collect at least one of the first sensor data and the second sensor data that is expected to reduce the prediction error associated with one of the first and second identified predefined features of interest, the one of the first and second identified predefined features of interest being determined, by the multi-sensor system, to have higher prediction uncertainty than the other of the first and second identified predefined features of interest;
• and
• said one or more sensor-configuration actions are performed, before new first sensor data is received by the first sensor processor, by adjusting a resolution of the first sensor, adjusting a focus of the first sensor, or changing an orientation of the first sensor so that the first sensor senses the second region;
• wherein at least one feature of the second plurality of sensed features is not directly detectable by the camera operating alone, and the camera changes its orientation based on the second plurality of sensed features, and
• wherein the cost function further comprises a term dependent on a gradient of a prediction-error component of the cost function with respect to candidate ones of the sensor-configuration actions, and said training further causes the camera to change its orientation based on the gradient-dependent term to reduce the prediction-error component subject to other terms in the cost function.