Autonomous Agricultural System Including a Cart Management System Having a Perception System Including an Array Sensor have a Plurality of Sensors, Systems for Fusing Sensor Data with Additional Data, and Related Methods

A method of monitoring and controlling operation of a cart of an autonomous agricultural system, the cart being operably coupled to an agricultural vehicle of the autonomous agricultural system, the method comprising:

• capturing, via at least one of a thermal camera, a time-of-flight camera, a gate camera, and an event camera of an array of sensors and in real-time, sensor data of the agricultural vehicle, the cart, and a transport vehicle;
• capturing, via one of a RGB camera or a LIDAR camera, additional sensor data of the agricultural vehicle, the cart, and the transport vehicle;
• analyzing the sensor data to identify and classify the agricultural vehicle, the cart, and the transport vehicle to generate labeled sensor data;
• fusing the labeled sensor data with the additional sensor data by correlating each point or pixel of the additional sensor data with detected features of the labeled sensor data to form labeled fused data;
• based at least partially on the labeled fused data, determining an aligned position of the cart relative to the transport vehicle;
• and
• responsive to determining the aligned position, causing the cart to automatically align with the transport vehicle.

A cart management system for monitoring and controlling operation of a cart of an autonomous agricultural system, the cart being operably coupled to an agricultural vehicle of the autonomous agricultural system, the cart management system comprising:

• an array of sensors mounted on at least one the agricultural vehicle or the cart, the array of sensors comprising:
• a thermal camera;
• a time-of-flight camera;
• a gated camera;
• an event camera;
• an RGB camera;
• and
• a LIDAR camera;
• at least one processor;
• and
• at least one non-transitory computer-readable storage medium storing instructions thereon that, when executed by the at least one processor, cause the cart management system to:
• capture, via at least one of the thermal camera, the time-of-flight camera, the gate camera, and the event camera and in real-time, sensor data of the agricultural vehicle, the cart, and a transport vehicle;
• capture, via one of the RGB camera or the LIDAR camera, additional sensor data of the agricultural vehicle, the cart, and the transport vehicle;
• analyze the sensor data to identify and classify the agricultural vehicle, the cart, and the transport vehicle to generate labeled sensor data;
• fuse the labeled sensor data with the additional sensor data by correlating each point or pixel of the additional sensor data with detected features of the labeled sensor data to form labeled fused data;
• based at least partially on the labeled fused data, determine an aligned position of the cart relative to the transport vehicle;
• and
• responsive to determining the aligned position, cause the cart to automatically align with the transport vehicle.