Autonomous Agricultural System Including a Cart Management System Having a Perception System Including Image Sensors and Depth Sensors, Systems for Fusing Image Data with Depth Data, and Related Methods

A method of monitoring and controlling operation of a cart of an autonomous agricultural system, the cart being operably coupled to an agricultural vehicle of the autonomous agricultural system, the method comprising:

• capturing, via an array of sensors mounted on at least one the agricultural vehicle or the cart and in real-time, image data of the agricultural vehicle, the cart, and a transport vehicle;
• analyzing the image data to identify and classify the agricultural vehicle, the cart, and the transport vehicle to generate labeled image data;
• capturing, via the array of sensors and in real-time, depth data of the agricultural vehicle, the cart, and a transport vehicle;
• fusing the labeled image data with the depth data by correlating each point of the depth data with detected features of the labeled image data to form labeled fused data;
• receiving GNSS data and IMU data related to at least one of the agricultural vehicle, the cart, and a transport vehicle;
• fusing the labeled fused data with the GNSS data and IMU data to generate enhanced fused data;
• based at least partially on the enhanced fused data, determining an aligned position of the cart relative to the transport vehicle;
• and
• responsive to determining the aligned position, causing the cart to automatically align with the transport vehicle.

A cart management system for monitoring and controlling operation of a cart of an autonomous agricultural system, the cart being operably coupled to an agricultural vehicle of the autonomous agricultural system, the cart management system comprising:

• an array of sensors mounted on at least one the agricultural vehicle or the cart;
• at least one processor;
• and
• at least one non-transitory computer-readable storage medium storing instructions thereon that, when executed by the at least one processor, cause the cart management system to:
• capture, via the array of sensors and in real-time, image data of the agricultural vehicle, the cart, and a transport vehicle;
• analyze the image data to identify and classify the agricultural vehicle, the cart, and the transport vehicle to generate labeled image data;
• capture, via the array of sensors and in real-time, depth data of the agricultural vehicle, the cart, and a transport vehicle;
• fuse the labeled image data with the depth data by correlating each point of the depth data with detected features of the labeled image data to form labeled fused data;
• receive GNSS data and IMU data related to at least one of the agricultural vehicle, the cart, and a transport vehicle;
• fuse the labeled fused data with the GNSS data and IMU data to generate enhanced fused data;
• based at least partially on the enhanced fused data, determine an aligned position of the cart relative to the transport vehicle;
• and
• responsive to determining the aligned position, cause the cart to automatically align with the transport vehicle.