Autonomous control of on-site movement of powered earth-moving construction or mining vehicles

A system, comprising:

• a plurality of sensors mounted on a powered earth-moving vehicle on a job site, including at least one track sensor, and one or more GPS antenna sensors;
• a plurality of additional sensors mounted on the powered earth-moving vehicle, including one or more LiDAR sensors, one or more image sensors, and one or more infrared sensors;
• and
• a self-contained control unit on the powered earth-moving vehicle that includes one or more hardware processors configured to, when executing stored instructions, perform automated operations including:
• obtaining, from the plurality of sensors mounted on the powered earth-moving vehicle, vehicle data about the powered earth-moving vehicle, including a track heading direction of one or more tracks of the powered earth-moving vehicle using the at least one track sensor, and GPS coordinates for one or more portions of a chassis of the powered earth-moving vehicle using the one or more GPS antenna sensors;
• obtaining, from the plurality of additional sensors, environment data about an area of the job site, including LiDAR data about the area from the one or more LiDAR sensors, and visual data about the area from the one or more image sensors, and infrared data for the area from the one or more infrared sensors;
• classifying, based at least in part on the environment data, an obstacle located on a path between a current position of the powered earth-moving vehicle on the job site and a target destination location on the job site, wherein the classifying of the obstacle is based in part on the infrared data and includes using one or more trained machine learning models to determine whether the obstacle is at least one of a person or an animal;
• monitoring, after the classifying and using one or more of the additional sensors, the obstacle to detect whether the obstacle is moving;
• determining, based at least in part on the classifying and on the monitoring of the obstacle to detect whether the obstacle is moving and on the infrared data for the obstacle being above a defined temperature threshold, information about the obstacle that includes the obstacle is the at least one of the person or the animal, or that includes the obstacle is movable out of the path by an attachment of the powered earth-moving vehicle;
• determining, based at least in part on a combination of the vehicle data and the environment data, to perform autonomous operations of the powered earth-moving vehicle that include inhibiting movement of the powered earth-moving vehicle if the obstacle is determined to be the at least one of the person or the animal, and otherwise that include movement of the powered earth-moving vehicle from the current position to the target destination location, including rotating, if a cabin heading direction of a cabin for the powered earth-moving vehicle is different from the track heading direction, the cabin until an updated cabin heading direction matches the track heading direction, and further including removing the obstacle from the path and moving along the path to the target destination location;
• and
• initiating the determined autonomous operations of the powered earth-moving vehicle, including continuing the inhibiting until the obstacle is no longer located on the path.

A non-transitory computer-readable medium having stored contents that cause one or more hardware processors to perform automated operations including at least:

• obtaining, by the one or more hardware processors and from a plurality of sensors mounted on a powered earth-moving vehicle on a job site, vehicle data about the powered earth-moving vehicle, including a track heading direction of one or more tracks of the powered earth-moving vehicle using at least one track sensor, and GPS coordinates for one or more portions of a chassis of the powered earth-moving vehicle using one or more GPS antenna sensors;
• obtaining, by the one or more hardware processors and from a plurality of additional sensors mounted on the powered earth-moving vehicle, environment data about an area of the job site, including LiDAR data about the area from one or more LiDAR sensors, and visual data about the area from one or more image sensors, and infrared data for the area from one or more infrared sensors mounted on the powered earth-moving vehicle;
• classifying, by the one or more hardware processors and based at least in part on the environment data, an obstacle located on a path between a current position of the powered earth-moving vehicle on the job site and a target destination location on the job site, wherein the classifying of the obstacle is based in part on the infrared data and includes using one or more trained machine learning models to determine whether the obstacle is at least one of a person or an animal;
• monitoring, by the one or more hardware processors and after the classifying and using one or more of the additional sensors, the obstacle to detect whether the obstacle is moving;
• determining, by the one or more hardware processors, and based at least in part on the classifying and on the monitoring of the obstacle to detect whether the obstacle is moving and on the infrared data for the obstacle being above a defined temperature threshold, information about the obstacle that includes the obstacle is the at least one of the person or the animal, or that includes the obstacle is movable out of the path by an attachment of the powered earth-moving vehicle;
• determining, by the one or more hardware processors and based at least in part on a combination of the vehicle data and the environment data, to perform autonomous operations of the powered earth-moving vehicle that include inhibiting movement of the powered earth-moving vehicle if the obstacle is determined to be the at least one of the person or the animal, and otherwise that include movement of the powered earth-moving vehicle from the current position to the target destination location, including rotating, if a cabin heading direction for a cabin of the powered earth-moving vehicle is different from the track heading direction, the cabin until an updated cabin heading direction matches the track heading direction, and further including removing the obstacle from the path and moving along the path to the target destination location;
• and
• initiating, by the one or more hardware processors, the determined autonomous operations of the powered earth-moving vehicle, including continuing the inhibiting until the obstacle is no longer located on the path.