# US 2023/0273624 A1 · Systems and methods for managing assignments of tasks for mining equipment using machine learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0273624 A1
- Document ID: US20230273624A1
- Status: Published application
- Kind: A1
- Application number: 17/652,926
- Filed: 2022-02-28
- Published: 2023-08-31
- Earliest claimed: 2022-02-28
- Assignees: Caterpillar Inc.
- Applicants: Caterpillar Inc.
- Inventors: Stefan Jacob WULF; Cameron T LANE; Andrew Seungyul YUN; Greg DAVIS; Thomas Frank DOHERTY; Darryl COLLINS
- Web page: https://patentgrep.com/patent/US20230273624A1

## CPC

- G05D 1/223: Command input arrangements on the remote controller, e.g. joysticks or touch screens
- B60L 53/14: Conductive energy transfer
- B60L 53/68: Off-site monitoring or control, e.g. remote control
- G05B 13/0265: {the criterion being a learning criterion}
- G05D 1/661: Docking at a base station (delivering or retrieving payloads G05D1/667)
- G06Q 10/0631: Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q 10/06316: {Sequencing of tasks or work}
- G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)}
- B60L 58/12: responding to state of charge [SoC]
- B60W 60/00256: {Delivery operations}
- B60W 2300/12: Trucks; Load vehicles
- B60W 2530/10: Weight
- B60W 2556/45: External transmission of data to or from the vehicle
- B60W 2720/10: Longitudinal speed
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B60L: PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES (arrangements or mounting of electrical propulsion units or of plural diverse prime-movers for mutual or common propulsion in vehicles B60K1/00, B60K6/20; arrangements or mounting of electrical gearing in vehicles B60K17/12, B60K17/14; preventing wheel slip by reducing power in rail vehicles B61C15/08; dynamo-electric machines H02K; control or regulation of electric motors H02P); SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES (electric coupling devices combined with mechanical couplings of vehicles B60D1/64; electric heating for vehicles B60H1/00); ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL (control or regulation of electric motors H02P); MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

Systems and methods are disclosed for managing task assignments for a fleet of haul trucks at a mine site. An assignment engine may: receive state data for a haul truck including haul weight data and second state data for the mine site that is indicative of a plurality of available tasks and associated task material weight data; assign a task to the haul truck by inputting the state data into a trained reinforcement-learning model, wherein: the model has been trained to learn an assignment policy that optimizes a reward function, such that the learned policy accounts for vehicle performance variance due to changing vehicle haul weight and/or road conditions; and cause the at least one haul truck to be operated according to the at least one task assignment.

## Claims

### Claim 1

A computer-implemented method for managing task assignments for a fleet of haul trucks at a mine site, comprising:
receiving, via one or more processors of an assignment engine system, first state data that includes state information for at least one haul truck of a fleet used to operate the mine site, the state information for the at least one haul truck including haul weight data;
receiving via the one or more processors of the assignment engine system, second state data that includes state information for the mine site, wherein the state information for the mine site is indicative of a plurality of tasks available in the mine site and task material weight data associated with the plurality of tasks;
assigning, via the one or more processors of the assignment engine system, at least one task from amongst the plurality of tasks to the at least one haul truck of the fleet by inputting the first state data and the second state data into a trained reinforcement-learning model, wherein:
the trained reinforcement-learning model has been trained, based on training first state data and training second state data, to learn an assignment policy that optimizes a reward function for the mine site, such that the trained reinforcement-learning model is configured to apply the learned assignment policy to the input first state data and second state data to select the at least one task; and
the training first state data includes training haul truck haul weight data, and the training second state data includes training task material weight data, such that the learned policy accounts for haul truck performance variance due to changing haul truck haul weight; and
causing, via the one or more processors of the assignment engine system, the at least one haul truck to be operated according to the at least one task assignment.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the state information for the mine site includes a material movement plan for the mine site and state information for one or more of a static charging station, a dynamic charging mechanism, a material source site, a material sink site, a material processing site, a weather condition associated with the mine site, or a road condition associated with the mine site

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the first state data includes a respective learning model for each haul truck in the fleet that is updated based on operating characteristics of the respective haul truck over time.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the at least one task assignment includes route instructions and speed instructions.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein:
the at least one haul truck is a green-house-gas free vehicle;
the state information for the at least one haul truck further includes a state of charge for the at least one haul truck; and
the training first state data further includes training haul truck state of charge data, and the training second state data includes training task charge requirement data, such that the learned policy accounts for capacity for the at least one haul truck to complete the at least one task assignment.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, the at least one haul truck is an autonomous vehicle.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the reward function accounts for at least one of total cost for operating the mine site, total material processed by the mine site, power consumption of the mine site, balance of charge across the fleet, state of charge of the fleet, an amount of opportunistic charging for the fleet, or capacity for the at least one haul truck to complete the at least one task assignment.

### Claim 8

A computer-implemented method for training a machine-learning model for managing task assignments for a fleet of haul trucks at a mine site, comprising:
obtaining, via one or more processors of an assignment engine system, first training state data that includes state information for at least one haul truck of a fleet used to operate a mine site, the state information for the at least one haul truck including haul weight data;
obtaining via the one or more processors of the assignment engine system, second training state data that includes state information for the mine site, wherein the state information for the mine site is indicative of a plurality of tasks available in the mine site and task material weight data associated with the plurality of tasks;
assigning, via the one or more processors of the assignment engine system, at least one task from amongst the plurality of tasks to the at least one haul truck of the fleet by inputting the first training state data and the second training state data into a reinforcement-learning model that includes a task assignment policy and a reward function;
causing, via the one or more processors of the assignment engine system, the at least one haul truck to be operated according to the at least one task assignment;
obtaining further first training state data and further second training state data from a period of time after the at least one task assignment is complete;
generating a score using the reward function based on the further first training state data and the further second training state;
updating the task assignment policy based on the generated score.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the method is iterated until one or more of:
the score at least meets a predetermined threshold;
a rate of change of the score across iterations is less than a further predetermined threshold;
a predetermined number of iterations have occurred; or
a predetermined period of time has passed.

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, further comprising:
receiving, from a user via a user device, one or more objective selections; and
generating or tuning the reward function based on the one or more objective selections.

### Claim 11 (depends on 8)

The computer-implemented method of claim 8, wherein at least a portion of the fleet is simulated, such that at least a portion of the first training state data is simulated information.

### Claim 12 (depends on 8)

The computer-implemented method of claim 8, wherein at least a portion of the mine site is simulated, such that at least a portion of the second training state data is simulated information.

### Claim 13 (depends on 8)

The computer-implemented method of claim 8, wherein the state information for the mine site includes a material movement plan for the mine site and state information for one or more of a static charging station, a dynamic charging mechanism, a material source site, a material sink site, a material processing site, a weather condition associated with the mine site, or a road condition associated with the mine site

### Claim 14 (depends on 8)

The computer-implemented method of claim 8, wherein the at least one task assignment includes route instructions and speed instructions.

### Claim 15 (depends on 1)

The computer-implemented method of claim 1, wherein:
the at least one haul truck is a green-house-gas free vehicle; and
the at least one haul truck is an autonomous vehicle.

### Claim 16 (depends on 8)

The computer-implemented method of claim 8, wherein the reward function is configured to account for at least one of total cost for operating the mine site, total material processed by the mine site, power consumption of the mine site, balance of charge across the fleet, state of charge of the fleet, or capacity for the at least one haul truck to complete the at least one task assignment.

### Claim 17

A system for operating a mine site, comprising:
a fleet of haul trucks that are green-house-gas free and autonomous;
a central controller, including:
at least one memory storing instructions; and
at least one processor operatively connected to the at least one memory and configured to execute the instructions to implement:
a production planner configured to determine at least one task assignment available for the fleet at the mine site and task material weight data for the at least one task; and
a dispatcher configured to assign the at least one task assignment to at least one haul truck of the fleet, the dispatcher having:
a trained reinforcement-learning model that has been trained, based on training first state data that includes state information for the fleet and training second state data that includes state information for the mine site, to learn an assignment policy that optimizes a reward function for the mine site, such that the trained reinforcement-learning model is configured to apply the learned assignment policy to input first state data for the fleet and second state data for the mine site to select at least one task to assign to the fleet, wherein the training first state data includes training haul truck haul weight data and the training second state data includes training task material weight data, such that the learned policy accounts for haul truck performance variance due to changing haul truck haul weight; and
an assignment engine configured to perform operations, including:
receiving the first state data including haul weight data for at least one haul truck in the fleet;
receiving the second state data;
assigning the at least one task to the at least one haul truck by inputting the first state data and the second state data into the trained reinforcement-learning model: and
causing the at least one haul truck to be operated according to the at least one task assignment.

### Claim 18 (depends on 17)

The system of claim 17, further comprising:
at least one production site;
at least one dump site;
at least one static charging station; and
at least one dynamic charging device;
wherein the at least one task assignment includes a route to one or more of the at least one production site, the at least one dump site, the at least one charging station, or the at least one dynamic charging device.

### Claim 19 (depends on 17)

The system of claim 17, wherein the at least one task assignment includes route information and speed information.

### Claim 20 (depends on 17)

The system of claim 17, wherein the reward function accounts for at least one of total cost for operating the mine site, total material processed by the mine site, power consumption of the mine site, balance of charge across the fleet, state of charge of the fleet, or capacity for the at least one haul truck to complete the at least one task assignment.
