US 2023/0419208 A1Published application

Systems and methods for managing assignments of tasks for work machines using machine learning

AssigneeCaterpillar Inc.

InventorsStefan J. WULF, Cameron T. LANE, Andrew S. YUN

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods are disclosed for managing task assignments for a plurality of work machines at a site. An assignment engine may: receive first state data for a work machine including historical data, operating condition, and location data, and second state data for the site including characteristic data for materials and a plurality of available tasks; predict performance data and energy consumption data of the work machine for a task; select a task for the work machine by inputting first state data and second state data into a trained reinforcement-learning model, wherein: the model has been trained to learn an assignment policy that optimizes a reward function such that the learned policy selects a task for at least one work machine from the plurality of tasks available at the site; and cause the at least one work machine to be operated according to the at least one task assignment.

Claims27

Dependent claims

A computer-implemented method for managing task assignments for a plurality of work machines at a site, comprising:

receiving, via one or more processors of an assignment engine system, first state data for at least one work machine operating at the site, wherein the first state data includes historical data of the at least one work machine, one or more operating condition of at least one component of the at least one work machine, real-time or near real-time location data of the at least work machine, or a combination thereof;
receiving, via the one or more processors of the assignment engine system, second state data for the site, wherein the second state data includes characteristic data for material in the site and a plurality of tasks available at the site, wherein the plurality of tasks are associated with manipulation of the material in the site, and the characteristic data is indicative of energy consumption associated with the manipulation of the material by the at least one work machine;
predict performance data and energy consumption data of the at least one work machine for a task;
and
selecting at least one task for the at least one work machine from the plurality of tasks available by inputting, via the one or more processors of the assignment engine system, the first state data and the second state data into a trained reinforcement-learning model, wherein the trained reinforcement-learning model has been trained, based on training first state data and training second state data, to:
learn an assignment policy that optimizes a reward function for the site, such that the trained reinforcement-learning model is configured to apply the learned assignment policy to the first state data, the second state data, and one or more predictions of the performance data and the energy consumption data of the at least one work machine to select the at least one task for the at least one work machine from the plurality of tasks available at the site;
and
causing, via the one or more processors of the assignment engine system, the at least one work machine to be operated according to the at least one selected task.

5 more elements

A computer-implemented method for training a machine-learning model for managing task assignments for a plurality of work machines at a site, comprising:

receiving, via one or more processors of an assignment engine system, first state data for at least one work machine operating at the site, wherein the first state data includes historical data of the at least one work machine, one or more operating condition of at least one component of the at least one work machine, real-time or near real-time location data of the at least work machine, or a combination thereof;
receiving, via the one or more processors of the assignment engine system, second state data for the site, wherein the second state data includes characteristic data for material in the site and a plurality of tasks available at the site, wherein the plurality of tasks are associated with manipulation of the material in the site, and the characteristic data is indicative of energy consumption associated with the manipulation of the material by the at least one work machine;
assigning, via the one or more processors of the assignment engine system, at least one task from the plurality of tasks to the at least one work machine by inputting the first state data and the second state data into a trained reinforcement-learning model that includes a task assignment policy and a reward function;
causing, via the one or more processors of the assignment engine system, the at least one work machine to be operated according to the at least one task assignment;
receiving, via the one or more processors of the assignment engine system, further first state data and further second state data after a completion of the at least one task assignment;
generating, via the one or more processors of the assignment engine system, a score using the reward function based on the further first state data and the further second state data;
and
updating the task assignment policy based, at least in part, on the generated score.

5 more elements

A system for operating a plurality of work machines at a site, comprising:

the plurality of work machines that are green-house-gas free and autonomous;
a central controller, including:
at least one memory storing instructions;
and
at least one processor operatively connected to the at least one memory and configured to execute the instructions to implement:
a production planner configured to determine at least one task assignment for the plurality of work machines at the site;
and
a dispatcher configured to assign the at least one task assignment to at least one work machine, the dispatcher having:
a trained reinforcement-learning model that has been trained, based on training first state data that includes historical data of the at least one work machine, one or more operating condition of at least one component of the at least one work machine, real-time or near real-time location data of the at least work machine, or a combination thereof and training second state data that includes characteristic data for material in the site indicative of energy consumption associated with manipulation of the material by the at least one work machine and a plurality of available tasks that are associated with the manipulation of the material in the site, to learn an assignment policy that optimizes a reward function for the site, such that the trained reinforcement-learning model is configured to apply the learned assignment policy to input first state data for the work machines and second state data for the site to select at least one task to assign to the work machines;
and
an assignment engine configured to perform operations, including:
receiving the first state data;
receiving the second state data;
assigning the at least one task to the at least one work machine by inputting the first state data and the second state data into the trained reinforcement-learning model:
and
causing the at least one work machine to be operated according to the at least one task assignment.

13 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/936,462Filed 29 Sep 2022
Priority: 24 Jun 2022
Published: 28 Dec 2023
Term ends: Not estimatedThe source records don’t support an estimate.

Classification26 codes

G06Q 10/063114
Scheduling, planning or task assignment for a person or group: status monitoring or status determination for a person or group
B60L 53/63
Monitoring or controlling charging stations: in response to network capacity
B60L 58/16
For monitoring or controlling batteries: responding to battery ageing, e.g. to the number of charging cycles or the state of health [SoH]
B60W 40/10
Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a…:…