Augmented learning model for autonomous earth-moving vehicles

A system for autonomous operation of a vehicle, the system comprising:

• a computer data store configured to maintain a first set of pre-existing offline sensor data;
• and
• one or more processors configured to:
• (a) train a first machine learning model of the one or more processors utilizing the first set of pre-existing offline sensor data, wherein the first machine learning model comprises a world model;
• (b) execute, a first set of instructions and a second set of instructions in parallel until a convergence condition is reached,
• (i) the first set of instructions comprising:
• (1) receiving a second set of sensor data, wherein the second set of sensor data is generated subsequently to the training of the first machine learning model and is not included within the first set of sensor data;
• (2) generating, by the world model, a first condensed vector from the second set of sensor data at least in part by processing the second set of sensor data with the first machine learning model;
• and
• (3) selecting or generating an action to be performed by the vehicle, or a component of the vehicle, at least in part by processing the first condensed vector with a second machine learning model, wherein the second machine learning model comprises a behavioral model;
• (ii) the second set of instructions comprising:
• (1) retrieving one or more samples of sensor data from the first set of sensor data and/or the second set of sensor data;
• (2) updating the first machine learning model at least in part by processing the one or more samples of sensor data to produce a second condensed vector;
• and
• (3) updating the second machine learning model at least in part by processing the second condensed vector, wherein the behavioral model of the second machine learning model selects or generates the action to be performed by the vehicle.