A system for sorting objects having different characteristics, the system comprising:
- a free-fall sorting section, the free-fall sorting section for sorting objects as the objects fall;
- a first sensor for observing the objects, including observing first observed characteristics of the objects;
- a sorting actuator, after the free-fall sorting section, for sorting the objects based on the first observed characteristics of the objects and using a policy matrix comprising sorting rules;
- a second sensor for observing second observed characteristics of the sorted objects after the sorting by the sorting actuator;
- and
- a controller comprising an action agent module for controlling the sorting actuator and a learning agent module for updating the policy matrix, the controller being configured to
- update, by the learning agent module, the policy matrix after sorting previously sorted objects, wherein the policy matrix includes state data of the previously sorted objects including for each of the previously sorted objects the sensor data, an action performed by the sorting actuator, and a sorting score, wherein the learning agent module performs reinforcement learning to update the policy matrix using a reward function derived from the sorting score,
- actuate, by the action agent module, the sorting actuator to perform one or more actions to sort one or more objects of the objects using the policy matrix comprising the sorting rules and the state data for the previously sorted objects, wherein the sorting actuator sorts the one or more objects such that a resulting batch of objects has a different threshold amount of objects of the first observed characteristics relative to objects of the second observed characteristics,
- calculate, by the learning agent module, after the sorting of the specific object by the sorting actuator, a specific sorting score based on the at least one specific action performed by the sorting actuator and a second observed characteristic of the specific sorted object of the objects, and
- update, by the learning agent module after the sorting of the specific object by the sorting actuator, the policy matrix comprising the sorting rules to create an updated policy matrix including updated sorting rules, using the state data of the specific object, wherein the updated policy matrix including the updated sorting rules is used in sorting subsequent objects;
- wherein the learning agent module aggregates pre-sort system states of prior sorting actions by the sorting actuator and prior object observations by the first and second sensors and uses the pre-sort system states to perform the sorting of a first set of the objects;
- wherein the learning agent module aggregates system states of sorting actions of the first set of the objects by the sorting actuator and object observations by the first and second sensors of the first set of the objects and uses the system states of the sorting actions of the first set of the objects to perform the sorting of a second set of the objects;
- wherein the policy matrix is initialized with no predefined sorting rules and the reinforcement learning occurs entirely from the aggregated system states of the produce objects;
- wherein the system trains the policy matrix using a first small batch of produce objects and subsequently applies the updated policy matrix to sort larger production batches of produce objects;
- and
- wherein the objects comprise produce objects.