Methods for tracking ground engaging tools between and within image frames

A non-transitory computer-readable medium storing instructions that, when executed by a computing system, cause the computing system to perform operations for ground engaging tool loss detection, the operations comprising:

• receiving, from at least one camera, at least one image of a first ground engaging tool and a second ground engaging tool attached to a first work machine at a worksite;
• inputting the at least one image into a neural network model to detect if the first ground engaging tool or the second ground engaging tool is missing by:
• generating, by the neural network model, a first segmentation mask around the first ground engaging tool and a second segmentation mask around the second ground engaging tool in the at least one image;
• determining, by the neural network model, a first centroid of the first segmentation mask and a second centroid of the second segmentation mask;
• calculating, by the neural network model, a distance from the first centroid of the first ground engaging tool to the second centroid of the second ground engaging tool;
• and
• in response to the distance not equaling a predetermined value, determining, by the neural network model, the second ground engaging tool is missing on the first work machine;
• and
• in response to determining the second ground engaging tool is missing sending a command to power down the first work machine or a second work machine at the worksite, wherein the second work machine is different than the first work machine.

A system comprising:

• one or more processors;
• and
• one or more memories storing instructions that, when executed by the one or more processors, cause the system to perform a process for ground engaging tool loss detection, the process comprising:
• receiving, from at least one camera, at least one image of a first ground engaging tool and a second ground engaging tool attached to a first work machine at a worksite;
• inputting the at least one image into a neural network model to detect if the first ground engaging tool or the second ground engaging tool is missing, by:
• generating, by the neural network model, a first segmentation mask around the first ground engaging tool and a second segmentation mask around the second ground engaging tool in the at least one image;
• determining by the neural network model, a first centroid of the first segmentation mask and a second centroid of the second segmentation mask;
• calculating, by the neural network model, a distance from the first centroid of the first ground engaging tool to the second centroid of the second ground engaging tool;
• and
• in response to the distance not equaling a predetermined value, determining, by the neural network model, the second ground engaging tool is missing on the first work machine;
• and
• in response to determining the second ground engaging tool is missing, sending a command to power down the first work machine or a second work machine at the worksite, wherein the second work machine is different than the first work machine.