# US 12,595,135 B1 · Predictive material handling equipment jam prevention

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,595,135 B1
- Document ID: US12595135B1
- Status: Granted
- Kind: B1
- Application number: 18/079,320
- Filed: 2022-12-12
- Granted: 2026-04-07
- Earliest claimed: 2022-12-12
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Jesse Edward Rochelle; Farouq Halawa; Raashid Mohammed; Chelsea Marie Beroske; Terrick Willoughby
- Web page: https://patentgrep.com/patent/US12595135B1

## CPC

- B65G 43/08: Control devices operated by article or material being fed, conveyed or discharged {(and controlling the discharging devices B65G47/42)}
- B65G 47/24: orientating the articles
- G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders
- G06T 7/0004: {Industrial image inspection}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- B65G 2203/0233: Position of the article
- B65G 2203/041: Camera
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20081: Training; Learning
- G06V 2201/06: Recognition of objects for industrial automation
- B65G: TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS (packaging B65B; handling thin or filamentary materials, e.g. paper sheets or thread, B65H; cranes B66C; portable mobile or lifting or hauling appliances, e.g. hoists, B66D; devices for lifting or lowering goods for loading or unloading purposes, e.g. fork-lift trucks, B66F9/00; emptying bottles, jars, casks, barrels or similar containers, not otherwise provided for, B67C9/00; delivering or transferring liquids B67D; filling or discharging vessels for liquefied, solidified or compressed gases F17C; pipe-line systems for fluids F17D)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Techniques for predicting and preventing jams in material handling equipment are described. In an example, a device includes a camera that can generate a video including image frames showing a material handling equipment. The device also includes a computer system that can generate an input including the image frames to a machine learning model. The computer system can determine an output of the machine learning model based on the input that includes a prediction that a jam in the material handling equipment is to occur within a next time interval and a classification of the jam. The jam can involve an object being handled by the material handling equipment. The computer system can determine a jam prevention action to perform based on the classification. The computer system can cause the jam prevention action to be performed within the next time interval in association with the object to prevent the jam.

## Claims

### Claim 1

A device comprising:
a camera configured to generate a video including images frames and showing at least portion of a material handling equipment;
one or more processors; and
one or more memory storing program code of a machine learning model and instructions that, upon execution by the one or more processors, configure the one or more processors to:
generate an input to the machine learning model, the input including a first image frame and a second image frame of the video, wherein the machine learning model is trained using training data that comprises an image and indicates, in the image, a bounding box around a first object that is being handled;
determine an output of the machine learning model based at least in part on the input, the output including a prediction that a jam in the material handling equipment is to occur within a next time interval and a classification of the jam, the jam involving a second object being handled by the material handling equipment and shown in the first image frame and the second image frame;
determine a jam prevention action to perform based at least in part on the classification; and
cause the jam prevention action to be performed within the next time interval in association with the second object such that the jam is prevented.

### Claim 2 (depends on 1)

The device of claim 1, further comprising:
an input/output (I/O) interface configured to communicatively couple the device with a controller of the material handling equipment, wherein the one or more memory store additional instructions that, upon execution by the one or more processors, configure the one or more processors to:
determine a predicted location of the jam from the output of the machine learning model; and
send, to the controller via the I/O interface, a command indicating the jam prevention action and the predicted location, wherein the jam prevention action is executed by the controller and includes controlling the material handling equipment.

### Claim 3 (depends on 1)

The device of claim 1, further comprising:
a controller, wherein the one or more memory store additional instructions that, upon execution by the one or more processors, configure the one or more processors to:
determine a predicted location of the jam from the output of the machine learning model; and
send, to the controller, a command indicating the jam prevention action and the predicted location, wherein the jam prevention action is controlled by the controller and includes handling the second object.

### Claim 4 (depends on 3)

The device of claim 3, further comprising:
a jam prevention mechanism coupled with the controller and configured to handle the second object, wherein the jam prevention mechanism includes at least one of: a dynamo, a set of arms, a set of blowers, a set of actuators, a set of stoppers, a set of air blades, or a set of vibration drums.

### Claim 5

A computer-implemented method comprising:
generating an input to a machine learning model, the input including a first image and a second image showing at least a portion of a material handling equipment, wherein the machine learning model is trained using training data that comprises another image and indicates, in the other image, a bounding box around an object that is being handled;
determining an output of the machine learning model based at least in part on the input, the output including a prediction that a jam in the material handling equipment is to occur;
determining, based at least in part on the prediction, that a jam prevention action is to be performed; and
causing the jam prevention action to be performed such that the jam is prevented.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, further comprising:
determining a classification of the jam, wherein the classification is indicated in the output of the machine learning model; and
determining the jam prevention action based at least in part on the classification.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, further comprising:
determining a predicted location of the jam in the material handling equipment, wherein the predicted location is indicated in the output of the machine learning model, and wherein the jam prevention action is performed at least at the predicted location.

### Claim 8 (depends on 5)

The computer-implemented method of claim 5, further comprising:
determining the jam prevention action based at least in part on the output of the machine learning model, wherein the output indicates the jam prevention action to perform.

### Claim 9 (depends on 5)

The computer-implemented method of claim 5, further comprising:
generating a second input to a second machine learning model, the second input including a third image showing at least a predicted location of the jam in the material handling equipment; and
determining a second output of the second machine learning model based at least in part on the second input, wherein the second output indicates whether the jam has occurred or not.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, further comprising:
determining, based at least in part on the second output, that the jam has occurred; and
sending, to a user device, an alert indicating that the jam has occurred and at least one of the predicted location, an actual location of the jam in the material handling equipment, or the third image.

### Claim 11 (depends on 9)

The computer-implemented method of claim 9, further comprising:
determining, based at least in part on the second output, that the jam has occurred;
determining a jam resolution action to perform; and
causing the jam resolution action to be performed such that the jam is resolved.

### Claim 12 (depends on 9)

The computer-implemented method of claim 9, further comprising:
generating additional training data based at least in part on the second output, wherein the additional training data includes the third image and an indication of whether the jam has occurred or not; and
further training the machine learning model based at least in part on the additional training data.

### Claim 13

One or more computer-readable storage media storing instructions that, upon execution by one or more processors, cause the one or more processors to perform operations comprising:
generating an input to a machine learning model, the input including a first image and a second image showing at least a portion of a material handling equipment, wherein the machine learning model is trained using training data that comprises another image and indicates, in the other image, a bounding box around an object that is being handled;
determining an output of the machine learning model based at least in part on the input, the output including a prediction that a jam in the material handling equipment is to occur;
determining, based at least in part on the prediction, that a jam prevention action is to be performed; and
causing the jam prevention action to be performed such that the jam is prevented.

### Claim 14 (depends on 13)

The one or more computer-readable storage media of claim 13 storing further instructions that, upon execution by one or more processors, cause the one or more processors to perform further operations comprising:
receiving a first plurality of images generated at a first rate;
selecting, from the first plurality of images, the first image and the second image to generate a first set of images at a second rate smaller than the first rate;
receiving a second plurality of images generated at the first rate;
selecting, from the second plurality of images, a third image to generate a second set of images at a third rate, wherein the third rate is based at least in part on the prediction and is larger than the second rate; and
generating a second input to the machine learning model based at least in part on the second set of images.

### Claim 15 (depends on 13)

The one or more computer-readable storage media of claim 13, wherein the training data is second training data comprising a second plurality of images, and wherein the one or more computer-readable storage media stores further instructions that, upon execution by one or more processors, cause the one or more processors to perform further operations comprising:
generating, by using a second machine learning model, first training data to train the machine learning model, wherein the first training data includes a first plurality of images and one or more labels indicating jam occurrence; and
training the machine learning model based at least in part on the first training data.

### Claim 16 (depends on 15)

The one or more computer-readable storage media of claim 15 storing additional instructions that, upon execution by one or more processors, cause the one or more processors to perform additional operations comprising:
generating the second training data that includes a second plurality of images and indicates, in the other image of the second plurality of images, at least one of: the bounding box around the object that is being handled or a material handling equipment location; and
training, based at least in part on the second training data, the second machine learning model to generate bounding boxes.

### Claim 17 (depends on 16)

The one or more computer-readable storage media of claim 16 storing other instructions that, upon execution by one or more processors, cause the one or more processors to perform other operations comprising:
generating a second input that includes the first plurality of images;
inputting, after the second machine learning model is trained, the second input to the second machine learning model;
determining a second output of the second machine learning model based at least in part on the second input, the second output indicating a bounding box position in a particular image of the first plurality of images;
determining a change to the bounding box position based at least in part on the first plurality of images;
determining the jam occurrence based at least in part on the change; and
including the first plurality of images and an indication of the jam occurrence in the first training data.

### Claim 18 (depends on 13)

The one or more computer-readable storage media of claim 13 storing further instructions that, upon execution by one or more processors, cause the one or more processors to perform further operations comprising:
receiving a first plurality of images generated at a first rate;
selecting, from the first plurality of images, the first image and the second image to generate a set of images at a second rate smaller than the first rate, wherein the input includes the set of images;
sending, to a remote computer system that hosts the machine learning model, the input; and
receiving, from the remote computer system, the output of the machine learning model.

### Claim 19 (depends on 18)

The one or more computer-readable storage media of claim 18 storing additional instructions that, upon execution by one or more processors, cause the one or more processors to perform additional operations comprising:
receiving, from the remote computer system, data indicating that the jam prevention action is to be performed; and
controlling performance of the jam prevention action.

### Claim 20 (depends on 13)

The one or more computer-readable storage media of claim 13 storing further instructions that, upon execution by one or more processors, cause the one or more processors to perform further operations comprising:
sending, to a remote computer system, data indicating that the jam prevention action is to be performed, wherein the remote computer system triggers performance of the jam prevention action.