# US 2024/0085887 A1 · Systems and methods for predicting anomalies in a manufacturing line

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0085887 A1
- Document ID: US20240085887A1
- Status: Published application
- Kind: A1
- Application number: 18/512,790
- Filed: 2023-11-17
- Published: 2024-03-14
- Earliest claimed: 2022-03-31
- Assignees: Ats Corporation
- Applicants: ATS Corporation
- Inventors: Stanley Kleinikkink; Steven Wardell; Michael Tze-Chun Chow
- Web page: https://patentgrep.com/patent/US20240085887A1

## CPC

- B65G 47/1407: {the articles being fed from a container, e.g. a bowl}
- B23Q 15/225: {in feed control, i.e. approaching of tool or work in successive decreasing velocity steps}
- B65G 43/08: Control devices operated by article or material being fed, conveyed or discharged {(and controlling the discharging devices B65G47/42)}
- B65G 47/14: arranging or orientating the articles by mechanical or pneumatic means during feeding (during transit by conveyor B65G47/24, B65G47/26)
- B65G 47/1421: {Vibratory movement}
- B65G 47/1428: {rotating movement}
- B65G 47/1464: {using the centrifugal effect to arrange or orientate the articles}
- G05B 19/4184: {characterised by fault tolerance, reliability of production system}
- G05B 19/41875: {characterised by quality surveillance of production}
- G05B 19/4189: {characterised by the transport system}
- G06T 7/0004: {Industrial image inspection}
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/40: Extraction of image or video features
- B65G 2203/0208: relating to the transported articles
- B65G 2203/025: Speed of the article
- B65G 2203/041: Camera
- G05B 2219/32194: Quality prediction
- G06T 2207/30164: Workpiece; Machine component
- B65G: TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS (packaging B65B; handling thin or filamentary materials, e.g. paper sheets or thread, B65H; cranes B66C; portable mobile or lifting or hauling appliances, e.g. hoists, B66D; devices for lifting or lowering goods for loading or unloading purposes, e.g. fork-lift trucks, B66F9/00; emptying bottles, jars, casks, barrels or similar containers, not otherwise provided for, B67C9/00; delivering or transferring liquids B67D; filling or discharging vessels for liquefied, solidified or compressed gases F17C; pipe-line systems for fluids F17D)
- B23Q: DETAILS, COMPONENTS, OR ACCESSORIES FOR MACHINE TOOLS, e.g. ARRANGEMENTS FOR COPYING OR CONTROLLING (tools of the kind used in lathes or boring machines B23B27/00); MACHINE TOOLS IN GENERAL CHARACTERISED BY THE CONSTRUCTION OF PARTICULAR DETAILS OR COMPONENTS; COMBINATIONS OR ASSOCIATIONS OF METAL-WORKING MACHINES, NOT DIRECTED TO A PARTICULAR RESULT
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Computer-implemented methods and systems for feeding workpieces to a manufacturing line are provided. An example method involves operating at least one processor to: receive, from at least one image device proximal to a bowl feeder, a sequence of images of workpieces within the bowl feeder; determine a flow velocity of the workpieces within the bowl feeder; generate bowl feeder control settings by applying the flow velocity to a predictive model; and automatically apply the bowl feeder control settings to the bowl feeder. Computer-implemented methods and systems for predicting anomalies in a manufacturing line are also provided. An example method involves operating at least one processor to: receive a sequence of images of workpieces in the manufacturing line; extract feature data from the sequence of images; apply the feature data to a predictive model to detect anomalies in the manufacturing line; and generate annotations to locate the anomalies within the images.

## Claims

### Claim 1 (depends on 1, 2, 4, 6, 8, 10, 13, 16, 17, 19, 21)

A method for predicting anomalies in a manufacturing line, the method comprising operating at least one processor to:
receive a sequence of images of one or more workpieces in the manufacturing line;
extract feature data from the sequence of images, the feature data comprising a representation of a motion and an appearance of the one or more workpieces in the manufacturing line;
apply the feature data to a predictive model to detect one or more anomalies in the manufacturing line;
generate one or more annotations to locate the one or more anomalies within the images of the manufacturing line; and
generate at least one notification to identify the anomalies, the at least one notification comprising the one or more annotations.
The method of claim 1 comprising operating the at least one processor to:
for each anomaly of the one or more anomalies,
identify at least one image amongst the sequence of images showing the anomaly;
select feature data associated with the anomaly; and
apply the feature data associated with the anomaly to the predictive model to determine a classification to be associated with the anomaly.
The method of claim 2, wherein the at least one notification comprises an indication of the classification associated with the anomaly.
The method of claim 2, further comprising operating the at least one processor to:
determine one or more corrective actions for the one or more anomalies based on the classifications associated with the one or more anomalies;
define a set of operating commands for one or more actuators of the manufacturing line based on the one or more corrective actions; and
operate the one or more actuators to implement the one or more corrective actions.
The method of claim 4, wherein the at least one notification comprises an indication of the one or more corrective actions.
The method of claim 2, wherein the manufacturing line comprises a transport mechanism.
The method of claim 6, comprising operating the at least one processor to classify the anomaly as at least one of a missing part of a workpiece or a change in a synchronous speed of a workpiece along the transport mechanism.
The method of claim 2, wherein the manufacturing line comprises a bowl feeder.
The method of claim 8, comprising operating the at least one processor to classify the anomaly as at least one of an accumulation of workpieces within the bowl feeder, a misalignment of workpieces within the bowl feeder, or insufficient workpieces within a lower portion of the bowl feeder.
The method of claim 1, further comprising operating the at least one processor to pre-process the sequence of images.
The method of claim 10, wherein operating the at least one processor to pre-process the sequence of images comprises operating the at least one processor to align each image of the sequence of images.
The method of claim 10, wherein operating the at least one processor to pre-process the sequence of images comprises operating the at least one processor to:
detect one or more moving workpieces in the sequence of images;
segment each moving workpiece of the one or more moving workpieces in a first image of the sequence of images;
select at least one moving workpiece of the one or more moving workpieces; and
identify a region of interest for each selected moving workpiece in each image of the sequence of images.
The method of claim 1, comprising operating the at least one processor to:
identify a plurality of images amongst the sequence of images showing a same moving workpiece of the one or more moving workpieces;
select feature data associated with the moving workpiece comprising a position and a timing associated with the position of the moving workpiece in each image of the plurality of images; and
apply the feature data associated with the moving workpiece to a regression model to determine the velocity of the moving workpiece.
The method of claim 13, comprising operating the at least one processor to reconstruct the motion of the moving workpiece across the plurality of images.
The method of claim 13, comprising operating the at least one processor to detect and mask the moving workpiece within each image of the plurality of images.
b). A system for predicting anomalies in a manufacturing line, the system comprising:
at least one processor operable to:
receive a sequence of images of one or more workpieces in the manufacturing line;
extract feature data from the sequence of images, the feature data comprising a representation of a motion and an appearance of the one or more workpieces in the manufacturing line;
apply the feature data to a predictive model to detect one or more anomalies in the manufacturing line;
generate one or more annotations to locate the one or more anomalies within the images of the manufacturing line; and
generate at least one notification to identify the anomalies, the at least one notification comprising the one or more annotations.
The system of claim 16, wherein the at least one processor is operable to:
for each anomaly of the one or more anomalies,
identify at least one image amongst the sequence of images showing the anomaly;
select feature data associated with the anomaly; and
apply the feature data associated with the anomaly to the predictive model to determine a classification to be associated with the anomaly.
The system of claim 17, wherein the at least one processor is operable to:
determine one or more corrective actions for the one or more anomalies based on the classifications associated with the one or more anomalies;
define a set of operating commands for one or more actuators of the manufacturing line based on the one or more corrective actions; and
operate the one or more actuators to implement the one or more corrective actions.
The system of claim 17, wherein the manufacturing line comprises a transport mechanism.
The system of claim 19, wherein the at least one processor is operable to classify the anomaly as at least one of a missing part of a workpiece or a change in a synchronous speed of a workpiece along the transport mechanism.
The system of claim 17, wherein the manufacturing line comprises a bowl feeder.
The system of claim 21, wherein the at least one processor is operable to classify the anomaly as at least one of an accumulation of workpieces within the bowl feeder, a misalignment of workpieces within the bowl feeder, or insufficient workpieces within a lower portion of the bowl feeder.
The system of claim 16, wherein the at least one processor is operable to:
detect one or more moving workpieces in the sequence of images;
segment each moving workpiece of the one or more moving workpieces in a first image of the sequence of images;
select at least one moving workpiece of the one or more moving workpieces; and
identify a region of interest for each selected moving workpiece in each image of the sequence of images.
The system of claim 16, wherein the at least one processor is operable to:
identify a plurality of images amongst the sequence of images showing a same moving workpiece of the one or more moving workpieces;
select feature data associated with the moving workpiece comprising a position and a timing associated with the position of the moving workpiece in each image of the plurality of images; and
apply the feature data associated with the moving workpiece to a regression model to determine the velocity of the moving workpiece.
