# US 12,117,812 B2 · Systems, methods, and media for manufacturing processes > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,117,812 B2 - Document ID: US12117812B2 - Status: Granted - Kind: B2 - Application number: 17/304,349 - Filed: 2021-06-18 - Published: 2021-10-07 - Granted: 2024-10-15 - Earliest claimed: 2019-11-06 - Assignees: Nanotronics Imaging, Inc. - Applicants: Nanotronics Imaging, Inc. - Inventors: Andrew Sundstrom; Eun-Sol Kim; Damas Limoge; Vadim Pinskiy; Matthew C. Putman - Web page: https://patentgrep.com/patent/US12117812B2 ## CPC - G05B 19/41875: {characterised by quality surveillance of production} - G05B 13/0265: {the criterion being a learning criterion} - G05B 19/4188: {characterised by CIM planning or realisation} - G06T 1/0014: {Image feed-back for automatic industrial control, e.g. robot with camera (robots B25J19/023)} - G06T 7/001: {using an image reference approach} - G06T 7/33: using feature-based methods - G06V 10/764: using classification, e.g. of video objects - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G05B 2219/31001: CIM, total factory control - G05B 2219/33034: Online learning, training - G06T 2207/20081: Training; Learning - G06T 2207/30164: Workpiece; Machine component - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A manufacturing system is disclosed herein. The manufacturing system includes one or more stations, a monitoring platform, and a control module. Each station of the one or more stations is configured to perform at least one step in a multi-step manufacturing process for a component. The monitoring platform is configured to monitor progression of the component throughout the multi-step manufacturing process. The control module is configured to dynamically adjust processing parameters of each step of the multi-step manufacturing process to achieve a desired final quality metric for the component. ## Claims ### Claim 1 A manufacturing system, comprising: one or more stations, each station configured to perform at least one step in a multi-step manufacturing process for a component; a monitoring platform configured to monitor progression of the component throughout the multi-step manufacturing process; and a control module configured to dynamically adjust processing parameters of a step of the multi-step manufacturing process to achieve a desired final quality metric for the component, the control module configured to perform operations, comprising: receiving image data of tooling of a first station of the one or more stations; determining, by a first machine learning model, a final quality metric for the component, based on the image data; determining that the final quality metric is outside an acceptable range of the desired final quality metric; based on the determining, generating an updated instruction set to be performed by at least one of the first station or a downstream station to achieve the desired final quality metric; predicting, by a second machine learning model, an updated final quality metric for the component based on the updated instruction set; and based on the updated final quality metric, providing the updated instruction set to at least one of a first station controller associated with the first station or a second station controller associated with the downstream station. ### Claim 2 (depends on 1) The manufacturing system of claim 1, wherein the second machine learning model is trained with a training set comprising a synthetic set of data. ### Claim 3 (depends on 1) The manufacturing system of claim 1, wherein the final quality metric cannot be measured until processing of the component is complete. ### Claim 4 (depends on 1) The manufacturing system of claim 1, further comprising: determining, based on positional information of the component, that an irreversible error is not present. ### Claim 5 (depends on 1) The manufacturing system of claim 1, further comprising: identifying a set of keypoints from the image data, the set of keypoints corresponding to position information of the tooling during processing at the first station. ### Claim 6 (depends on 5) The manufacturing system of claim 5, wherein the set of keypoints are used by the first machine learning model to determine the final quality metric. ### Claim 7 (depends on 5) The manufacturing system of claim 5, further comprising: comparing the set of keypoints corresponding to coordinates of the component to a canonical set of keypoints corresponding to a canonical component. ### Claim 8 A method for dynamically adjusting processing parameters of a step of a multi-step manufacturing process to achieve a desired final quality metric for a component undergoing the multi-step manufacturing process, comprising: receiving, by a computing system, image data of tooling of a first station of a plurality of stations involved in the multi-step manufacturing process; determining, by a first machine learning model of the computing system, a final quality metric for the component, based on the image data, the final quality metric representing a metric associated with the component that cannot be measured until the multi-step manufacturing process is complete; determining, by the computing system, that the final quality metric is outside an acceptable range of the desired final quality metric; based on the determining an updated instruction set to be performed by at least one of the first station or a downstream station to achieve the desired final quality metric; predicting, by a second machine learning model of the computing system, an updated final quality metric for the component based on the updated instruction set; and based on the updated final quality metric, providing, by the computing system, the updated instruction set to at least one of a first station controller associated with the first station or a second station controller associated with the downstream station. ### Claim 9 (depends on 8) The method of claim 8, wherein the second machine learning model is trained with a training set comprising a synthetic set of data. ### Claim 10 (depends on 8) The method of claim 8, wherein the final quality metric cannot be measured until processing of the component is complete. ### Claim 11 (depends on 8) The method of claim 8, further comprising: determining, based on positional information of the component, that an irreversible error is not present. ### Claim 12 (depends on 8) The method of claim 8, further comprising: identifying a set of keypoints from the image data, the set of keypoints corresponding to position information of the tooling during processing at the first station. ### Claim 13 (depends on 12) The method of claim 12, wherein the set of keypoints are used by the first machine learning model to determine the final quality metric. ### Claim 14 (depends on 12) The method of claim 12, further comprising: comparing the set of keypoints corresponding to coordinates of the component to a canonical set of keypoints corresponding to a canonical component. ### Claim 15 A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by a computing system, causes a processor to perform operations for dynamically adjusting processing parameters of a step of a multi-step manufacturing process to achieve a desired final quality metric for a component undergoing the multi-step manufacturing process, comprising: receiving, by the computing system, image data of tooling of a first station of one or more stations involved in the multi-step manufacturing process; determining, by a first machine learning model of the computing system, a final quality metric for the component, based on the image data; determining, by the computing system, that the final quality metric is outside an acceptable range of the desired final quality metric; based on the determining, generating an updated instruction set to be performed by at least one of the first station or a downstream station to achieve the desired final quality metric; predicting, by a second machine learning model of the computing system, an updated final quality metric for the component based on the updated instruction set; and based on the updated final quality metric, providing, by the computing system, the updated instruction set to at least one of a first station controller associated with the first station or a second station controller associated with the downstream station. ### Claim 16 (depends on 15) The non-transitory computer readable medium of claim 15, wherein the second machine learning model is trained with a training set comprising a synthetic set of data. ### Claim 17 (depends on 15) The non-transitory computer readable medium of claim 15, wherein the final quality metric cannot be measured until processing of the component is complete. ### Claim 18 (depends on 15) The non-transitory computer readable medium of claim 15, further comprising: determining, based on positional information of the component, that an irreversible error is not present. ### Claim 19 (depends on 15) The non-transitory computer readable medium of claim 15, further comprising: identifying a set of keypoints from the image data, the set of keypoints corresponding to position information of the tooling during processing at the first station. ### Claim 20 (depends on 19) The non-transitory computer readable medium of claim 19, wherein the set of keypoints are used by the first machine learning model to determine the final quality metric.