# US 11,079,745 B2 · Rapid closed-loop control based on machine learning > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,079,745 B2 - Document ID: US11079745B2 - Status: Granted - Kind: B2 - Application number: 14/952,665 - Filed: 2015-11-25 - Published: 2017-05-25 - Granted: 2021-08-03 - Earliest claimed: 2015-11-25 - Assignees: Lawrence Livermore National Security, LLC - Applicants: Lawrence Livermore National Security, LLC - Inventors: Brian Giera - Web page: https://patentgrep.com/patent/US11079745B2 ## CPC - G05B 19/41875: {characterised by quality surveillance of production} - B22F 10/85: for controlling or regulating additive manufacturing processes - B29C 64/393: for controlling or regulating additive manufacturing processes - G05B 13/0265: {the criterion being a learning criterion} - G06N 20/20: Ensemble learning - B22F 10/28: Powder bed fusion, e.g. selective laser melting [SLM] or electron beam melting [EBM] - B22F 10/38: to achieve specific product aspects, e.g. surface smoothness, density, porosity or hollow structures - B22F 12/90: Means for process control, e.g. cameras or sensors - B33Y 50/02: for controlling or regulating additive manufacturing processes - G05B 2219/32018: Adapt process as function of results of quality measuring until maximum quality - G05B 2219/33034: Online learning, training - Y02P 10/25: Process efficiency - Y02P 90/02: Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS] - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - B22F: WORKING METALLIC POWDER; MANUFACTURE OF ARTICLES FROM METALLIC POWDER; MAKING METALLIC POWDER (making alloys by powder metallurgy C22C); APPARATUS OR DEVICES SPECIALLY ADAPTED FOR METALLIC POWDER - B29C: SHAPING OR JOINING OF PLASTICS; SHAPING OF MATERIAL IN A PLASTIC STATE, NOT OTHERWISE PROVIDED FOR; AFTER-TREATMENT OF THE SHAPED PRODUCTS, e.g. REPAIRING (making preforms B29B11/00; making laminated products by combining previously unconnected layers which become one product whose layers will remain together B32B37/00 - B32B41/00) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - B33Y: ADDITIVE MANUFACTURING, i.e. MANUFACTURING OF THREE-DIMENSIONAL [3D] OBJECTS BY ADDITIVE DEPOSITION, ADDITIVE AGGLOMERATION OR ADDITIVE LAYERING, e.g. BY 3D PRINTING, STEREOLITHOGRAPHY OR SELECTIVE LASER SINTERING - Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS ## Abstract A system for rapidly adapting production of a product based on classification of production data using a classifier trained on prior production data is provided. A production control system includes a learning system and an adaptive system. The learning system trains a production classifier to label or classify previously collected production data. The adaptive system receives production data in real time and classifies the production data in real time using the production classifier. If the classification indicates a problem with the manufacturing of the product, the adaptive system controls the manufacturing to rectify the problem by taking some corrective action. The production classifier is trained using bootstrap data and corresponding example data extracted from prior production data. Once the bootstrap data is labeled, the corresponding example data is automatically labeled for use as training data. ## Claims ### Claim 1 A method performed by a computing device of generating a classifier to label data of a target data type, the data representing an activity, the method comprising: receiving bootstrap data and example data, the bootstrap data and the example data having a correlation in space or time, the bootstrap data and the example data representing the activity, the example data being of the target data type; receiving from a user a labeling of the bootstrap data to generate labeled bootstrap data; automatically labeling the example data based on the correlation to generate training data that includes the example data and labels; and training the classifier based on the training data, the classifier for inputting data of the target data type and outputting a label. ### Claim 2 (depends on 1) The method of claim 1 wherein the activity is additive manufacturing. ### Claim 3 (depends on 1) The method of claim 1 wherein the activity is in a field that is selected from a group consisting of manufacturing, oceanography, meteorology, traffic engineering, container processing, medical imaging, autonomous vehicle operations, and satellite monitoring. ### Claim 4 (depends on 1) The method of claim 1 wherein the classifier is a neural network. ### Claim 5 (depends on 1) The method of claim 1 wherein the classifier is selected from a group consisting of a support vector machine, a decision tree, a k-nearest neighbor algorithm, a linear regression algorithm, a relevance vector machine, and ensembles of classifiers. ### Claim 6 A method performed by a computing device of generating a classifier to label data of a target data type, the data representing an activity, the method comprising: receiving bootstrap data and example data, the bootstrap data and the example data having a correlation in space or time, the bootstrap data and the example data representing the activity, the example data being of the target data type; receiving from a user a labeling of the bootstrap data to generate labeled bootstrap data; automatically labeling the example data based on the correlation to generate training data; and training the classifier using a machine learning technique based on the training data wherein the activity is a manufacturing process and the bootstrap data is snapshot data of the manufacturing process and the example data is video data of the manufacturing process. ### Claim 7 (depends on 6) The method of claim 6 further comprising applying the trained classifier to label newly collected video data during the manufacturing process and providing instructions to alter the manufacturing process in real time. ### Claim 8 (depends on 7) The method of claim 7 wherein a label indicates whether a characteristic of a product is acceptable or not acceptable. ### Claim 9 (depends on 8) The method of claim 8 wherein labels that indicate that the characteristic is not acceptable include a label indicating that manufacturing of the product should be aborted and a label indicating that the characteristic is correctable during the manufacturing process. ### Claim 10 A method performed by a computing device of generating a neural network classifier to label data of a target data type, the data representing an activity, the method comprising: receiving bootstrap data and example data, the bootstrap data and the example data having a correlation in space or time, the bootstrap data and the example data representing the activity, the example data being of the target data type; receiving from a user a labeling of the bootstrap data to generate labeled bootstrap data; automatically labeling the example data based on the correlation to generate training data; and training the neural network classifier based on the training data wherein the activity is a manufacturing process and the bootstrap data is from a first stream of sensor data of the manufacturing process and the example data is from a second stream of sensor data of the manufacturing process. ### Claim 11 (depends on 10) The method of claim 10 further comprising applying the trained neural network classifier to label newly collected example data during the manufacturing process and providing instructions to alter the manufacturing process in real time. ### Claim 12 (depends on 11) The method of claim 11 wherein a label indicates whether a characteristic of a product is acceptable or not acceptable. ### Claim 13 (depends on 12) The method of claim 12 wherein labels that indicate that the characteristic is not acceptable include a label indicating that manufacturing of the product should be aborted and a label indicating that the characteristic is correctable during the manufacturing process. ### Claim 14 A method performed by a computing device of generating a classifier to label data of a target data type, the method comprising: receiving seed data having a seed data type; receiving from a user a labeling of the seed data to generate labeled seed data; training a bootstrap classifier to label data of the seed data type using the labeled seed data; receiving bootstrap data and example data, the bootstrap data and example data representing different data streams, the bootstrap data being of the seed data type and the example data being of the target data type; applying the bootstrap classifier to label the bootstrap data automatically labeling the example data based on the labeled bootstrap data to generate training data; and training a production classifier to label data of the target data type using the training data. ### Claim 15 (depends on 14) The method of claim 14 wherein the seed data, the bootstrap data, and the example data are collected during different occurrences of same activity. ### Claim 16 (depends on 15) The method of claim 15 further comprising applying the production classifier to label newly collected video data during a manufacturing process and providing instructions to alter the manufacturing process in real time. ### Claim 17 (depends on 16) The method of claim 16 wherein a label indicates whether a characteristic of a product is acceptable or not acceptable. ### Claim 18 (depends on 17) The method of claim 17 wherein labels that indicate that the characteristic is not acceptable include a label indicating that manufacturing of the product should be abandoned and a label indicating that the characteristic is correctable during the manufacturing process. ### Claim 19 (depends on 14) The method of claim 14 wherein the bootstrap data and the corresponding example data are collected during the same occurrence of an activity. ### Claim 20 (depends on 14) The method of claim 14 wherein the bootstrap data is from a first stream of sensor data of the activity and the example data is from a second stream of sensor data of the activity. ### Claim 21 (depends on 14) The method of claim 14 wherein the seed data, the bootstrap data, and the example data are collected during additive manufacturing. ### Claim 22 (depends on 14) The method of claim 14 wherein the received data relates to a field that is selected from a group consisting of manufacturing, oceanography, meteorology, traffic engineering, container processing, medical imaging, autonomous vehicle operations, and satellite monitoring. ### Claim 23 (depends on 14) The method of claim 14 wherein the classifier is a neural network. ### Claim 24 A method performed by a computing device of generating a classifier to label data of a target data type, the method comprising: receiving seed data having a seed data type; receiving from a user a labeling of the seed data to generate labeled seed data; training a bootstrap classifier to label data of the seed data type by applying a machine leaning technique with the labeled seed data as input; receiving bootstrap data and example data, the bootstrap data and example data representing different data streams, the bootstrap data being of the seed data type and the example data being of the target data type; applying the bootstrap classifier to label the bootstrap data to generate training data; and training a production classifier to label data of the target data type by applying a machine learning technique with training data as input wherein the bootstrap data is snapshot data of a manufacturing process and the example data is video data of the manufacturing process. ### Claim 25 A method performed by a computing device of generating a classifier to label data of a target data type, the method comprising: receiving seed data having a seed data type; receiving from a user a labeling of the seed data to generate labeled seed data; training a bootstrap classifier to label data of the seed data type using the labeled seed data; receiving bootstrap data and example data, the bootstrap data and example data representing different data streams, the bootstrap data being of the seed data type and the example data being of the target data type; applying the bootstrap classifier to label the bootstrap data to generate training data; training a neural network as a production classifier to label data of the target data type using the training data; and applying the production classifier to label newly collected example data during the activity and providing instructions to alter the activity in real time wherein the bootstrap data is from a first stream of sensor data of the activity and the example data is from a second stream of sensor data of the activity. ### Claim 26 A method performed by a computing device of generating a classifier to label data of a target data type, the method comprising: receiving seed data having a seed data type; receiving from a user a labeling of the seed data to generate labeled seed data; training a bootstrap classifier to label data of the seed data type using the labeled seed data; receiving bootstrap data and example data, the bootstrap data and example data representing different data streams, the bootstrap data being of the seed data type and the example data being of the target data type; applying the bootstrap classifier to label the bootstrap data to generate training data; training a production classifier to label data of the target data type using the training data; and applying the production classifier to label newly collected example data during the activity and providing instructions to alter the activity in real time wherein the bootstrap classifier and the production classifier are selected from a group consisting of a support vector machine, a decision tree, a k-nearest neighbor algorithm, a linear regression algorithm, a relevance vector machine, and ensembles of classifiers wherein the bootstrap data is from a first stream of sensor data of the activity and the example data is from a second stream of sensor data of the activity and wherein a label indicates whether a characteristic of an output of the activity is acceptable or not acceptable. ### Claim 27 (depends on 26) The method of claim 26 wherein labels that indicate that the characteristic is not acceptable include a label indicating that the activity should be abandoned and a label indicating that the characteristic is correctable during the activity. ### Claim 28 One or more computing devices for generating a classifier to control an additive manufacturing process, a computing device comprising: one or more memories storing computer-executable instructions for controlling a computing device, the instructions including: a production classifier to label production data with a label, the production classifier being trained using labeled example data as training data, the labeled example data being automatically labeled based on labeled bootstrap data, the bootstrap data and the example data collected from sensors of an additive manufacturing process, the example data being of a target data type; and a component that receives production data from sensors during an additive manufacturing process, the production data being of the target data type; applies the production classifier to label the production data; and controls in real time the additive manufacturing process based on the labels; and one or more processors for executing the instructions stored in the one or more memories. ### Claim 29 (depends on 28) The one or more computing devices of claim 28 wherein a label of the production data indicates whether the product is acceptable and, if it is not acceptable, whether a problem with the product is correctable or not. ### Claim 30 (depends on 28) The one or more computing devices of claim 28 wherein the instructions further comprise instructions including: a bootstrap classifier to label bootstrap data, the bootstrap classifier being trained using labeled seed data as training data, the seed data being collected from sensors of the additive manufacturing process; and a component to automatically label the bootstrap data using the bootstrap classifier. ### Claim 31 (depends on 30) The one or more computing devices of claim 30 wherein the instructions further comprise instructions including a component to retrain the production classifier using bootstrap data and example data of current production data, the bootstrap data of the current production data being labeled using the bootstrap classifier. ### Claim 32 One or more computing systems for controlling production equipment during an additive manufacturing process, the one or more computing systems comprising: one or more computer-readable storage mediums storing computer-executable instructions for controlling the one or more computing system to: access a production classifier to label production data with a label, the production classifier being trained using machine learning with labeled example data as training data, the labeled example data being automatically labeled based on labeled bootstrap data, the bootstrap data and the example data collected from sensors of an additive manufacturing process, the example data being of a target data type; and receive production data from sensors of the production equipment during an additive manufacturing process, the production data being of the target data type; apply the production classifier to label the production data; and control in real time the production equipment during the additive manufacturing process based on the labels; and one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage mediums. ### Claim 33 One or more computing systems for training a neural network for use in controlling production equipment during manufacturing process, the one or more computing systems comprising: one or more computer-readable storage mediums storing computer-executable instructions for controlling the one or more computing system to: access bootstrap data and example data, the bootstrap data and the example data having a correlation in space or time, the bootstrap data and the example data representing the activity, the example data being of the target data type, the bootstrap data and the example data collected from production equipment during a manufacturing process; receive from a user a labeling of the bootstrap data to generate labeled bootstrap data; automatically label the example data based on the correlation to generate training data; and train the neural network based on the training data; and one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage mediums. ### Claim 34 One or more computing systems for identifying whether corrective action is needed when manufacturing a product, the one or more computing systems comprising: one or more computer-readable storage mediums storing computer-executable instructions for controlling the one or more computing system to: access a production classifier for labelling production data to indicate whether corrective action is needed, the production classifier being trained using labeled example data as training data, the labeled example data being automatically labeled based on labeled bootstrap data; and receive production data from sensors during the manufacturing of the product; and apply the production classifier to label the production data to indicate whether corrective action is needed; and one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage mediums.