# US 2021/0342736 A1 · Machine learning model retraining pipeline for robotic process automation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0342736 A1 - Document ID: US20210342736A1 - Status: Published application - Kind: A1 - Application number: 16/864,000 - Filed: 2020-04-30 - Published: 2021-11-04 - Earliest claimed: 2020-04-30 - Assignees: UiPath, Inc. - Applicants: UiPath, Inc. - Inventors: Prabhdeep SINGH; Marco Alban HIDALGO; Anton MCGONNELL - Web page: https://patentgrep.com/patent/US20210342736A1 ## CPC - G06N 20/00: Machine learning - B25J 9/163: {learning, adaptive, model based, rule based expert control} - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 5/02: Knowledge representation; Symbolic representation - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) ## Abstract A machine learning (ML) model retraining pipeline for robotic process automation (RPA) is disclosed. When an ML model is deployed in a production or development environment, RPA robots send requests to the ML model when executing their workflows. When a confidence level of the ML model falls below a certain confidence, training data is collected, potentially from a large number of computing systems. The ML model is then trained using at least in part the collected training data, and a new version of the ML model is deployed. ## Claims ### Claim 1 A computer-implemented method for implementing a machine learning (ML) model retraining pipeline for robotic process automation (RPA), comprising: calling a machine learning (ML) model, by an RPA robot; receiving a result from the execution of the ML model, by the RPA robot; determining whether one or more trigger conditions are met for labeling of data for the ML model, by the RPA robot; and when the one or more trigger conditions are met: prompting a user to provide labeled data for training or retraining the ML model and sending the labeled data to a server for training or retraining of the ML model, by the RPA robot, or adding information pertaining to the result from the ML model to a queue for subsequent labeling, by the RPA robot. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the one or more trigger conditions comprise ML model performance falling below a confidence threshold, ML model results deviating from a statistical distribution, ML model results that fall outside of a previously observed range, or any combination thereof. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the one or more trigger conditions comprise receipt of an initiation command for collection of the labeled data. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprising: receiving the labeled data, by the server; and storing the labeled data in a database, by the server. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: when one or more training criteria are met, training the ML model, by the server, using the labeled data to produce a new version of the ML model; and deploying the new version of the ML model, by the server. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein the training of the ML model additionally comprises using previous training data that was used to train a version of the ML model trained prior to the new version of the ML model. ### Claim 7 (depends on 5) The computer-implemented method of claim 5, wherein the one or more training criteria comprise a predetermined time period elapsing, a predetermined amount of training data being received, a predetermined number of training data samples being received, the data falling outside of a distribution of data, or any combination thereof. ### Claim 8 (depends on 5) The computer-implemented method of claim 5, wherein the new version of the ML model replaces a previous version of the ML model. ### Claim 9 (depends on 5) The computer-implemented method of claim 5, further comprising: modifying a workflow of the RPA robot to call the new version of the ML model. ### Claim 10 (depends on 5) The computer-implemented method of claim 5, further comprising: modifying a workflow of the RPA robot to call multiple versions of the ML model, the multiple versions comprising the new version of the ML model. ### Claim 11 (depends on 10) The computer-implemented method of claim 10, further comprising: calling the new version of the ML model, by the RPA robot; and when the one or more trigger conditions are not met, calling a previous version of the ML model and selecting a best result from the current version of the ML model and the previous version of the ML model, by the RPA robot. ### Claim 12 (depends on 10) The computer-implemented method of claim 10, further comprising: calling the new version of the ML model and at least one previous version of the ML model, by the RPA robot; and selecting a best result from the current version of the ML model and the at least one previous version of the ML model, by the RPA robot. ### Claim 13 (depends on 1) The computer-implemented method of claim 1, wherein collection of the labeled data and training of the new version of the ML model is uniform for a plurality of ML models called by a plurality of RPA robots. ### Claim 14 A computer-implemented method for implementing a machine learning (ML) model retraining pipeline for robotic process automation (RPA), comprising, determining whether one or more trigger conditions are met for labeling of data for an ML model, by the RPA robot; and when the one or more trigger conditions are met: prompting a user to provide labeled data for training or retraining the ML model and sending the labeled data to a server for training or retraining of the ML model, by the RPA robot, or adding information pertaining to the result from the ML model to a queue for subsequent labeling, by the RPA robot. ### Claim 15 (depends on 14) The computer-implemented method of claim 14, wherein the one or more trigger conditions comprise ML model performance falling below a confidence threshold, ML model results deviating from a statistical distribution, ML model results that fall outside of a previously observed range, or any combination thereof. ### Claim 16 (depends on 14) The computer-implemented method of claim 14, further comprising: when one or more training criteria are met, training the ML model, by the server, using the labeled data to produce a new version of the ML model; and deploying the new version of the ML model, by the server. ### Claim 17 (depends on 16) The computer-implemented method of claim 16, wherein the one or more training criteria comprise a predetermined time period elapsing, a predetermined amount of training data being received, a predetermined number of training data samples being received, the data falling outside of a distribution of data, or any combination thereof. ### Claim 18 (depends on 16) The computer-implemented method of claim 16, further comprising: modifying a workflow of the RPA robot to call multiple versions of the ML model, the multiple versions comprising the new version of the ML model. ### Claim 19 A computer-implemented method for implementing a machine learning (ML) model retraining pipeline for robotic process automation (RPA), comprising, calling a machine learning (ML) model, by an RPA robot; receiving a result from the execution of the ML model, by the RPA robot; determining whether one or more trigger conditions are met for labeling of data for the ML model, by the RPA robot; when the one or more trigger conditions are met: prompting a user to provide labeled data for training or retraining the ML model and sending the labeled data to a server for training or retraining of the ML model, by the RPA robot, or adding information pertaining to the result from the ML model to a queue for subsequent labeling, by the RPA robot; when one or more training criteria are met, training the ML model, by the server, using the labeled data to produce a new version of the ML model; and deploying the new version of the ML model, by the server. ### Claim 20 (depends on 19) The computer-implemented method of claim 19, wherein the one or more trigger conditions comprise ML model performance falling below a confidence threshold, ML model results deviating from a statistical distribution, ML model results that fall outside of a previously observed range, or any combination thereof, and the one or more training criteria comprise a predetermined time period elapsing, a predetermined amount of training data being received, a predetermined number of training data samples being received, the data falling outside of a distribution of data, or any combination thereof.