Retraining a computer vision model for robotic process automation
AssigneeUiPath, Inc.
InventorCosmin Voicu
Life of this patent
Estimating the term…
Abstract
A Computer Vision (CV) model generated by a Machine Learning (ML) system may be retrained for more accurate computer image analysis in Robotic Process Automation (RPA). A designer application may receive a selection of a misidentified or non-identified graphical component in an image form a user, determine representative data of an area of the image that includes the selection, and transmit the representative data and the image to an image database. A reviewer may execute the CV model, or cause the CV model to be executed, to confirm that the error exists, and if so, send the image and a correct label to an ML system for retraining. While the CV model is being retrained, an alternative image recognition model may be used to identify the misidentified or non-identified graphical component.
Claims22
A non-transitory computer-readable medium storing a computer program, the computer program configured to cause at least one processor to:
- receive an image comprising a graphical element and receive representative data for the graphical element, the image and the representative data captured by a robotic process automation (RPA) designer application;
- run the image through a computer vision (CV) model that identifies graphical elements in the image;
- and
- provide a labeled image and the representative data for subsequent retraining of the CV model for a confirmed non-identified or misidentified graphical element to a machine learning system or a database accessible by the machine learning system, the machine learning system configured to retrain the CV model using the labeled image and the representative data.
A computer-implemented method, comprising:
- receiving an image comprising a graphical element and receiving representative data for the graphical element, by a computing system, the image and the representative data captured by a robotic process automation (RPA) designer application;
- running the image through a computer vision (CV) model that identifies graphical elements in the image, by the computing system;
- and
- providing a labeled image and the representative data for subsequent retraining of the CV model for a confirmed non-identified or misidentified graphical element to a machine learning system or a database accessible by the machine learning system, by the computing system, the machine learning system configured to retrain the CV model using the labeled image and the representative data.
A computing system, comprising:
- memory storing computer program instructions;
- and
- at least one processor configured to execute the computer program instructions, wherein the computer program instructions are configured to cause the at least one processor to:
- receive an image comprising a graphical element and receive representative data for the graphical element, the image and the representative data captured by a robotic process automation (RPA) designer application;
- run the image through a computer vision (CV) model that identifies graphical elements in the image;
- and
- provide a labeled image and the representative data for subsequent retraining of the CV model for a confirmed non-identified or misidentified graphical element to a machine learning system or a database accessible by the machine learning system, the machine learning system configured to retrain the CV model using the labeled image and the representative data.
Dependencies are inferred from the claim text.