# US 2022/0032470 A1 · Artificial intelligence-based process identification, extraction, and automation for robotic process automation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0032470 A1
- Document ID: US20220032470A1
- Status: Published application
- Kind: A1
- Application number: 17/506,219
- Filed: 2021-10-20
- Published: 2022-02-03
- Earliest claimed: 2019-10-15
- Assignees: UiPath, Inc.
- Applicants: UiPath, Inc.
- Inventors: Prabhdeep SINGH; Christian BERG
- Web page: https://patentgrep.com/patent/US20220032470A1

## CPC

- B25J 13/006: {by means of a wireless system for controlling one or several manipulators}
- G06N 20/00: Machine learning
- G06F 8/30: Creation or generation of source code
- G06F 9/451: Execution arrangements for user interfaces
- G06F 11/3438: {monitoring of user actions (tracking the activity of the user H04L67/535)}
- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06F 11/3476: {Data logging (G06F11/14, G06F11/2205 take precedence)}
- G06Q 10/0633: Workflow analysis
- G06Q 10/10: Office automation; Time management
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Artificial intelligence (AI)-based process identification, extraction, and automation for robotic process automation (RPA) is disclosed. Listeners may be deployed to user computing systems to collect data pertaining to user actions. The data collected by the listeners may then be sent to one or more servers and be stored in a database. This data may be analyzed by AI layers to recognize patterns of user behavioral processes therein. These recognized processes may then be distilled into respective RPA workflows and deployed to automate the processes.

## Claims

### Claim 1

A cloud robotic process automation (RPA) system, comprising:
one or more cloud computing systems configured to perform artificial intelligence (AI)-based process identification, extraction, and automation for RPA; and
a plurality of user computing systems comprising respective listener applications, the listener applications configured to generate logs comprising user interactions with respective user computing systems and send log data from the logs to the one or more cloud computing systems, wherein
the one or more cloud computing systems are configured to:
access log data collected from the listeners and run the log data through at least one AI layer, the at least one AI layer configured to process the log data and identify a potential RPA process therein, and
automatically generate an RPA workflow including the identified RPA process.

### Claim 2 (depends on 1)

The cloud RPA system of claim 1, wherein the one or more cloud computing systems are further configured to:
generate an RPA robot implementing the RPA workflow; and
push the generated RPA robot out to the user computing systems to be executed thereon.

### Claim 3 (depends on 1)

The cloud RPA system of claim 1, wherein the at least one AI layer comprises a sequence extraction layer, a clustering detection layer, a visual component detection layer, a text recognition layer, an audio-to-text translation layer, or any combination thereof.

### Claim 4 (depends on 1)

The cloud RPA system of claim 1, wherein
the one or more cloud computing systems comprise a conductor application, and
the log data is sent periodically to the conductor application by the listeners of the plurality of user computing systems as part of a heartbeat message.

### Claim 5 (depends on 1)

The cloud RPA system of claim 1, wherein the log data is sent to the one or more cloud computing systems once a predetermined amount of log data has been collected, after a predetermined time period has elapsed, or both.

### Claim 6 (depends on 1)

The cloud RPA system of claim 1, further comprising:
a database communicably coupled to the one or more cloud computing systems, wherein
the one or more cloud computing systems are configured to store the received log data from the listeners in the database.

### Claim 7 (depends on 1)

The cloud RPA system of claim 1, wherein the one or more cloud computing systems are further configured to:
determine similarities between the identified RPA process and a previously existing RPA process executed by an RPA robot on the user computing systems; and
replace an RPA workflow of the RPA robot executing the previously existing RPA process with the identified RPA process.

### Claim 8 (depends on 7)

The cloud RPA system of claim 7, wherein similarities between the identified RPA process and the previously existing RPA process are determined by a common beginning, a common end, and a predetermined amount of statistical commonality in steps taking in between in the RPA processes.

### Claim 9 (depends on 1)

The cloud RPA system of claim 1, wherein the listeners are configured to record where a user clicks on a screen and in which application, keystrokes, which button was clicked, instances of the user switching between applications, focus changes, that an email was sent and what the email pertains to, or any combination thereof.

### Claim 10 (depends on 1)

The cloud RPA system of claim 1, wherein the listeners are configured to determine which applications users of the user computing systems are using, what percentage of time the users are using a given application, which features within the applications the users are using, which features within the applications that the users are not using, or any combination thereof.

### Claim 11

A computer program configured to perform artificial intelligence (AI)-based process identification, extraction, and automation for robotic process automation (RPA) embodied on a non-transitory computer-readable medium of a cloud computing system, the computer program configured to cause at least one processor to:
access log data collected via a cloud RPA system from respective listener applications of a plurality of user computing systems;
run the log data through at least one AI layer, the at least one AI layer configured to process the log data and identify a potential RPA process therein, and
automatically generate an RPA workflow including the identified RPA process.

### Claim 12 (depends on 11)

The computer program of claim 11, wherein the computer program is further configured to cause the at least one processor to:
generate an RPA robot implementing the RPA workflow; and
push the generated robot out to the user computing systems, via the cloud RPA system, to be executed thereon.

### Claim 13 (depends on 11)

The computer program of claim 11, wherein the at least one AI layer comprises a sequence extraction layer, a clustering detection layer, a visual component detection layer, a text recognition layer, an audio-to-text translation layer, or any combination thereof.

### Claim 14 (depends on 11)

The computer program of claim 11, wherein the computer program is further configured to cause the at least one processor to:
determine similarities between the identified RPA process and a previously existing RPA process executed by a robot on the user computing systems; and
replace an RPA workflow of the robot executing the previously existing RPA process with the identified RPA process, via the cloud RPA system.

### Claim 15 (depends on 11)

The computer program of claim 11, wherein similarities between the identified RPA process and the previously existing RPA process are determined by a common beginning, a common end, and a predetermined amount of statistical commonality in steps taking in between in the RPA processes.

### Claim 16

A computer-implemented method for performing artificial intelligence (AI)-based process identification, extraction, and automation for robotic process automation (RPA), comprising:
generating, by a listener, a log comprising user interactions with a user computing system; and
sending log data from the log to one or more cloud computing systems of a cloud RPA system, by the listener, wherein
the log data comprises where a user clicks on a screen and in which application, keystrokes, which button was clicked, instances of the user switching between applications, focus changes, that an email was sent and what the email pertains to, or any combination thereof.

### Claim 17 (depends on 16)

The computer-implemented method of claim 16, wherein the log data is sent periodically to a conductor application of the one or more cloud computing systems, via the cloud RPA system, as part of a heartbeat message.

### Claim 18 (depends on 16)

The computer-implemented method of claim 16, wherein the log data is sent to the one or more cloud computing systems, via the cloud RPA system, once a predetermined amount of log data has been collected, after a predetermined time period has elapsed, or both.

### Claim 19 (depends on 16)

The computer-implemented method of claim 16, wherein the user interactions comprise which applications the user is using, what percentage of time the user is using a given application, which features within the applications the user is using, which features within the applications that the user is not using, or any combination thereof.

### Claim 20 (depends on 16)

The computer-implemented method of claim 16, wherein the user interactions comprise which features within the applications the user is using and which features within the applications that the user is not using.
