# US 11,934,175 B2 · Robotic process automation for remote assistance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,934,175 B2 - Document ID: US11934175B2 - Status: Granted - Kind: B2 - Application number: 17/061,999 - Filed: 2020-10-02 - Published: 2022-04-07 - Granted: 2024-03-19 - Earliest claimed: 2020-10-02 - Assignees: Sap SE - Applicants: Sap SE - Inventors: Ganna Monakova - Web page: https://patentgrep.com/patent/US11934175B2 ## CPC - G05B 19/4155: characterised by program execution, i.e. part program or machine function execution, e.g. selection of a program - G06F 9/451: Execution arrangements for user interfaces - G05B 2219/31151: Lan local area network - G05B 2219/31156: Network structure, internet - G05B 2219/50391: Robot - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed herein are system, method, and computer program product embodiments for remotely executing a robotic process automation (RPA) bot as a function. An embodiment operates by generating an RPA bot function in a first network environment and transmitting the RPA bot function to an application located in a second network environment. The embodiment further operates by receiving results data from the application based on an execution of the RPA bot function in the second network environment. Subsequently, the embodiment operates by generating results user interface (UI) data based on the results data and transmitting the results UI data to a user input/output (I/O) device located in the first network environment. ## Claims ### Claim 1 A computer-implemented method for remotely executing a robotic process automation bot as a function, comprising: receiving, at a first device, one or more screenshots for an application, wherein the application is not installed on the first device; generating, on the first device connected to a first network environment, a robotic process automation bot function for the application based on the one or more screenshots, wherein the robotic process automation bot function comprises one or more actions to be performed on the application; transmitting, by the first device, the robotic process automation bot function to the application executable from a second device connected to a second network environment; receiving, by the at least one processor, results data from the application based on an execution of the robotic process automation bot function in the second network environment; generating, by the at least one processor, results user interface data based on the results data; and transmitting, by the at least one processor, the results user interface data to a user input/output device located in the first network environment. ### Claim 2 (depends on 1) The method of claim 1, wherein the generating the robotic process automation bot function comprises: capturing, by the at least one processor, a set of user actions performed on a set of live applications or application screenshots using the user input/output device; and generating, by the at least one processor, the robotic process automation bot function based on the set of user actions. ### Claim 3 (depends on 2) The method of claim 2, wherein the generating the robotic process automation bot function comprises: generating, by the at least one processor, the robotic process automation bot function according to an on-the-fly robotic process automation remote assistance hot specification generation technique. ### Claim 4 (depends on 1) The method of claim 1, further comprising: generating, by at least one processor, input user interface data based on the robotic process automation bot function or one or more robotic process automation hot function parameters; and transmitting, by the at least one processor, the input user interface data to the user input/output device. ### Claim 5 (depends on 4) The method of claim 4, further comprising: receiving, by the at least one processor, input data generated by a user using the user input/output device in response to an electronic presentation of an input user interface generated by the user input/output device based on the input user interface data; and transmitting, by the at least one processor, the robotic process automation bot function and the input data to the application located in the second network environment. ### Claim 6 (depends on 1) The method of claim 1, wherein: the first network environment is a local network environment; and the second network environment is a remote network environment. ### Claim 7 (depends on 1) The method of claim 1, wherein the robotic process automation bot function corresponds to a robotic process automation hot remote assistant execution mode. ### Claim 8 (depends on 1) The method of claim 1, wherein the receiving one or more screenshots comprises: generating, by the first device, an input marker, on an element of a first screenshot of the one or more screenshots, wherein the input marker causes the robotic process automation to display an input user interface to collect data associated with the element. ### Claim 9 A system, comprising: a memory; and at least one processor coupled to the memory and configured to: receive, at a first device, one or more screenshots for an application, wherein the application is not installed on the first device; generate, on the first device connected to a first network environment, a robotic process automation bot function for the application based on the one or more screenshots, wherein the robotic process automation bot function comprises one or more actions to be performed on the application; transmit, by the first device, the robotic process automation bot function to the application executable from a second device connected to a second network environment; receive results data from the application based on an execution of the robotic process automation bot function in the second network environment; generate results user interface data based on the results data; and transmit the results user interface data to a user input/output device located in the first network environment. ### Claim 10 (depends on 9) The system of claim 9, wherein to generate the robotic process automation bot function, the at least one processor is configured to: capture a set of user actions performed on a set of live applications or application screenshots using the user input/output device; and generate the robotic process automation bot function based on the set of user actions. ### Claim 11 (depends on 10) The system of claim 10, wherein to generate the robotic process automation bot function, the at least one processor is configured to: generate the robotic process automation bot function according to an on-the-fly robotic process automation remote assistance bot specification generation technique. ### Claim 12 (depends on 9) The system of claim 9, the at least one processor further configured to: generate input user interface data based on the robotic process automation bot function or one or more robotic process automation bot function parameters; and transmit the input user interface data to the user input/output device. ### Claim 13 (depends on 12) The system of claim 12, the at least one processor further configured to: receive input data generated by a user using the user input/output device in response to an electronic presentation of an input user interface generated by the user input/output device based on the input user interface data; and transmit the robotic process automation but function and the input data to the application located in the second network environment. ### Claim 14 (depends on 9) The system of claim 9, wherein: the first network environment is a local network environment; and the second network environment is a remote network environment. ### Claim 15 A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising: receiving, at a first device, one or more screenshots for an application, wherein the application is not installed on the first device; generating, on the first device connected to a first network environment, a robotic process automation bot function for the application based on the one or more screenshots, wherein the robotic process automation bot function comprises one or more actions to be performed on the application; transmitting, by the first device, the robotic process automation bot function to the application executable from a second device connected to a second network environment; receiving results data from the application based on an execution of the robotic process automation bot function in the second network environment; generating results user interface data based on the results data; and transmitting the results user interface data to a user input/output device located in the first network environment. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the generating the robotic process automation bot function comprises: capturing a set of user actions performed on a set of live applications or application screenshots using the user input/output device; and generating the robotic process automation bot function based on the set of user actions. ### Claim 17 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the generating the robotic process automation bot function comprises: generating the robotic process automation bot function according to an on-the-fly robotic process automation remote assistance bot specification generation technique. ### Claim 18 (depends on 15) The non-transitory computer-readable medium of claim 15, the operations further comprising: generating input user interface data based on the robotic process automation bot function or one or more robotic process automation bot function parameters; and transmitting the input user interface data to the user input/output device. ### Claim 19 (depends on 18) The non-transitory computer-readable medium of claim 18, the operations further comprising: receiving input data generated by a user using the user input/output device in response to an electronic presentation of an input user interface generated by the user input/output device based on the input user interface data; and transmitting the robotic process automation bot function and the input data to the application located in the second network environment. ### Claim 20 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein: the first network environment is a local network environment; the second network environment is a remote network environment; and the robotic process automation hot function corresponds to a robotic process automation bot remote assistant execution mode.