# US 11,907,740 B2 · Method for creating RPA script data, method for executing RPA script data, terminal device, image processing apparatus, RPA script data, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,907,740 B2
- Document ID: US11907740B2
- Status: Granted
- Kind: B2
- Application number: 18/135,223
- Filed: 2023-04-17
- Published: 2023-11-30
- Granted: 2024-02-20
- Earliest claimed: 2022-05-31
- Assignees: Konica Minolta, Inc.
- Applicants: Konica Minolta, Inc.
- Inventors: Yuichi Kawamura
- Web page: https://patentgrep.com/patent/US11907740B2

## CPC

- G06F 9/451: Execution arrangements for user interfaces
- G06F 8/34: Graphical or visual programming
- G06F 9/45512: {Command shells}
- G06F 11/3041: {where the computing system component is an input/output interface (interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units G06F13/00)}
- G06F 11/3438: {monitoring of user actions (tracking the activity of the user H04L67/535)}
- G06Q 10/10: Office automation; Time management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A method for creating robotic process automation (RPA) script data to perform an operation in a terminal device that displays an operation screen for an image processing apparatus on a host terminal and receives the operation by the RPA, includes: detecting a position of an operation target displayed on the operation screen and selected by an operator; and normalizing the detected position of the operation target on a basis of a size of the operation screen, generating position information indicating the normalized position, and describing the position information in the RPA script data.

## Claims

### Claim 1

A method for creating robotic process automation (RPA) script data to perform an operation in a terminal device that displays an operation screen for an image processing apparatus on a host terminal and receives the operation by the RPA, the method comprising:
detecting a position of an operation target displayed on the operation screen and selected by an operator; and
normalizing the detected position of the operation target on a basis of a size of the operation screen, generating position information indicating the normalized position, and describing the position information in the RPA script data.

### Claim 2 (depends on 1)

The method for creating RPA script data according to claim 1, wherein
the terminal device performs window display of the operation screen, and
the normalization of the position of the operation target in the normalizing is performed using a display size at time of performing the window display of the operation screen and in-window coordinates of the position of the operation target to be operated.

### Claim 3 (depends on 2)

The method for creating RPA script data according to claim 2, wherein
the normalizing includes:
generating a script that acquires a size of the window display, and
generating a calculation script that calculates the in-window coordinates indicating the position of the operation target by using the acquired size of the window display and the normalized position of the operation target indicated in the position information, and describing the calculation script in the RPA script data.

### Claim 4 (depends on 3)

The method for creating RPA script data according to claim 3, wherein
the RPA script data includes resolution information indicating a screen resolution of the terminal device at time of detecting the position of the operation target to be operated, and
the calculation script generated by the normalizing uses, to calculate the in-window coordinates, a ratio between a screen resolution indicated in the resolution information of the RPA script data and the screen resolution of the terminal device at time of executing the RPA script data.

### Claim 5 (depends on 1)

The method for creating RPA script data according to claim 1, wherein
the normalizing includes
generating a calculation script that calculates coordinates in the operation screen of the image processing apparatus by using a screen resolution of the image processing apparatus and the normalized position of the operation target indicated in the position information, and describing the calculation script in the RPA script data.

### Claim 6

A terminal device, comprising:
a hardware processor that:
receives a bitmap representing an operation screen of an image forming apparatus from the image processing apparatus;
writes and displays the received bitmap in a video random access memory (VRAM);
receives, from an operator, designation of a position to be an operation target in the bitmap written in the VRAM when registering an operation to be executed by RPA; and
obtains position information indicating a designated position by normalizing the designated position according to a size of the displayed bitmap, and generates RPA script data describing the position information.