# US 11,907,621 B2 · Electronic product testing systems for providing automated product testing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,907,621 B2
- Document ID: US11907621B2
- Status: Granted
- Kind: B2
- Application number: 16/403,359
- Filed: 2019-05-03
- Published: 2020-11-05
- Granted: 2024-02-20
- Earliest claimed: 2019-05-03
- Assignees: Rainforest QA, Inc.
- Applicants: Rainforest QA, Inc.
- Inventors: Maciej Siwek
- Web page: https://patentgrep.com/patent/US11907621B2

## CPC

- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 30/20: Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30)
- G06F 8/34: Graphical or visual programming
- G06F 9/45512: {Command shells}
- G06F 11/3688: {for test execution, e.g. scheduling of test suites}
- G06F 11/3696: {Methods or tools to render software testable}
- G06F 11/3698: Environments for analysis, debugging or testing of software
- G06Q 10/0639: Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

An electronic product testing system includes: a testing instruction generator configured to provide a user interface for generating an electronic file, the electronic file containing product testing instruction; and a testing device having a processing unit configured to execute the electronic file to perform testing of a product based on the product testing instruction in the electronic file; wherein the processing unit is configured to perform testing of the product by simulating human actions based on the product testing instruction in the electronic file.

## Claims

### Claim 1

An electronic product testing system, comprising:
a testing instruction generator configured to provide a user interface for generating an electronic file, the electronic file containing product testing instruction; and
a product testing device having a processing unit configured to execute the electronic file to perform testing of a product based on the product testing instruction in the electronic file;
wherein the processing unit is configured to perform testing of the product by simulating human actions based on the product testing instruction in the electronic file;
wherein the user interface provided by the testing instruction generator is configured to allow a user to input an image for use by the processing unit as a reference image when the processing unit performs the testing of the product, wherein the image for use by the processing unit as the reference image comprises an image-capture of a part of a screen that is less than an entirety of the screen;
wherein the user interface provided by the testing instruction generator is configured to allow the user to capture the part of the screen to obtain the image by creating a boundary circumscribing the part of the screen, and wherein the the electronic product testing system is configured to store the captured part of the screen in association with the product testing instruction; and
wherein the user interface provided by the testing instruction generator is configured to provide a list of testing actions for selection by the user before the testing of the product is performed, obtain a selected one of the testing actions, and allow the user to capture the part of the screen to obtain the image after the selected one of the testing actions is obtained and before the testing of the product is performed.

### Claim 2 (depends on 1)

The electronic product testing system of claim 1, wherein the processing unit is configured to move a cursor without input from a cursor control.

### Claim 3 (depends on 1)

The electronic product testing system of claim 1, wherein the processing unit is configured to make a selection of an object without input from a cursor control.

### Claim 4 (depends on 1)

The electronic product testing system of claim 1, wherein the product testing device is configured to locate a text field of the product based on at least a part of the product testing instruction, and automatically type a text in the text field without input from a keyboard.

### Claim 5 (depends on 4)

The electronic product testing system of claim 4, wherein the product testing device is configured to automatically click a button after the text is automatically typed in the text field.

### Claim 6 (depends on 1)

The electronic product testing system of claim 1, wherein the processing unit comprises an interpreter configured to interpret the product testing instruction in the electronic file.

### Claim 7 (depends on 1)

The electronic product testing system of claim 1, further comprising a script generator configured to generate a script based on the product testing instruction in the electronic file.

### Claim 8 (depends on 7)

The electronic product testing system of claim 7, wherein the processing unit is configured to run the script for testing the product.

### Claim 9 (depends on 1)

The electronic product testing system of claim 1, wherein the user interface comprises a graphical user interface providing a first set of items and a second set of items for user-selection, wherein the first set of items comprises a plurality of action identifiers, and the second set of items comprises a plurality of objects.

### Claim 10 (depends on 9)

The electronic product testing system of claim 9, wherein one of the action identifiers identifies an action to be performed by the product testing device, and one of the object identifiers identifies an object on which the action is to be performed by the product testing device.

### Claim 11 (depends on 9)

The electronic product testing system of claim 9, wherein one of the action identifiers identifies a click action, a fill action, a type action, a press key action, a hover action, a dropdown select action, a checkbox check action, a checkbox uncheck action, a refresh action, a navigate action, a new tab action, a close tab action, a scroll cation, a drag and drop action, or a click and hold action.

### Claim 12 (depends on 9)

The electronic product testing system of claim 9, wherein one of the object identifiers identifies a button, a field, a dropdown menu, a dropdown option, a link, an icon, a checkbox, a header, a window, a text, a modal, or an user interface element.

### Claim 13 (depends on 1)

The electronic product testing system of claim 1, wherein the product testing instruction in the electronic file has a data structure that associates an action identifier with a corresponding object identifier.

### Claim 14 (depends on 13)

The electronic product testing system of claim 13, wherein the action identifier identifies an action to be performed by the product testing device, and the object identifier identifies an object on which the action is to be performed by the product testing device.

### Claim 15 (depends on 13)

The electronic product testing system of claim 13, wherein the action identifier identifies a click action, a fill action, a type action, a press key action, a hover action, a dropdown select action, a checkbox check action, a checkbox uncheck action, a refresh action, a navigate action, a new tab action, a close tab action, a scroll cation, a drag and drop action, or a click and hold action.

### Claim 16 (depends on 13)

The electronic product testing system of claim 13, wherein the object identifier identifies a button, a field, a dropdown menu, a dropdown option, a link, an icon, a checkbox, a header, a window, a text, a modal, or an user interface element.

### Claim 17 (depends on 1)

The electronic product testing system of claim 1, further comprising a non-transitory medium storing the electronic file in association with an identity of the product.

### Claim 18 (depends on 1)

The electronic product testing system of claim 1, wherein the product testing device is configured to check if an element is visible after the processing unit performs a testing action.

### Claim 19 (depends on 1)

The electronic product testing system of claim 1, wherein the product testing device is configured to check if an element is not visible after the processing unit performs a testing action.

### Claim 20 (depends on 1)

The electronic product testing system of claim 1, wherein the product testing device is configured to check if a specified page has loaded after the processing unit performs a testing action.

### Claim 21 (depends on 1)

The electronic product testing system of claim 1, wherein the processing unit is configured to test multiple features of the product based on the product testing instruction in the electronic file.

### Claim 22 (depends on 1)

The electronic product testing system of claim 1, wherein the processing unit is configured to test a first feature of the product based on the product testing instruction in the electronic file.

### Claim 23 (depends on 22)

The electronic product testing system of claim 22, wherein the processing unit is also configured to test a second feature of the product based on product testing instruction in another electronic file.

### Claim 24 (depends on 1)

The electronic product testing system of claim 1, wherein the testing instruction generator comprises a control for allowing the user to select between viewing the product testing instruction in plain language or viewing the product testing instruction in a programming format.

### Claim 25 (depends on 1)

The system of claim 1, wherein the processing unit is configured to:
obtain a first image that is associated with the testing of the product, wherein the first image is based on a completion of a first task performed during a product testing; and
determine whether at least a part of the first image matches the reference image.

### Claim 26 (depends on 25)

The electronic product testing system of claim 25, wherein the product testing device is configured to determine the product as failing at least a part of the product testing if no part of the first image matches the reference image.

### Claim 27 (depends on 25)

The electronic product testing system of claim 25, wherein the first image comprises a first content of the product, the first content indicating a first result of a first task for testing the product.

### Claim 28 (depends on 1)

The electronic product testing system of claim 1, further comprising a tracker configured to track an interaction of the product testing device with the product.

### Claim 29 (depends on 28)

The electronic product testing system of claim 28, wherein the tracker is configured to track a movement of a cursor operated by the testing device.

### Claim 30 (depends on 28)

The electronic product testing system of claim 28, wherein the tracker is configured to track a selection of a tab, a selection of a button, a selection of an icon, a selection of a text, or any combination of the foregoing, by the testing device.

### Claim 31 (depends on 28)

The electronic product testing system of claim 28, wherein the tracker is configured to track a text input by the testing device.

### Claim 32 (depends on 1)

The electronic product testing system of claim 1, wherein the product comprises a web page, a web site, a computer application, a mobile device application, or a processor application.

### Claim 33 (depends on 1)

The electronic product testing system of claim 1, wherein the processing unit is configured to perform machine-based testing of the product.

### Claim 34 (depends on 1)

The electronic product testing system of claim 1, wherein the user interface provided by the testing instruction generator is configured to obtain the image-capture of the part of the screen as the image associated with the product.

### Claim 35 (depends on 1)

The electronic product testing system of claim 1, wherein the user interface provided by the testing instruction generator is configured to obtain the image based on a user-defined area in a screen.

### Claim 36 (depends on 1)

The electronic product testing system of claim 1, wherein the image is stored in association with a task to be performed by the product testing device.

### Claim 37

An electronic system, comprising:
a testing instruction generator configured to provide a user interface for generating an electronic file, the electronic file containing product testing instruction;
wherein the electronic file is processable by a product testing device to perform testing of a product based on the product testing instruction in the electronic file;
wherein the user interface provided by the testing instruction generator is configured to allow a user to input an image for use by the product testing device as a reference image when the product testing device performs the testing of the product, wherein the image for use by the product testing device as the reference image comprises an image-capture of a part of a screen that is less than an entirety of the screen;
wherein the user interface provided by the testing instruction generator is configured to allow the user to capture the part of the screen to obtain the image by creating a boundary circumscribing the part of the screen, and wherein the electronic system is configured to store the captured part of the screen in association with the product testing instruction; and
wherein the user interface provided by the testing instruction generator is configured to provide a list of testing actions for selection by the user, obtain a selected one of the testing actions, and allow the user to capture the part of the screen to obtain the image after the selected one of the testing actions is obtained and before the testing of the product is performed.

### Claim 38 (depends on 37)

The electronic system of claim 37, wherein the user interface comprises a graphical user interface providing a first set of items and a second set of items for user-selection, wherein the first set of items comprises a plurality of action identifiers, and the second set of items comprises a plurality of objects.

### Claim 39 (depends on 38)

The electronic system of claim 38, wherein one of the action identifiers identifies an action to be performed by the product testing device, and one of the object identifiers identifies an object on which the action is to be performed by the product testing device.

### Claim 40 (depends on 38)

The electronic system of claim 38, wherein one of the action identifiers identifies a click action, a fill action, a type action, a press key action, a hover action, a dropdown select action, a checkbox check action, a checkbox uncheck action, a refresh action, a navigate action, a new tab action, a close tab action, a scroll cation, a drag and drop action, or a click and hold action.

### Claim 41 (depends on 38)

The electronic system of claim 38, wherein one of the object identifiers identifies a button, a field, a dropdown menu, a dropdown option, a link, an icon, a checkbox, a header, a window, a text, a modal, or an user interface element.

### Claim 42 (depends on 37)

The electronic system of claim 37, wherein the product testing instruction in the electronic file has a data structure that associates an action identifier with a corresponding object identifier.

### Claim 43 (depends on 42)

The electronic system of claim 42, wherein the action identifier identifies an action to be performed by the product testing device, and the object identifier identifies an object on which the action is to be performed by the product testing device.

### Claim 44 (depends on 42)

The electronic system of claim 42, wherein the action identifier identifies a click action, a fill action, a type action, a press key action, a hover action, a dropdown select action, a checkbox check action, a checkbox uncheck action, a refresh action, a navigate action, a new tab action, a close tab action, a scroll cation, a drag and drop action, or a click and hold action.

### Claim 45 (depends on 42)

The electronic system of claim 42, wherein the object identifier identifies a button, a field, a dropdown menu, a dropdown option, a link, an icon, a checkbox, a header, a window, a text, a modal, or an user interface element.