Back to results
US 6,184,880 B2Granted
Automatic GUI system operation device and operation macro execution device
AssigneeNEC Corporation
InventorHidehiko Okada
Life of this patent
Estimating the term…
Abstract
A screen of a system is comprehensively shifted by automatically operating a system having a GUI and images on the screen are recorded at respective time instances. Operation objects of the system displayed on the screen is detected by an operation object detection unit (102) and sequentially selected by an operation object selection unit (108) such that the detected operation objects are comprehensively operated. An operation event for executing an operation of the selected operation object is generated by an operation event generator unit (109) and output to a program execution unit (101) for controlling an execution of an execution program of the system. A screen image recording unit (103) acquires the screen image from the program execution unit (101) and records it.
Claims10
An automatic GUI system operation device that automatically operates a system having a GUI, and that is connected to program execution means for controlling an execution program of said system, and that is connected to operation object detection means for detecting operation objects of said system displayed on a display screen by said program execution means, comprising:
- program execution start means for making said program execution means start an execution of said execution program of said system;
- screen image recording means for recording an image on said display screen displayed by said program execution means;
- operation object table recording means for producing tables of operation objects from data of operation objects detected by said operation object detection means and recording the tables of operation objects;
- part table stack managing means for adding identifiers to a table stack of identifiers of the tables of operation objects or deleting identifiers from the table stack of identifiers of the tables of operation objects;
- operation object selection means for selecting data of the operation objects from the tables of operation objects recorded in said operation object recording means;
- operation event generation means for generating an event for executing an operation of operation objects of said system;
- and
- program execution end means for ending the execution of said execution program of said system which is being executed by said program execution means,
- wherein said operation object table recording means extracts data which is not recorded in said operation object table recording means from the data of the operation objects detected by said operation object detection means, produces the tables of operation objects by using the data and records the tables, instructs said table stack managing means to add identifiers of the newly recorded operation object table to the table stack, and, in a case where all of the data of the operation objects detected by said operation object detection means have been recorded in said operation object table recording means already, instructs said program execution means to terminate the execution of the execution program of said system which is being executed by means of said program execution means and, in a case where, in the data of the operation objects detected by said operation object detection means, there are data which are not recorded in said operation object table recording means as yet, instructs said operation object selection means to select data of operation objects from the tables of operation objects recorded in said operation object table recording means,
- wherein said operation object selection means searches the tables of operation objects having the top identifier in the table stack from said operation object table recording means, searches data of operation objects which are not operated as yet from the searched tables of operation objects and instructs said table stack managing means to select one of data of operation objects which are not operated as yet, if any, or to select one of data of operation objects from the tables of operation objects recorded in said operation object table recording means if there is no such data, or to delete the identifier in the top of the table stack until all identifiers in the table stack are deleted and then repeats the search processing of the data of the non-operated operation objects,
- wherein said operation event generator means generates an event for executing an operation of operation objects of said system using the data of operation objects selected by said operation object selection means and outputs it to said program execution means,
- wherein said program execution end means instructs said program execution start means to start the execution of said execution program of said system after the execution of said execution program of said system, which is being executed by said program execution means, is ended,
- wherein said program execution start means instructs said operation object selection means to select the data of operation objects, after it instructs said program execution means to start the execution of said execution program in response to the instruction from said program execution end means, and
- wherein, when one of data of non operated operation objects is selected in response to the instruction from said program execution start means instructing the selection of data of operation objects, said operation object selection means searches, from the tables of operation objects recorded in said operation object table recording means, data of operation objects to be executed in order to shift a content on said display screen from an initial content of said system till a content displaying the selected data of operation objects.
11 more elements
An automatic GUI system operation device that automatically operates a system having a GUI, and that is connected to a program execution unit for controlling an execution program of said system, and that is connected to operation object detection unit for detecting operation objects of said system displayed on a display screen by said program execution unit, comprising:
- a program execution start unit for making said program execution unit start an execution of said execution program of said system;
- a screen image recording unit for recording an image on said display screen displayed by said program execution unit;
- an operation object table recording unit for producing tables of operation objects from data of operation objects detected by said operation object detection unit and recording the tables of operation objects;
- a part table stack managing unit for adding identifiers to a table stack of identifiers of the tables of operation objects or deleting identifiers from the table stack of identifiers of the tables of operation objects;
- an operation object selection unit for selecting data of the operation objects from the tables of operation objects recorded in said operation object recording unit;
- an operation event generation unit for generating an event for executing an operation of operation objects of said system;
- and
- a program execution end unit for ending the execution of said execution program of said system which is being executed by said program execution unit,
- wherein said operation object table recording unit extracts data which is not recorded in said operation object table recording unit from the data of the operation objects detected by said operation object detection unit, produces the tables of operation objects by using the data and records the tables, instructs said table stack managing unit to add identifiers of the newly recorded operation object table to the table stack, and, in a case where all of the data of the operation objects detected by said operation object detection unit have been recorded in said operation object table recording unit already, instructs said program execution unit to terminate the execution of the execution program of said system which is being executed by unit of said program execution unit and, in a case where, in the data of the operation objects detected by said operation object detection unit, there are data which are not recorded in said operation object table recording unit as yet, instructs said operation object selection unit to select data of operation objects from the tables of operation objects recorded in said operation object table recording unit,
- wherein said operation object selection unit searches the tables of operation objects having the top identifier in the table stack from said operation object table recording unit, searches data of operation objects which are not operated as yet from the searched tables of operation objects and instructs said table stack managing unit to select one of data of operation objects which are not operated as yet, if any, or to select one of data of operation objects from the tables of operation objects recorded in said operation object table recording unit if there is no such data, or to delete the identifier in the top of the table stack until all identifiers in the table stack are deleted and then repeats the search processing of the data of the non-operated operation objects.
7 more elements
Dependencies are inferred from the claim text.