# US 2018/0322004 A1 · Systems and methods for detecting anomalies in execution of computer programs > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322004 A1 - Document ID: US20180322004A1 - Status: Published application - Kind: A1 - Application number: 15/968,401 - Filed: 2018-05-01 - Published: 2018-11-08 - Earliest claimed: 2017-05-02 - Assignees: Soroco Private Limited - Inventors: Nishant Kumar Jain; Rohan Narayan Murty; George Peter Nychis; Harsh Gupta; Yoongu Kim - Web page: https://patentgrep.com/patent/US20180322004A1 ## CPC - G06F 11/3476: {Data logging (G06F11/14, G06F11/2205 take precedence)} - G06F 11/3688: {for test execution, e.g. scheduling of test suites} - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06F 11/076: {by exceeding a count or rate limit, e.g. word- or bit count limit} - G06F 2201/81: Threshold - G06F 2209/545: Gui - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract According to at least one aspect, a system is provided. The system comprises at least one hardware processor; and at least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform: monitoring execution of a first computer program configured to control a second computer program to perform a task at least in part by controlling the second computer program, through a graphical user interface (GUI) and/or an application programming interface (API) for the second computer program, to perform at least one action in furtherance of the task; detecting at least one anomaly in the execution of the first computer program; and outputting the detected at least one anomaly in the execution of the first computer program. ## Claims ### Claim 1 A system, comprising: at least one hardware processor; and at least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform: monitoring execution of a first computer program configured to control a second computer program to perform a task at least in part by controlling the second computer program, through a graphical user interface (GUI) and/or an application programming interface (API) for the second computer program, to perform at least one action in furtherance of the task; detecting at least one anomaly in the execution of the first computer program; and outputting the detected at least one anomaly in the execution of the first computer program. ### Claim 2 (depends on 1) The system of claim 1, wherein the first computer program is configured to control the second computer program through the GUI. ### Claim 3 (depends on 2) The system of claim 2, wherein controlling the second computer program through the GUI comprises emulating an action through the GUI and wherein the action is a keystroke or a mouse click. ### Claim 4 (depends on 3) The system of claim 3, wherein the at least one anomaly comprises failure of the emulated action through the GUI of the second computer program. ### Claim 5 (depends on 1) The system of claim 1, wherein the first computer program is configured to control the second computer program through the API. ### Claim 6 (depends on 1) The system of claim 1, wherein detecting the at least one anomaly in the execution of the first computer program comprises detecting a change in the second computer program being controlled by the first computer program. ### Claim 7 (depends on 6) The system of claim 6, wherein detecting the change in the second computer program comprises detecting an anomaly in a value output by the second computer program. ### Claim 8 (depends on 6) The system of claim 6, wherein detecting the change in the second computer program comprises detecting a change in at least one element in the GUI of the second computer program. ### Claim 9 (depends on 8) The system of claim 8, wherein detecting the change to the at least one element in the GUI of the second computer program comprises detecting a change in a function of the at least one element in the GUI. ### Claim 10 (depends on 1) The system of claim 1, wherein monitoring the execution of the first computer program comprises receiving program state information during execution of the first computer program indicative of a state of the first computer program and/or the second computer program being controlled by the first computer program. ### Claim 11 (depends on 10) The system of claim 10, wherein detecting the at least one anomaly in the execution of the first computer program comprises: determining, using at least one validator, whether the program state information complies with one or more rules; and identifying an anomaly in the execution of the first computer program responsive to the program state information not complying with the one or more rules. ### Claim 12 (depends on 11) The system of claim 11, wherein determining whether the program state information complies with the one or more rules comprises providing an input generated based on the program state information to a trained model to obtain an output indicative of whether the program state information complies with the one or more rules. ### Claim 13 (depends on 11) The system of claim 11, wherein receiving the program state information comprises receiving information indicative of a selector string to be employed by the first computer program in controlling an element of the GUI, the selector string comprising information that identifies the element within the GUI. ### Claim 14 (depends on 13) The system of claim 13, wherein determining whether the program state information complies with one or more rules comprises determining whether at least one feature of the selector string contains an anomaly. ### Claim 15 (depends on 14) The system of claim 14, wherein the at least one feature comprises a feature selected from the group consisting of: an X coordinate of the element within the GUI, a Y coordinate of the element within the GUI, a width of the element, a height of the element, text in the element, a name of the element, a type of the element, a permission of the element, and a position of the element relative to another element in the GUI. ### Claim 16 (depends on 1) The system of claim 1, wherein outputting the detected at least one anomaly comprises adding an anomaly event to a log file. ### Claim 17 (depends on 16) The system of claim 16, wherein the processor-executable instructions further cause the at least one hardware processor to perform: identifying a number of anomaly events in the log file; and stopping execution of the first computer program and/or the second computer program responsive to the number of anomaly events in the log file exceeding a threshold. ### Claim 18 (depends on 1) The system of claim 1, wherein outputting the detected at least one anomaly comprises sending an electronic communication comprising an indication of the detected at least one anomaly. ### Claim 19 A method, performed by a computing device, comprising: monitoring execution of a first computer program configured to control a second computer program to perform a task at least in part by controlling the second computer program, through a graphical user interface (GUI) and/or an application programming interface (API) for the second computer program, to perform at least one action in furtherance of the task; detecting at least one anomaly in the execution of the first computer program; and outputting the detected at least one anomaly in the execution of the first computer program. ### Claim 20 At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to perform: monitoring execution of a first computer program configured to control a second computer program to perform a task at least in part by controlling the second computer program, through a graphical user interface (GUI) and/or an application programming interface (API) for the second computer program, to perform at least one action in furtherance of the task; detecting at least one anomaly in the execution of the first computer program; and outputting the detected at least one anomaly in the execution of the first computer program.