# US 2021/0279099 A1 · Program flow monitoring and control of an event-triggered system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279099 A1 - Document ID: US20210279099A1 - Status: Published application - Kind: A1 - Application number: 16/810,557 - Filed: 2020-03-05 - Published: 2021-09-09 - Earliest claimed: 2020-03-05 - Assignees: NVIDIA Corporation - Applicants: NVIDIA Corporation - Inventors: Peter Alexander Boonstoppel; Michael Cox; Daniel Perrin - Web page: https://patentgrep.com/patent/US20210279099A1 ## CPC - G06F 9/4887: {involving deadlines, e.g. rate based, periodic} - G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)} - G06F 9/542: {Event management; Broadcasting; Multicasting; Notifications} - G06F 11/3055: {Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available (error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction of the data by redundancy in operations G06F11/14; error detection or correction by redundancy in hardware G06F11/16)} - G06F 11/3072: {where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting} - G06F 11/3495: {for systems} - G06F 2201/86: Event-based monitoring - G06F 2201/865: Monitoring of software - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract In various examples, a system is provided for monitoring and controlling program flow in an event-triggered system. A program (e.g., application, algorithm, routine, etc.) may be organized into operational units (e.g., nodes executed by one or more processors), each of which tasked with executing one or more respective events (e.g., tasks) within the larger program. At least some of the events of the larger program may be successively executed in a flow, one after another, using triggers sent directly from one node to the next. In addition, the system of the present disclosure may include a manager that may exchange communications with the nodes to monitor or assess a status of the system (e.g., determine when a node has completed an event) or to control or trigger a node to initiate an event. ## Claims ### Claim 1 A system comprising: a computing device including one or more processing devices and one or more memory devices communicatively coupled to the one or more processing devices storing programmed instructions thereon, which when executed by the processor causes the instantiation of: a task loader to manage an ordered representation of a program including a plurality of nodes corresponding to execution of the program, each node of the plurality of nodes corresponding to at least one task within the program; a first subset of the plurality of nodes to execute corresponding tasks according to the ordered representation by transmitting node-originating triggers between nodes of the first subset of the plurality of nodes, at least one node of the first subset of the plurality of nodes starting execution of a corresponding task responsive to receiving a node-originating trigger from a preceding node; a monitoring manager to receive event reports based on output from the plurality of nodes, each event report indicating at least one of a start of a task corresponding to a respective node or an end of the task corresponding to the respective node; and a control manager to transmit manager-originating triggers to at least a second subset of the plurality of nodes based at least in part on the event reports, at least one node of the second subset of the plurality of nodes starting execution of a corresponding task responsive to receiving a manager-originating trigger from the control manager. ### Claim 2 (depends on 1) The system of claim 1, wherein at least one node of the plurality of nodes starts execution of a corresponding task responsive to receiving both a respective manager-originating trigger from the control manager and a respective node-originating trigger from a preceding node. ### Claim 3 (depends on 1) The system of claim 1, wherein at least a first task corresponding to a first node is executed using a first processing engine and a second task corresponding to a second node is executed using a second processing engine heterogeneous from the first processing engine, and further wherein the control manager sends a first manager-originating trigger to the first node and a second manager-originating trigger to the second node based on a determined priority of the first task with respect to the second task. ### Claim 4 (depends on 1) The system of claim 1, wherein at least a first task corresponding to a first node and a second task corresponding to a second node are executed using one or more homologous processing engines, and further wherein the control manager sends a first manager-originating trigger to the first node and a second manager-originating trigger to the second node based on a determined priority of the first task with respect to the second task. ### Claim 5 (depends on 1) The system of claim 1, wherein at least a first manager-originating trigger is generated for a time-based condition. ### Claim 6 (depends on 1) The system of claim 1, wherein at least a first manager-originating trigger is generated for an event-based condition. ### Claim 7 (depends on 1) The system of claim 1, wherein the monitoring manager further determines, based on one or more conditions, when the manager-originating triggers are to be sent to the second subset of the plurality of nodes, and the monitoring manager provides indications of the manager-originating triggers to the control manager. ### Claim 8 (depends on 1) The system of claim 1, wherein the monitoring manager compares received event reports to a schedule representing one or more time conditions by which one or more events of the plurality of events are to be completed, and wherein the monitoring manager reports a failed condition when at least one of the one or more time conditions are not satisfied. ### Claim 9 A method comprising: generating a representation of a program, the representation including a plurality of nodes in an order, each node of the plurality of nodes associated with a task within the program; transmitting a node-originating trigger between a first node of the plurality of nodes and a second node of the plurality of nodes subsequent the first node in the order, the node-originating trigger indicating to the second node that an event-based condition has been met, the event-based condition including completion of a respective task of the first node; and based at least in part on the respective task having been completed, transmitting an event report to a manager monitoring execution of the plurality of nodes within the program. ### Claim 10 (depends on 9) The method of claim 9, further comprising determining, by the manager, that a time-based condition has been met and, responsive to the determining, transmitting a manager-originating trigger to a third node of the plurality of nodes. ### Claim 11 (depends on 9) The method of claim 9, further comprising determining, by the manager, that an event-based condition has been met based on receipt of another event report from a third node of the plurality of nodes and, responsive to the determining, transmitting a manager-originating trigger to a fourth node of the plurality of nodes. ### Claim 12 (depends on 9) The method of claim 9, further comprising determining, by the manager, that an event-based condition has been met based on receipt of a signal from a source external to the plurality of nodes and, responsive to the determining, transmitting a manager-originating trigger to a third node of the plurality of nodes. ### Claim 13 (depends on 9) The method of claim 9, wherein the program includes event-triggered scheduling comprising one or more nodes of the plurality of nodes sending node-originating triggers to successively positioned nodes, and wherein the method further comprises the manager determining whether events executed by the one or more nodes satisfy time-based conditions. ### Claim 14 (depends on 9) The method of claim 9, wherein the program includes event-triggered scheduling comprising a first set of one or more nodes of the plurality of nodes receiving node-originating triggers from other of the plurality of nodes, and further wherein the method further comprises the manager transmitting manager-originating triggers to a second set of one or more nodes of the plurality of nodes. ### Claim 15 (depends on 9) The method of claim 9, wherein the plurality of nodes includes a first set of nodes for execution using a first hardware engine and a second set of nodes for execution using a second hardware engine, the second hardware engine being homologous to the first hardware engine, and further wherein the manager monitors and controls the first hardware engine and the second hardware engine by receiving event reports from, and sending manager-originating triggers to, one or more nodes in both the first set of nodes and the second set of nodes. ### Claim 16 (depends on 9) The method of claim 9, wherein the plurality of nodes includes a first set of nodes for execution using a first hardware engine and a second set of nodes for execution using a second hardware engine, which is heterogeneous to the first hardware engine, and further wherein the manager monitors and controls the first hardware engine and the second hardware engine by receiving event reports from, and sending manager-originating triggers to, nodes in both the first set of nodes and the second set of nodes. ### Claim 17 A method comprising: receiving sensor data generated by one or more sensors of a vehicle; applying the sensor data to a program configured to generate an output using the sensor data; executing the program to generate the output, at least in part, by: generating an ordered representation of the program across a plurality of nodes, each associated with a respective task within the program; executing a first task associated with a first node of the plurality of nodes; responsive to the executing the first task, transmitting an event report to a manager monitoring the execution of the program and a node-originating trigger to a second node of the plurality of nodes subsequent the first node according to the ordered representation; based at least in part on a condition being met, transmitting, by the manager, a manager-originating trigger to the second node; and based at least in part on receiving both the node-originating trigger and the manager-originating trigger, executing a second task associated with the second node; and performing one or more operations by the vehicle based at least in part on the output of the program. ### Claim 18 (depends on 17) The method of claim 17, further comprising, responsive to the executing the second task, transmitting another event report to the manager. ### Claim 19 (depends on 17) The method of claim 17, wherein the condition is one of an event-based condition or a time-based condition, and wherein the executing of the second task is dependent on both the node-originating trigger and the manager-originating trigger. ### Claim 20 (depends on 17) The method of claim 17, wherein the program is further configured to generate the output using two or more different processing engines, the two or more different processing engines including two or more of a graphics processing unit (GPU), a central processing unit (CPU), a digital signal processors (DSP), a system on chip (SoC), an image signal processor (ISP), or a 2D transform.