System and method for authoring human-involved context-aware applications

A method for authoring an automation in which an operation is performed responsive to human actions in an environment, the method comprising:

• synchronously recording, with at least one sensor, (i) human motions of a user in an environment, the recorded human motions including a plurality of recorded positions of the user in the environment over a period of time, and (ii) values for a context attribute over the period of time;
• displaying, on a display, a graphical user interface including, a graphical representation of the recorded human motions including a graphical representation of the user, the graphical representation of the user representing a position of the user at a particular time in the recorded human motions;
• selecting, with a processor, based on user inputs received from the user, a value for the context attribute;
• moving, with the processor, in response to selecting the value for the context attribute, the graphical representation of the user to represent a position of the user at a time in the recorded human motions at which the context attribute had the selected value;
• defining, with a processor, a first event by identifying, based on user inputs received from the user, (i) a first segment of the recorded human motions that corresponds to the first event and (ii) a first value for the context attribute that corresponds to the first event;
• and
• defining, with the processor, based on user inputs received from the user, a first operation to be performed in response to the first event occurring.

A method for authoring an automation in which an operation is performed responsive to human actions in an environment, the method comprising:

• synchronously recording, with at least one sensor, (i) human motions of a user in an environment, the recorded human motions including a plurality of recorded poses of the user in the environment over a period of time, and (ii) values for a context attribute over the period of time;
• defining, with a processor, a first event by identifying, based on user inputs received from the user, (i) a first segment of the recorded human motions that corresponds to the first event and (ii) a first value for the context attribute that corresponds to the first event;
• identifying, with the processor, a second segment of the recorded human motions that has a less than threshold difference compared to the first segment;
• labeling, with the processor, in response to user inputs received from the user, the second segment with a label indicating that second segment corresponds to the first event;
• and
• defining, with the processor, based on user inputs received from the user, a first operation to be performed in response to the first event occurring.