Methods for alignment of plan to execution using multimodal cues for script-based performance monitoring

One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

• receiving a performance plan, wherein the performance plan includes one or more documents associated with a scripted performance;
• identifying, via a first machine learning model and based on the one or more documents, one or more segments included in the scripted performance;
• predicting, via a second machine learning model, a current segment of the one or more segments based at least on one or more observation signals associated with the execution of the scripted performance;
• and
• initiating an execution of one or more automated actions based on the predicted current segment.

A system comprising:

• one or more memories storing instructions;
• and
• one or more processors for executing the instructions to:
• receive a performance plan, wherein the performance plan includes one or more documents associated with a scripted performance;
• identify, via a first machine learning model and based on the one or more documents, one or more segments included in the scripted performance;
• predict, via a second machine learning model, a current segment of the one or more segments based at least on one or more observation signals associated with the execution of the scripted performance;
• and
• initiate an execution of one or more automated actions based on the predicted current segment.