Execution-time telemetry for teams group copilot evaluation framework

A computing system, comprising:

• a processor;
• a computer-readable storage medium in communication with the processor, the computer-readable storage medium having computer-executable instructions stored thereupon which are executable by the processor for causing the processor to implement a method that includes the computing system:
• accessing an event stream, the event stream containing a plurality of event entries associated with components of a software program during execution of the software program;
• generating an object model comprising a hierarchy of objects and object properties associated with the plurality of events by at least:
• generating objects for the components of the software program, the objects including object identifiers, object properties and timestamps associated with events corresponding to the components during execution of the software program;
• creating an AI object for the object model, the AI object including an AI object identifier and a property set of one or more properties of an artificial intelligent (AI) entity instantiated for utilization during execution of the software program;
• identifying a new event from the event stream corresponding to the utilization of the AI entity during the execution of the software program;
• and
• dynamically modifying the AI object in response to detecting the new event.