# US 2026/0086918 A1 · Execution-time telemetry for teams group copilot evaluation framework > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2026/0086918 A1 - Document ID: US20260086918A1 - Status: Published application - Kind: A1 - Application number: 18/893,500 - Filed: 2024-09-23 - Published: 2026-03-26 - Earliest claimed: 2024-09-23 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Madhu SUDAN; Siunie Aquawati SUTJAHJO; Pritesh Rajesh KANANI - Web page: https://patentgrep.com/patent/US20260086918A1 ## CPC - G06F 11/3476: {Data logging (G06F11/14, G06F11/2205 take precedence)} - G06F 11/3612: {by runtime analysis (performance monitoring G06F11/3466)} - G06F 11/302: {where the computing system component is a software system} - G06F 11/3072: {where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting} - G06F 11/323: {Visualisation of programs or trace data} - G06F 2201/835: Timestamp - G06F 2201/86: Event-based monitoring - G06F 2201/865: Monitoring of software - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The execution-time reporting of telemetry of execution of a software program includes identifying a software program and accessing an event stream with multiple event entries associated with the software's components. An object model is generated that comprises a hierarchy of objects and properties associated with the events by creating objects for the software's components, including identifiers, properties, and timestamps. When an AI entity is instantiated for use during the software's execution, an AI object is created with an identifier and properties. The AI object is dynamically modified to reflect events associated with the AI entity during operation of the software program. The telemetry associated with the AI entity is reported according to subscriptions for the telemetry sessions. ## Claims ### Claim 1 A computing system, comprising: a processor; a computer-readable storage medium in communication with the processor, the computer-readable storage medium having computer-executable instructions stored thereupon which are executable by the processor for causing the processor to implement a method that includes the computing system: accessing an event stream, the event stream containing a plurality of event entries associated with components of a software program during execution of the software program; generating an object model comprising a hierarchy of objects and object properties associated with the plurality of events by at least: generating objects for the components of the software program, the objects including object identifiers, object properties and timestamps associated with events corresponding to the components during execution of the software program; creating an AI object for the object model, the AI object including an AI object identifier and a property set of one or more properties of an artificial intelligent (AI) entity instantiated for utilization during execution of the software program; identifying a new event from the event stream corresponding to the utilization of the AI entity during the execution of the software program; and dynamically modifying the AI object in response to detecting the new event. ### Claim 2 (depends on 1) The computing system of claim 1, wherein the new event identifies a task performed by the AI entity and wherein the AI object is dynamically modified to reflect the task performed by the AI entity ### Claim 3 (depends on 1) The computing system of claim 1, wherein the new event identifies a feature enabled or disabled for the AI entity and wherein the AI object is dynamically modified to reflect the feature that is enabled or disabled for the AI entity. ### Claim 4 (depends on 1) The computing system of claim 1, wherein the new event identifies a change in a status of the AI entity and wherein the AI object is dynamically modified to reflect the change in the status of the AI entity. ### Claim 5 (depends on 4) The computing system of claim 4, wherein the new event identifies the AI entity as a participant in a session of the software program being executed and wherein the AI object is dynamically modified to reflect the status of the AI entity as a participant. ### Claim 6 (depends on 1) The computing system of claim 1, wherein the method further comprises: dynamically modifying the AI object in response to a subscription request for telemetry corresponding to the new event. ### Claim 7 (depends on 1) The computing system of claim 1, wherein the method further comprises: updating a timeline data structure with event data from the identified events to correlate the events from the event stream that are received out of an actual sequential order of operation due to a latency in transmission of some of the event data to reflect a modified and actual sequential ordering that matches the actual order of operation of the identified events. ### Claim 8 (depends on 1) The computing system of claim 1, wherein the method further comprises: accessing a subscription to telemetry scope of execution of the software program; identifying telemetry from a portion of the AI object of the object model that is applicable to the subscribed telemetry scope; and in response to determining that the telemetry is to be reported in accordance with the subscription, causing the determined telemetry to be reported in accordance with the subscription. ### Claim 9 (depends on 8) The computing system of claim 8, the subscription to telemetry scope being associated with a skill associated with the AI entity. ### Claim 10 (depends on 9) The computing system of claim 9, the subscription to telemetry scope further being associated with a video channel of the video conferencing application. ### Claim 11 (depends on 9) The computing system of claim 9, the subscription to telemetry scope further being associated with an audio channel of the video conferencing application. ### Claim 12 (depends on 9) The computing system of claim 9, the subscription to telemetry scope further being associated with a data channel of the video conferencing application. ### Claim 13 (depends on 1) The computing system of claim 1, wherein the method further comprises: creating a different AI object for a different AI entity invoked during execution of the software program, the different AI object having a different AI object identifier and a different property set of one or properties of the different AI entity that are distinguished from the set of one or more properties of the AI entity; and modifying the object model to include the different AI object. ### Claim 14 (depends on 1) The computing system of claim 1, wherein the software program comprises a video conferencing application and wherein execution of the video conferencing application includes instantiating a video conferencing session with at least one human participant and wherein the AI entity comprises another participant in the video conferencing session. ### Claim 15 (depends on 1) The computing system of claim 1, wherein the event stream comprises a log file of log entries associated with operations of the software program during execution of the software program. ### Claim 16 A computer-implemented method for facilitating the reporting of telemetry for AI entities utilized by software programs, the method including: accessing an event stream, the event stream containing a plurality of event entries associated with components of a software program during execution of the software program; generating an object model comprising a hierarchy of objects and object properties associated with the plurality of events by at least: generating objects for the components of the software program, the objects including object identifiers, object properties and timestamps associated with events corresponding to the components during execution of the software program; identifying a new event from the event stream corresponding to the utilization of the AI entity during the execution of the software program; and dynamically modifying the AI object model in response to detecting the new event and by at least creating an AI object for the object model, the AI object including an AI object identifier and a property set of one or more properties of AI entity being utilized during execution of the software program. ### Claim 17 (depends on 16) The method of claim 16, wherein the new event identifies a task performed by the AI entity and wherein the AI object is dynamically modified to reflect the task performed by the AI entity. ### Claim 18 (depends on 16) The method of claim 16, wherein the method further comprises: updating a timeline data structure with event data from the identified events to correlate the events from the event stream that are received out of an actual sequential ordering of event occurrence due to a latency in transmission of some of the event data to reflect a modified and actual sequential ordering that matches the actual sequential ordering of event occurrence. ### Claim 19 (depends on 16) The method of claim 16, wherein the method further comprises: accessing a subscription to telemetry scope of execution of the software program; identifying telemetry from a portion of the AI object of the object model that is applicable to the subscribed telemetry scope; and in response to determining that the telemetry is to be reported in accordance with the subscription, causing the determined telemetry to be reported in accordance with the subscription. ### Claim 20 (depends on 1) The method of claim 1, wherein the software program comprises a video conferencing application and wherein execution of the video conferencing application includes instantiating a video conferencing session with at least one human participant and wherein the AI entity comprises another participant in the video conferencing session.