Veto-based model for measuring product health

A system for determining the performance of a cloud-based software product over time, comprising:

• processing circuitry including at least one processor;
• and
• a memory device having instructions stored therein, wherein the instructions, which when executed by the processing circuitry, configure the at least one processor to:
• collect telemetry data representing whether different features of online sessions of the software product are operating properly, where the telemetry data represents shared performance metrics of participant experiences with the software product across different participants and components participating in an online session;
• correlate the collected telemetry data with session identifiers identifying the online session from which the telemetry data was collected;
• process the telemetry data for the online session to establish a unit of failure when the telemetry data indicates that the online session operated outside of predefined performance metrics wherein processing the telemetry data comprises applying vetoes to a candidate list, wherein collecting telemetry data comprises creating at least one shared performance metric that begins with the candidate list of online sessions of the software product potentially with performance outside of the predefined performance metrics, wherein the candidate list of online sessions contains a session identifier, metadata about the online sessions, and why each online session was chosen as part of a candidate list of online sessions;
• and
• determine the performance of the software product as a function of the unit of failure over time.

A non-transitory machine-readable medium having instructions stored thereon that when executed by one or more processors cause the one or more processors to implement a method of determining the performance of a cloud-based software product over time, the method including:

• collecting telemetry data representing whether different features of online sessions of the software product are operating properly, where the telemetry data represents shared performance metrics of participant experiences with the software product across different participants and components participating in an online session;
• correlating the collected telemetry data with session identifiers identifying the online session from which the telemetry data was collected;
• processing the telemetry data for the online session to establish a unit of failure when the telemetry data indicates that the online session operated outside of predefined performance metrics wherein processing the telemetry data comprises applying vetoes to a candidate list, wherein collecting telemetry data comprises creating at least one shared performance metric that begins with the candidate list of online sessions of the software product potentially with performance outside of the predefined performance metrics, wherein the candidate list of online sessions contains a session identifier, metadata about the online sessions, and why each online session was chosen as part of a candidate list of online sessions;
• and
• determining the performance of the software product as a function of the unit of failure over time.