# US 2008/0313507 A1 · Software reliability analysis using alerts, asserts and user interface controls

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0313507 A1
- Document ID: US20080313507A1
- Status: Published application
- Kind: A1
- Application number: 11/818,612
- Filed: 2007-06-15
- Published: 2008-12-18
- Earliest claimed: 2007-06-15
- Assignees: Microsoft Corporation
- Inventors: Adnan Azfar Mahmud; Yingnong Dang; Haidong Zhang; Qiang Wang; Wenli Zhu; Yantao Li; Jian Wang
- Web page: https://patentgrep.com/patent/US20080313507A1

## CPC

- G06F 11/3616: {using software metrics}
- G06F 11/008: {Reliability or availability analysis}
- G06F 11/366: {using diagnostics (G06F11/0703 takes precedence)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Described is a technology by which software instrumentation data collected during software program usage sessions is analyzed to identify potential problems with software program usage, such as based on frequency of problem occurrence during the usage sessions. Reliability metrics may be calculated from the information. Failure data additionally collected during the usage sessions may be accessed to derive details that correspond to the potential problems. In one example, the information may be analyzed to determine which alerts and/or asserts occurred most often, and/or to determine a relationship between user interface control operations (e.g., clicks and usage of commands) and alerts or asserts.

## Claims

### Claim 1

In a computing environment, a method comprising:
analyzing information corresponding to a set of software instrumentation data collected during software program usage sessions to identify potential problems with at least one software program corresponding to the software instrumentation data, in which the potential problems are identified by frequency of occurrence during the usage sessions; and
accessing, based upon the potential problems, failure data additionally collected during the usage sessions, to derive details from the failure data that correspond to the potential problems.

### Claim 2 (depends on 1)

The method of claim 1 wherein analyzing the information includes, filtering the software instrumentation data to locate a subset of session data corresponding to only program sessions that terminated abnormally.

### Claim 3 (depends on 2)

The method of claim 2 wherein analyzing the information further comprises processing the subset to determine which alerts or asserts, or both alerts and asserts, occurred most often within that subset of sessions.

### Claim 4 (depends on 2)

The method of claim 2 wherein analyzing the information further comprises calculating reliability metrics from the information.

### Claim 5 (depends on 4)

The method of claim 4 wherein calculating the reliability metrics comprises computing at least one of: crash information corresponding to a number of sessions of a user that crashed, failure information corresponding to a number of sessions that failed of a user, mean time to crash information corresponding to session length and sessions of a user that crashed, mean time to failure information corresponding to session length and number of sessions of a user that failed, mean commands to crash information, or mean commands to failure information, or any combination of crash information, failure information, mean time to crash information, mean commands to crash information, or mean commands to failure information.

### Claim 6 (depends on 5)

The method of claim 5 further comprising, outputting a distribution mapping for at least one calculated reliability metric.

### Claim 7 (depends on 1)

The method of claim 1 further comprising, determining the frequency of consecutive failures for a software product.

### Claim 8 (depends on 1)

The method of claim 1 further comprising, determining which computer programs including at least one application of a selected software product were running concurrently when a problem occurred.

### Claim 9 (depends on 1)

The method of claim 1 wherein analyzing the information comprises determining a relationship between user interface control operations and alerts or asserts.

### Claim 10 (depends on 1)

The method of claim 1 wherein analyzing the information comprises performing an analysis on a selected group of users, or a selected program version, or a combination of a selected group of users and a selected program version.

### Claim 11

A computer-readable medium having computer executable instructions, which when executed perform steps comprising, locating a set of users having problems with a software product, including determining from recorded session data one or more reliability metrics comprising at least one of: crash information corresponding to a number of sessions of a user that crashed, failure information corresponding to a number of sessions that failed of a user, mean time to crash information corresponding to session length and sessions of a user that crashed, mean time to failure information corresponding to session length and number of sessions of a user that failed, mean commands to crash information, or mean commands to failure information, or any combination of crash information, failure information, mean time to crash information, mean commands to crash information, or mean commands to failure information, and for that set of users, determining whether each user belongs to a class of users for which failure data was recorded, and if so, accessing the failure data to derive details from the failure data that correspond to the recorded session data for that user.

### Claim 12 (depends on 11)

The computer-readable medium of claim 11 having further computer-executable instructions comprising, outputting information corresponding to a subset of users based on their problems with the software product.

### Claim 13 (depends on 11)

The computer-readable medium of claim 11 having further computer-executable instructions comprising, filtering the recorded session data to locate the set of users based on filtering criterion or criteria corresponding to at least one parameter in the recorded session data.

### Claim 14 (depends on 11)

The computer-readable medium of claim 11 having further computer-executable instructions comprising, outputting a distribution mapping for each determined reliability metric.

### Claim 15 (depends on 11)

The computer-readable medium of claim 11 having further computer-executable instructions comprising, determining the frequency of consecutive failures for a software product.

### Claim 16 (depends on 11)

The computer-readable medium of claim 11 having further computer-executable instructions comprising, determining which computer programs including at least one application of the software product were running concurrently when a problem occurred.

### Claim 17

In a computing environment, a system comprising:
an analyzer that processes information corresponding to software instrumentation data recorded from user software program usage sessions to determine which alerts or asserts, or both alerts and asserts, occurred during sessions;
a locator component that accesses failure data for at least one alert or assert determined by the analyzer; and
means for outputting results corresponding to alert or assert, or alert and assert occurrences, and any located failure data for each alert or assert.

### Claim 18 (depends on 17)

The system of claim 17 wherein the analyzer processes the information to determine a relationship between user interface operations and alerts.

### Claim 19 (depends on 17)

The system of claim 17 wherein the analyzer processes the information to determine a relationship between user interface operations and asserts.

### Claim 20 (depends on 17)

The system of claim 17 further comprising means for filtering the information based on one or more session criteria or user criteria, or both session criteria and user criteria, to determine which alerts or asserts, or both alerts and asserts occurred during only those sessions that meet the criteria.
