# US 2008/0313617 A1 · Analyzing software users with instrumentation data and user group modeling and analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0313617 A1
- Document ID: US20080313617A1
- Status: Published application
- Kind: A1
- Application number: 11/818,610
- Filed: 2007-06-15
- Published: 2008-12-18
- Earliest claimed: 2007-06-15
- Assignees: Microsoft Corporation
- Inventors: Wenli Zhu; Yantao Li; Qiang Wang; Haidong Zhang; Bing Sun; Shuguang Ye; Min Wang; Jian Wang
- Web page: https://patentgrep.com/patent/US20080313617A1

## CPC

- G06F 11/321: {Display for diagnostics, e.g. diagnostic result display, self-test user interface}
- G06F 11/328: {Computer systems status display (G06F11/327 takes precedence)}
- G06F 11/3414: {Workload generation, e.g. scripts, playback}
- G06F 11/3466: {Performance evaluation by tracing or monitoring}
- G06F 11/3604: Analysis of software for verifying properties of programs (testing of software G06F11/3668)
- G06F 2201/81: Threshold
- G06F 2201/865: Monitoring of software
- G06F 2201/88: Monitoring involving counting
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Described is a technology by which software instrumentation data collected from user program sessions are analyzed, including by determining program usage metrics and/or command usage metrics. Information representative of the program usage metrics and/or the command usage metrics is output, such as in the form of a report. The software instrumentation data may be further analyzed, such as to determine at least one usage trend over time, and to determine user groups. For example, a usage subset of sessions that meet specified session usage criteria based on a set of session data may be located, along with a subset of users based on users whose sessions meet specified user criteria. The usage and user subsets may be combined via Boolean logic to produce a result set.

## Claims

### Claim 1

In a computing environment, a method comprising:
analyzing information corresponding to software instrumentation data collected from user sessions corresponding to one or more programs, including determining program usage metrics or determining command usage metrics, or determining program usage metrics and command usage metrics; and
outputting information representative of the program usage metrics, the command usage metrics, or both the program usage metrics and the command usage metrics.

### Claim 2 (depends on 1)

The method of claim 1 further comprising, collecting the instrumentation data during actual user sessions.

### Claim 3 (depends on 1)

The method of claim 1 wherein determining the program usage metrics comprises determining for a set of users at least one of: session count information based on a number of application sessions, session frequency information based on a time measurement between sessions, running time information based on session time, session length information based on session time and session count, or depth of usage information based on a percentage of commands used, or any combination of session count information, session frequency information, running time information, session length information or depth of usage information.

### Claim 4 (depends on 1)

The method of claim 1 wherein determining the command usage metrics comprises determining for a set of users and a selected command at least one of: user count information based on a number of users of the set who use the selected command, percentage of users information corresponding to a percentage of users of the set who use the selected command, session count information based on a number of sessions in which the selected command occurred, percentage of session information corresponding to a percentage of application sessions in which the selected command was used, click count information corresponding to a number of clicks corresponding to the selected command, percentage of click count information corresponding to a percentage of program clicks corresponding to the selected command, click count per user information based on click count and user count of the selected command, or click count per session information corresponding to a click count per session, or any combination of user count information, percentage of users information, session count information, percentage of session information, click count information, percentage of click count information, click count per user information, or click count per session information.

### Claim 5 (depends on 1)

The method of claim 1 further comprising, analyzing the information corresponding to the software instrumentation data to determine at least one usage trend over time.

### Claim 6 (depends on 5)

The method of claim 5 wherein analyzing the information corresponding to the software instrumentation data to determine at least one usage trend over time comprises determining at least one of: user count information corresponding to a number of users that used a program during a reporting interval, session count information corresponding to a number of sessions of a program during a reporting interval, session count per user information based on a number of sessions and a total number of users that used a program during a reporting interval, running time per user information corresponding to a session length of users and a number of users that used a program during a reporting interval, cumulative session count per user information corresponding to a total number of sessions and a number of users from a start time of a period to analyze to an end of each reporting interval, or cumulative running time per user information corresponding to a session length of users that used a program and a number of users from a start time of a period to analyze to an end of each reporting interval, or any combination of user count information, session count information, session count per user information, running time per user information, cumulative session count per user information, or cumulative running time per user information.

### Claim 7 (depends on 1)

The method of claim 1 further comprising, analyzing the information corresponding to the software instrumentation data to determine at least one type of user.

### Claim 8 (depends on 7)

The method of claim 7 wherein analyzing the information corresponding to the software instrumentation data to determine at least one type of user comprises categorizing users by depth of usage, or categorizing users by types of activities in which they engage, or categorizing users by depth of usage and categorizing users by the types of activities in which they engage.

### Claim 9 (depends on 1)

The method of claim 1 further comprising, analyzing the information corresponding to the software instrumentation data to determine at least potential outlier corresponding to command usage that appears different from command usage of other users.

### Claim 10 (depends on 1)

The method of claim 1 wherein analyzing the information corresponding to the software instrumentation data to determine at least potential outlier comprises computing an entropy value corresponding to an occurrence distribution of a command, computing an average occurrence per session of the command usage by a potential outlier, and using the entropy value or the average occurrence per session, or both, as outlier criterion or criteria.

### Claim 11 (depends on 1)

The method of claim 1 further comprising, locating a subset of the user sessions based on at least one session criterion, wherein each session criteria comprises a dimension and a value for that dimension, and each dimension comprises a variable recorded in a session, a feature, or a set of one or more variables computed from a plurality of variables recorded in a session, and wherein analyzing the information corresponding to the software instrumentation data comprises analyzing the subset.

### Claim 12 (depends on 11)

The method of claim 11 wherein locating the subset comprises mathematically combining each of at least two session criteria.

### Claim 13 (depends on 1)

The method of claim 1 further comprising, providing a mechanism for modeling a user group, including providing an interface for receiving one or more measures that specify one or more conditions that any user needs to meet in order to belong to the user group, and locating the users from the information corresponding to the software instrumentation data based on the one or more measures.

### Claim 14 (depends on 13)

The method of claim 13 wherein locating the users comprises determining at least one of: session length information corresponding to a session length of the sessions of a user, crash information corresponding to a number of sessions of a user that crashed, depth of usage information corresponding to which commands were used by a user, enrollment length information corresponding to a time between an earliest session and a latest session of a user, failure information corresponding to a number of sessions that failed of a user, mean time to crash information corresponding to session length and sessions of a user that crashed, mean time to failure information corresponding to session length and number of sessions of a user that failed, session count information corresponding to a total number of sessions of a user, session frequency information corresponding to time between consecutive sessions of a user, or total running time information corresponding to total session time of sessions of a user, or any combination of session length information, crash information, depth of usage information, enrollment length information, failure information, mean time to crash information, mean time to failure information, session count information, session frequency information, or total running time information.

### Claim 15

A computer-readable medium having computer executable instructions, which when executed perform steps comprising, locating a subset of sessions that meet specified session criteria based on a set of session data, locating a subset of users based on users whose sessions meet specified user criteria based on the set of session data, and combining the subset of sessions with the subset of users via Boolean logic to produce a result set.

### Claim 16 (depends on 15)

The computer-readable medium of claim 15 wherein the session criteria comprise at least one session criterion and another session criterion combined with each other via Boolean logic, and wherein the user criteria comprise at least one user criterion and another user criterion combined with each other via Boolean logic.

### Claim 17

In a computing environment, a system comprising:
an analyzer that processes information corresponding to software instrumentation data recorded from user software program usage sessions to produce a first subset comprising software usage data;
a group modeling mechanism that processes the information corresponding to the software instrumentation data a second subset comprising user data; and
means for combining the first subset with the second subset to provide an output that corresponds to a selected group of users and their software program usage.

### Claim 18 (depends on 17)

The system of claim 17 further comprising a user interface for facilitating selection of one or more criteria by which the first subset and second subset are located from the information.

### Claim 19 (depends on 17)

The system of claim 17 further comprising means for recording the software instrumentation data.

### Claim 20 (depends on 17)

The system of claim 17 wherein the analyzer comprises means for performing at least one of an application usage analysis, a command usage analysis, or a trend analysis, or any combination of an application usage analysis, a command usage analysis, or a trend analysis.
