# US 2019/0317885 A1 · Machine-Assisted Quality Assurance and Software Improvement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0317885 A1
- Document ID: US20190317885A1
- Status: Published application
- Kind: A1
- Application number: 16/455,380
- Filed: 2019-06-27
- Published: 2019-10-17
- Earliest claimed: 2019-06-27
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Alexander Heinecke; Cesar Martinez-Spessot; Dario Oliver; Justin Gottschlich; Marcos Carranza; Mateo Guzman; Mats Agerstam
- Web page: https://patentgrep.com/patent/US20190317885A1

## CPC

- G06F 11/3698: Environments for analysis, debugging or testing of software
- G06F 11/302: {where the computing system component is a software system}
- G06F 11/3409: {for performance assessment}
- G06F 11/3447: {Performance evaluation by modeling}
- G06F 11/3616: {using software metrics}
- G06F 11/3668: Testing of software
- G06F 11/3676: {for coverage analysis}
- G06F 11/3684: {for test design, e.g. generating new test cases}
- G06F 21/6245: {Protecting personal data, e.g. for financial or medical purposes}
- G06F 11/3612: {by runtime analysis (performance monitoring G06F11/3466)}
- G06F 2201/865: Monitoring of software
- G06F 2221/2101: Auditing as a secondary aspect
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Apparatus, systems, methods, and articles of manufacture for automated quality assurance and software improvement are disclosed. An example apparatus includes a data processor to process data corresponding to events occurring with respect to a software application in i) a development and/or a testing environment and ii) a production environment. The example apparatus includes a model tool to: generate a first model of expected software usage based on the data corresponding to events occurring in the development and/or the testing environment; and generate a second model of actual software usage based on the data corresponding to events occurring in the production environment. The example apparatus includes a model comparator to compare the first model to the second model. The example apparatus includes a correction generator to generate an actionable recommendation to adjust the development and/or the testing environment to reduce a difference between the first model and the second model.

## Claims

### Claim 1

An apparatus comprising:
a data processor to process data corresponding to events occurring with respect to a software application in i) at least one of a development environment or a testing environment and ii) a production environment;
a model tool to:
generate a first model of expected software usage based on the data corresponding to events occurring in the at least one of the development environment or the testing environment; and
generate a second model of actual software usage based on the data corresponding to events occurring in the production environment;
a model comparator to compare the first model to the second model to identify a difference between the first model and the second model; and
a correction generator to generate an actionable recommendation to adjust the at least one of the development environment or the testing environment to reduce the difference between the first model and the second model.

### Claim 2 (depends on 1)

The apparatus of claim 1, further including a metrics aggregator to consolidate the data collected with respect to the software application in the at least one of the development environment or the testing environment, and the data collected in the production environment.

### Claim 3 (depends on 1)

The apparatus of claim 1, further including a multidimensional database to store the data.

### Claim 4 (depends on 1)

The apparatus of claim 1, further including:
a metric collector to collect the data from the at least one of the development environment or the testing environment; and
a monitoring engine to collect the data from the production environment.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the monitoring engine includes a data collector to filter the data from the production environment to protect user privacy.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the actionable recommendation includes implementing a test case to test operation of the software application.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the correction generator is to generate a graphical user interface including usage information.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the usage information includes a measure of test effectiveness between the first model and the second model.

### Claim 9

A non-transitory computer readable storage medium comprising computer readable instructions that, when executed, cause at least one processor to at least:
process data corresponding to events occurring with respect to a software application in i) at least one of a development environment or a testing environment and ii) a production environment;
generate a first model of expected software usage based on the data corresponding to events occurring in the at least one of the development environment or the testing environment;
generate a second model of actual software usage based on the data corresponding to events occurring in the production environment;
compare the first model to the second model to identify a difference between the first model and the second model; and
generate an actionable recommendation to adjust the at least one of the development environment or the testing environment to reduce the difference between the first model and the second model.

### Claim 10 (depends on 9)

The non-transitory computer readable storage medium of claim 9, wherein the instructions, when executed, cause the at least one processor to consolidate the data collected with respect to the software application from the at least one of the development environment or the testing environment, and the data collected in the production environment.

### Claim 11 (depends on 9)

The non-transitory computer readable storage medium of claim 9, wherein the instructions, when executed, cause the at least one processor to filter the data from the production environment to protect user privacy.

### Claim 12 (depends on 9)

The non-transitory computer readable storage medium of claim 9, wherein the actionable recommendation includes implementing a test case to test operation of the software application.

### Claim 13 (depends on 9)

The non-transitory computer readable storage medium of claim 9, wherein the instructions, when executed, cause the at least one processor to generate a graphical user interface including usage information.

### Claim 14 (depends on 13)

The non-transitory computer readable storage medium of claim 13, wherein the usage information includes a measure of test effectiveness between the first model and the second model.

### Claim 15

A method comprising:
processing, by executing an instruction with at least one processor, data corresponding to events occurring with respect to a software application in i) at least one of a development environment or a testing environment and ii) a production environment;
generating, by executing an instruction with the at least one processor, a first model of expected software usage based on the data corresponding to events occurring in the at least one of the development environment or the testing environment;
generating, by executing an instruction with the at least one processor, a second model of actual software usage based on the data corresponding to events occurring in the production environment;
comparing, by executing an instruction with the at least one processor, the first model to the second model to identify a difference between the first model and the second model; and
generating, by executing an instruction with the at least one processor, an actionable recommendation to adjust the at least one of the development environment or the testing environment to reduce the difference between the first model and the second model.

### Claim 16 (depends on 15)

The method of claim 15, further including consolidating the data collected with respect to the software application in the at least one of the development environment or the testing environment, and the data collected in the production environment.

### Claim 17 (depends on 15)

The method of claim 15, further including filtering the data from the production environment to protect user privacy.

### Claim 18 (depends on 15)

The method of claim 15, wherein the actionable recommendation includes implementing a test case to test operation of the software application.

### Claim 19 (depends on 15)

The method of claim 15, further including generating a graphical user interface including usage information.

### Claim 20 (depends on 19)

The method of claim 19, wherein the usage information includes a measure of test effectiveness between the first model and the second model.

### Claim 21

An apparatus comprising:
memory including machine reachable instructions; and
at least one processor to execute the instructions to:
process data corresponding to events occurring with respect to a software application in i) at least one of a development environment or a testing environment and ii) a production environment;
generate a first model of expected software usage based on the data corresponding to events occurring in the at least one of the development environment or the testing environment;
generate a second model of actual software usage based on the data corresponding to events occurring in the production environment;
compare the first model to the second model to identify a difference between the first model and the second model; and
generate an actionable recommendation to adjust the at least one of the development environment or the testing environment to reduce the difference between the first model and the second model.

### Claim 22 (depends on 21)

The apparatus of claim 21, wherein the instructions, when executed, cause the at least one processor to consolidate the data collected with respect to the software application in the at least one of the development environment or the testing environment, and the data collected in the production environment.

### Claim 23 (depends on 21)

The apparatus of claim 21, wherein the instructions, when executed, cause the at least one processor to filter the data from the production environment to protect user privacy.

### Claim 24 (depends on 21)

The apparatus of claim 21, wherein the actionable recommendation includes implementing a test case to test operation of the software application.

### Claim 25 (depends on 21)

The apparatus of claim 21, wherein the instructions, when executed, cause the at least one processor to generate a graphical user interface including usage information.

### Claim at position 26

-27. (canceled)
