# US 2017/0286462 A1 · Data stream quality management for analytic environments

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286462 A1
- Document ID: US20170286462A1
- Status: Published application
- Kind: A1
- Application number: 15/624,295
- Filed: 2017-06-15
- Published: 2017-10-05
- Earliest claimed: 2012-05-04
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Randy George; Robert J. McKeown
- Web page: https://patentgrep.com/patent/US20170286462A1

## CPC

- G06F 16/22: Indexing; Data structures therefor; Storage structures
- G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
- G06F 16/2365: {Ensuring data consistency and integrity}
- G06F 16/24568: {Data stream processing; Continuous queries}
- G06F 40/169: Annotation, e.g. comment data or footnotes
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Data stream quality management for analytic environments includes deploying, into a runtime environment upstream from an analytic module, an ingress quality specification (IQS) module. The IQS module receives a data stream and analyzes a subset of data of the data stream to determine if the subset of data meets a quality expectation of the analytic module. The subset of data is annotated to indicate a quality status based on whether the subset of data meets the quality expectation of the analytic module. The data stream is output with the annotated subset of data to the analytic module, and the analytic module analyzes the data stream to assess an operating characteristic of an upstream device.

## Claims

### Claim 1

A method, comprising:
deploying, into a runtime environment upstream from an analytic module, an ingress quality specification (IQS) module;
receiving, by the IQS module, a data stream;
analyzing, by the IQS module, a subset of data of the data stream to determine if the subset of data meets a quality expectation of the analytic module;
annotating the subset of data to indicate a quality status based on whether the subset of data meets the quality expectation of the analytic module; and
outputting the data stream with the annotated subset of data to the analytic module, the analytic module analyzing the data stream to assess an operating characteristic of an upstream device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising receiving a selection of the IQS module from a plurality of IQS modules associated with the analytic module.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
selecting, by the IQS module, a portion of the data stream as the subset of data; and
applying, by the IQS module, a predicate to the subset of data.

### Claim 4 (depends on 3)

The method of claim 3, further comprising flagging data of the subset passing the predicate.

### Claim 5 (depends on 4)

The method of claim 4, further comprising flagging data of the subset failing the predicate.

### Claim 6 (depends on 1)

The method of claim 1, further comprising determining, by the IQS module, whether the selected subset of data of the data stream includes a minimum quantity of data samples based on the quality expectation of the analytic module.

### Claim 7 (depends on 1)

The method of claim 1, wherein annotating comprises annotating the subset of data via an additional tuple field of the data.

### Claim 8

A method, comprising:
deploying, into a runtime environment upstream from an analytic module, an ingress quality specification (IQS) module;
receiving, by the IQS module, a data stream;
selecting, by the IQS module, a subset of data of the data stream;
determining if the subset of data meets a quality expectation of the analytic module by analyzing whether the subset of data includes a minimum quantity of data samples based on the quality expectation of the analytic module;
responsive to determining that the subset of data meets the quality expectation, annotating the subset of data to indicate a quality status; and
outputting the data stream with the annotated subset of data to the analytic module.

### Claim 9 (depends on 8)

The method of claim 8, wherein annotating comprises flagging data of the subset meeting the quality expectation.

### Claim 10 (depends on 9)

The method of claim 9, wherein annotating comprises flagging data of the subset failing to meet the quality expectation.

### Claim 11 (depends on 8)

The method of claim 8, further comprising receiving, via an interface, a selection of the IQS modules to deploy with the analytic module.

### Claim 12 (depends on 8)

The method of claim 8, further comprising:
independently deploying the IQS module relative to the analytic module; and
linking the IQS module to the analytic module.

### Claim 13 (depends on 9)

The method of claim 9, wherein the data stream comprises a first data stream received from a first source and second data stream received from a second source.

### Claim 14

A method, comprising:
receiving, by a first ingress quality specification (IQS) module deployed in a runtime environment, a data stream;
applying, by the first IQS module, a first predicate to a subset of data of the data stream to determine if the subset of data meets a first quality expectation of a downstream analytic module;
making a first annotation in the subset of data to indicate a quality status based on a result of applying the first predicate;
outputting by the first IQS module the data stream to a second IQS module;
applying, by the second IQS module, a second predicate to the subset of data to determine if the subset of data meets a second quality expectation of the analytic module;
making a second annotation in the subset of data to indicate a quality status based on a result of applying the second predicate;
receiving, by the analytic module, the data stream from the second IQS module and analyzing the subset of data to assess a performance level of a monitored object, the analytic module discerning between the first annotation and the second annotation to determine whether to include the subset of data in the assessment of the monitored object.

### Claim 15 (depends on 14)

The method of claim 14, wherein making the first and second annotation comprises flagging data of the subset meeting the respective first and second predicates.

### Claim 16 (depends on 15)

The method of claim 15, wherein making the first and second annotation further comprises flagging data of the subset failing the respective first and second predicates.

### Claim 17 (depends on 14)

The method of claim 14, further comprising omitting, by the analytic module, the subset of data from an analysis of the data stream in response to the subset failing the first or second predicate.

### Claim 18 (depends on 14)

The method of claim 14, wherein making the first and second annotation comprises annotating the subset of data via an additional tuple field of the data.

### Claim 19 (depends on 14)

The method of claim 14, further comprising receiving, via an interface, a selection of the first and second IQS modules to deploy with the analytic module.

### Claim 20 (depends on 14)

The method of claim 14, further comprising:
independently deploying the first and second IQS modules relative to the analytic module; and
linking the first and second IQS modules to the analytic module.
