# US 2019/0378034 A1 · Prediction of return path data quality for audience measurement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378034 A1
- Document ID: US20190378034A1
- Status: Published application
- Kind: A1
- Application number: 16/230,663
- Filed: 2018-12-21
- Published: 2019-12-12
- Earliest claimed: 2018-06-06
- Assignees: The Nielsen Company (US), LLC
- Applicants: The Nielsen Company (US), LLC
- Inventors: Samantha M. Mowrer; Michael Grotelueschen; Vince Tambellini; Demetrios Fassois; Jean Guerrettaz; David Kurzynski
- Web page: https://patentgrep.com/patent/US20190378034A1

## CPC

- G06N 20/00: Machine learning
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/20: Ensemble learning
- H04N 21/2408: {Monitoring of the upstream path of the transmission network, e.g. client requests (monitoring data switching networks utilization H04L43/0876; scheduling or organising the servicing of application requests H04L67/60)}
- H04N 21/251: {Learning process for intelligent management, e.g. learning user preferences for recommending movies (details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08)}
- H04N 21/44222: {Analytics of user selections, e.g. selection of programmes or purchase activity (monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33)}
- H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)}
- G06N 5/04: Inference or reasoning models
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Example methods and apparatus to predict return path data quality for audience measurement are disclosed herein. Example apparatus disclosed herein to predict return path data quality include a classification engine to compute a first data set of model features from validation tuning data reported from media metering devices and a second data set of model features from return path data reported from return path data devices. The example apparatus also include a prediction engine to train a machine learning algorithm based on the first data set, and apply the trained machine learning algorithm to the second data set to predict quality of the return path data reported from the return path data devices.

## Claims

### Claim 1

An apparatus to predict return path data quality, the apparatus comprising:
a classification engine to compute a first data set of model features from validation tuning data reported from media metering devices and a second data set of model features from return path data reported from return path data devices; and
a prediction engine to:
train a machine learning algorithm based on the first data set; and
apply the trained machine learning algorithm to the second data set to predict quality of the return path data reported from the return path data devices.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the classification engine is to filter a portion of the validation tuning data and a portion of the return path data not associated with a first viewing period.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the classification engine is to determine normalized statistics by calculating at least one of the following: 1) a difference between gap tuning minutes pre-bridging and gap tuning minutes post-bridging; 2) a percentage of conflicted tuning minutes; 3) a percentage of overloaded tuning minutes; 4) a percentage of under loaded tuning minutes; 5) a percentage of fragmented tuning minutes; 6) a percentage of unidentified tuning minutes; or 7) a percentage of mixed tuning minutes.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the classification engine is to determine a missing rate for the return path tuning data based on determining a percentage of the return path tuning data that is missing as compared to the corresponding validation tuning data.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the prediction engine is to train the machine learning algorithm based on the first data set by separating the first data set into a training data set and a holdout data set, the training data set further separated into a plurality of training data subsets and test data subsets.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the prediction engine is to train the machine learning algorithm based on the plurality of training data subsets and test subsets using cross validation to determine a first configuration of the machine learning algorithm having a higher accuracy than a second configuration of the machine learning algorithm, and
applying the holdout data set to the first configuration of the machine learning algorithm to produce metrics to analyze subsequent data sets.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein the prediction engine to apply the metrics and the first configuration of the machine learning algorithm to the second data set to determine a probability indicative of an amount of missing return path data, the amount of missing return path data applied to a missing rate threshold to determine whether the return path data is to be included in subsequent processing.

### Claim 8

A non-transitory computer readable medium comprising instructions that, when executed, cause a machine to at least:
compute a first data set of model features from validation tuning data reported from media metering devices and a second data set of model features from return path data reported from return path data devices;
train a machine learning algorithm based on the first data set; and
apply the trained machine learning algorithm to the second data set to predict quality of the return path data reported from the return path data devices.

### Claim 9 (depends on 8)

The non-transitory computer readable medium of claim 8, wherein the instructions further cause the machine to remove a portion of the validation tuning data and a portion of the return path data not associated with a first viewing period.

### Claim 10 (depends on 8)

The non-transitory computer readable medium of claim 8, wherein the instructions further cause the machine to calculate at least one of the following: 1) a difference between gap tuning minutes pre-bridging and gap tuning minutes post-bridging; 2) a percentage of conflicted tuning minutes; 3) a percentage of overloaded tuning minutes; 4) a percentage of under loaded tuning minutes; 5) a percentage of fragmented tuning minutes; 6) a percentage of unidentified tuning minutes; or 7) a percentage of mixed tuning minutes.

### Claim 11 (depends on 8)

The non-transitory computer readable medium of claim 8, wherein the instructions further cause the machine to determine a missing rate for the return path tuning data based on determining a percentage of the return path tuning data that is missing as compared to the corresponding validation tuning data.

### Claim 12 (depends on 8)

The non-transitory computer readable medium of claim 8, wherein the instructions further cause the machine to separate the first data set into a training data set and a holdout data set, the training data set further separated into a plurality of training data subsets and test data subsets.

### Claim 13 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the instructions further cause the machine to train the machine learning algorithm based on the plurality of training data subsets and test subsets using cross validation to determine a first configuration of the machine learning algorithm having a higher accuracy than a second configuration of the machine learning algorithm; and
apply the holdout data set to the first configuration of the machine learning algorithm to produce metrics to analyze subsequent data sets.

### Claim 14 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the instructions further cause the machine to apply the metrics and the first configuration of the machine learning algorithm to the second data set to determine a probability indicative of an amount of missing return path data, the amount of missing return path data applied to a missing rate threshold to determine whether the return path data is to be included in subsequent processing.

### Claim 15

A method to predict return path data quality, the method comprising:
computing, by executing an instruction with a processor, a first data set of model features from validation tuning data reported from media metering devices and a second data set of model features from return path data reported from return path data devices;
training, by executing an instruction with the processor, a machine learning algorithm based on the first data set; and
applying, by executing an instruction with the processor, the trained machine learning algorithm to the second data set to predict quality of the return path data reported from the return path data devices.

### Claim 16 (depends on 15)

The method of claim 15, further including filtering a portion of the validation tuning data and a portion of the return path data not associated with a first viewing period.

### Claim 17 (depends on 15)

The method of claim 15, further including determining a missing rate for the return path tuning data based on determining a percentage of the return path tuning data that is missing as compared to the corresponding validation tuning data.

### Claim 18 (depends on 15)

The method of claim 15, further including training the machine learning algorithm based on the first data set by separating the first data set into a training data set and a holdout data set, the training data set further separated into a plurality of training data subsets and test data subsets.

### Claim 19 (depends on 18)

The method of claim 18, further including training the machine learning algorithm based on the plurality of training data subsets and test subsets using cross validation to determine a first configuration of the machine learning algorithm having a higher accuracy than a second configuration of the machine learning algorithm; and
applying the holdout data set to the first configuration of the machine learning algorithm to produce metrics to analyze subsequent data sets.

### Claim 20 (depends on 19)

The method of claim 19, further including applying the metrics and the first configuration of the machine learning algorithm to the second data set to determine a probability indicative of an amount of missing return path data, the amount of missing return path data applied to a missing rate threshold to determine whether the return path data is to be included in subsequent processing.
