# US 9,003,436 B2 · Apparatus and methods for data collection, analysis and validation including error correction in a content delivery network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,003,436 B2 - Document ID: US9003436B2 - Status: Granted - Kind: B2 - Application number: 13/936,055 - Filed: 2013-07-05 - Published: 2013-11-07 - Granted: 2015-04-07 - Earliest claimed: 2010-07-01 - Assignees: Time Warner Cable Enterprises LLC - Applicants: Time Warner Cable Enterprises LLC - Inventors: Justin Tidwell; Eduardo Samame; Craig Engel - Web page: https://patentgrep.com/patent/US9003436B2 ## CPC - H04N 21/2408: {Monitoring of the upstream path of the transmission network, e.g. client requests (monitoring data switching networks utilization H04L43/0876; scheduling or organising the servicing of application requests H04L67/60)} - G06F 11/008: {Reliability or availability analysis} - H04H 60/64: for providing detail information - H04N 21/44204: {Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched (monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31)} - H04N 21/44224: {Monitoring of user activity on external systems, e.g. Internet browsing} - H04N 21/458: Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules {; time-related management operations (arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10)} - H04N 21/4667: {Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections} - H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)} - H04H 60/33: Arrangements for monitoring the users' behaviour or opinions - H04H 60/46: for recognising users' preferences - H04N 21/4524: {involving the geographical location of the client (retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00)} - H04N 21/4532: {involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04H: BROADCAST COMMUNICATION (multiplex communication H04J; pictorial communication aspects of broadcast systems H04N) ## Abstract Methods and apparatus for collection, validation, analysis, and automated error correction of data regarding user interaction with content. In one embodiment, statistical methods are used to arrive at expected values for the collected data. The data is compared to the expected value and must meet one or more acceptance criteria (e.g., be within a prescribed range) to be considered valid. The prescribed range is determined by the network operator, or a computer program adapted to generate this value. The invention enables a network operator to assess a large volume of data without requiring significant amounts of manual monitoring and/or error correction. The ability to collect, validate and analyze data across multiple platforms is also provided. Still further, an automated system capable of learning evaluation and error correction patterns is disclosed. ## Claims ### Claim 1 A method of providing data collection in a network, said method comprising: collecting a plurality of data relating to interaction of a plurality of users with content or with an apparatus controlling said content; deriving an expected value for said plurality of data; based at least in part on said expected value, configuring a range of acceptable values for said plurality of data; comparing said plurality of data to said range of acceptable values; and transmitting said plurality of data to at least one entity for analysis of said plurality of data when said plurality of data is determined to be within said range. ### Claim 2 (depends on 1) The method of claim 1, wherein said act of deriving an expected value for said plurality of data comprises deriving a value for an average number of individual instances of said plurality of data expected to be collected over a given period of time. ### Claim 3 (depends on 1) The method of claim 1, wherein said act of deriving an expected value for said plurality of data comprises deriving a value for an average number of user devices within a given geographic area from which said plurality of data is expected to be collected. ### Claim 4 (depends on 1) The method of claim 1, wherein said act of deriving an expected value for said plurality of data comprises deriving a value for an average number of user devices expected to be tuned to each channel over a given period of time. ### Claim 5 (depends on 1) The method of claim 1, wherein said act of deriving an expected value for said plurality of data comprises deriving a value for an average number of user devices expected to be tuned to each network over a given period of time. ### Claim 6 (depends on 1) The method of claim 1, wherein said act of deriving an expected value for said plurality of data comprises deriving a value for an average number of programs expected to be viewed, requested, provided, and/or made available on a given channel. ### Claim 7 (depends on 1) The method of claim 1, wherein said network comprises a content delivery network, said apparatus controlling said content comprises a content server, said data comprises a total number of tuning events received at said content server, and said act of deriving an expected value for said plurality of data comprises deriving a value for an average number of tuning events expected to be received at said content server. ### Claim 8 In a content delivery network comprising a plurality of users receiving content at a respective plurality of user devices via one or more platforms, a method of collecting data comprising: collecting data relating to an interaction of said plurality of users with said content received at said respective plurality of user devices via said one or more platforms; comparing values of individual ones of a plurality of data fields in said data to one or more acceptance criteria for each of said individual ones of said plurality of data fields; and transmitting said data to at least one entity for analyzing said data when said one or more acceptance criteria are met. ### Claim 9 (depends on 8) The method of claim 8, further comprising deriving said one or more acceptance criteria. ### Claim 10 (depends on 9) The method of claim 9, wherein said act of deriving comprises generating a value for an average number of advertisements delivered to a portion of said plurality of users within a given advertisement zone via said one or more platforms. ### Claim 11 (depends on 9) The method of claim 9, wherein said act of deriving comprises generating a value for an average number of advertisements received at said plurality of user devices via said one or more platforms. ### Claim 12 (depends on 9) The method of claim 9, wherein said act of deriving comprises generating a value for an average number of each type of a plurality of tune-out events occurring at individual ones of said plurality of user devices. ### Claim 13 (depends on 9) The method of claim 9, wherein said act of deriving comprises generating a value for an average number of each type of a plurality of channel changing events occurring at individual ones of said plurality of user devices. ### Claim 14 (depends on 9) The method of claim 9, wherein said act of deriving comprises generating a value for an average number of each of: tune-in, tune-out, and tune-away events occurring at individual ones of said plurality of user devices. ### Claim 15 Non-transitory computer readable apparatus comprising a medium configured to store a computer program thereon, said computer program comprising a plurality of instructions which are configured to, when executed: collect a plurality of data records, individual ones of said plurality of said data records each comprising a plurality of data fields having values contained therein, said values each configured to be validated; compare said values of said plurality of data fields for each of said collected plurality of data to one or more accepted values specific to individual ones of said plurality of data fields; transmit said plurality of data to at least one entity for analysis when said values of each of said plurality of data fields are within a predetermined range of said one or more accepted values therefor; and implement a substantially automated error correction process of individual ones of said data fields which are determined to have values that are not within said predetermined range of said one or more accepted values therefor. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein at least one of said values of said plurality of data fields comprises a value indicative of an amount of data provided to a particular client device over a given time window. ### Claim 17 (depends on 15) The apparatus of claim 15, wherein said predetermined range of said one or more accepted values comprises a range of content types of which said values are expected to be. ### Claim 18 (depends on 15) The apparatus of claim 15, wherein at least one of said values of said plurality of data fields comprises a value indicative of a number of records expected to be received at a particular client device over a given time window. ### Claim 19 (depends on 15) The apparatus of claim 15, wherein at least one of said values of said plurality of data fields comprises a value indicative of a number of records which contain a null value in a required field and said predetermined range of said one or more accepted values therefor is zero. ### Claim 20 (depends on 15) The apparatus of claim 15, wherein at least one of said values of said plurality of data fields comprises a value indicative of a number of records which contain a value that is not present in any other of said plurality of collected data records. ### Claim 21 Apparatus configured to provide data collection in a network, said apparatus comprising: a data processing apparatus; and computerized logic in data communication with said data processing apparatus, said logic configured to: collect a plurality of data relating to interaction of a plurality of users with content or with an apparatus configured to control said content; derive an expected value for said plurality of data; based at least in part on said expected value, configure a range of acceptable values for said plurality of data; compare said plurality of data to said range of acceptable values; and based at least on said comparison: transmit said plurality of data to at least one entity for analysis of said plurality of data when said plurality of data is determined to be within said range; or when said plurality of data is determined not to be within said range: generate at least one error message; implement an error correction process to correct said plurality of data; and transmit corrected data to said at least one entity for analysis. ### Claim 22 (depends on 21) The apparatus of claim 21, wherein said error correction process comprises a test thereof, when said error correction process fails said test, said computerized logic being further configured to: accept manual correction to said plurality of data; and update an error correction profile to improve said error correction process based on said manual correction. ### Claim 23 (depends on 22) The apparatus of claim 22, wherein said range of acceptable values is based at least in part on said updated error correction profile. ### Claim 24 (depends on 21) The apparatus of claim 21, wherein said range of acceptable values is based at least in part on a regression analysis. ### Claim 25 (depends on 24) The apparatus of claim 24, wherein a regression period of said regression analysis is set based on a statistical significance of said regression analysis.