Method and system for collecting and managing tv viewership data

A distributed computer system, comprising:

• multiple computers that are connected to a computer network, each computer having one or more processors and memory storing one or more programs and multiple event records, each event record including a predefined time interval, wherein the multiple computers are configured to perform the following operations when executing the one or more programs:
• receiving an event log file that includes a plurality of data source IDs, each data source ID having an associated set of event log entries and each event log entry including a time interval;
• dynamically selecting one or more computers by applying the plurality of data source IDs to a predefined sharding function;
• at each of the selected computers:
• identifying at least one of the plurality of data source IDs that matches at least one of the event records stored on the computer;
• for each identified data source ID and each of the set of event log entries associated with the identified data source ID:
• determining a location in the memory of the computer in accordance with the event log entry's time interval and the time intervals of the event records stored on the memory of the computer;
• generating a new event record for the event log entry, wherein the new event record includes the time interval associated with the event log entry;
• storing the new event record at the determined location within the memory of the computer;
• and
• in accordance with a predefined schedule, replicating the compressed event records from a respective one of the selected computers to one or more other computers of the distributed system such that there are at least two replicas of any event record on at least two computers of the distributed computer system.

A non-transitory computer readable storage medium storing one or more programs configured for execution by a distributed computer system that includes multiple computers, each computer having one or more processors and memory storing multiple event records, each event record including a predefined time interval, the one or more programs comprising instructions for:

• receiving an event log file that includes a plurality of data source IDs, each data source ID having an associated set of event log entries and each event log entry including a time interval;
• dynamically selecting one or more computers by applying the plurality of data source IDs to a predefined sharding function;
• at each of the selected computers:
• identifying at least one of the plurality of data source IDs that matches at least one of the event records stored on the computer;
• for each identified data source ID and each of the set of event log entries associated with the identified data source ID:
• determining a location in the memory of the computer in accordance with the event log entry's time interval and the time intervals of the event records stored on the memory of the computer;
• generating a new event record for the event log entry, wherein the new event record includes the time interval associated with the event log entry;
• storing the new event record at the determined location within the memory of the computer;
• and
• in accordance with a predefined schedule, replicating the compressed event records from a respective one of the selected computers to one or more other computers of the distributed system such that there are at least two replicas of any event record on at least two computers of the distributed computer system.