Data sampling method that maintains accuracy for data analysis
AssigneeInternational Business Machines Corporation
InventorsShuho Kidokoro, Yojiro Adachi, Tomoyuki Kawamukoh, Tomokazu Ohtsu
Life of this patent
Estimating the term…
Abstract
According to an aspect, a computer-implemented method includes collecting statistics of an original database and collecting statistics of a sampling database that includes a subset of the original database. The statistics of the original database are periodically updated. The statistics of the original database and the statistics of the sampling database are periodically compared to determine whether the sampling database is within a predetermined threshold of the original database. In response to determining that the sampling database is not within the predetermined threshold of the original database, an update to the sampling database is initiated.
Claims17
A computer-implemented method comprising:
- collecting statistics of an original database;
- collecting statistics of a sampling database that comprises a subset of the original database;
- periodically updating the statistics of the original database in response to an update to the original database and based at least in part on a log of the original database;
- periodically comparing the statistics of the original database and the statistics of the sampling database to determine whether the sampling database is within a predetermined threshold of the original database;
- and
- in response to determining that the sampling database is not within the predetermined threshold of the original database, initiating an update to the sampling database and causing an analytics service to access the sampling database instead of the original database to perform data analysis, thereby reducing processing overhead and network traffic.
A system comprising:
- a memory having computer readable instructions;
- and
- one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:
- collecting statistics of an original database;
- collecting statistics of a sampling database that comprises a subset of the original database;
- periodically updating the statistics of the original database in response to an update to the original database and based at least in part on a log of the original database;
- periodically comparing the statistics of the original database and the statistics of the sampling database to determine whether the sampling database is within a predetermined threshold of the original database;
- and
- in response to determining that the sampling database is not within the predetermined threshold of the original database, initiating an update to the sampling database and causing an analytics service to access the sampling database instead of the original database to perform data analysis, thereby reducing processing overhead and network traffic.
A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:
- collecting statistics of an original database;
- collecting statistics of a sampling database that comprises a subset of the original database;
- periodically updating the statistics of the original database in response to an update to the original database and based at least in part on a log of the original database;
- periodically comparing the statistics of the original database and the statistics of the sampling database to determine whether the sampling database is within a predetermined threshold of the original database;
- and
- in response to determining that the sampling database is not within the predetermined threshold of the original database, initiating an update to the sampling database and causing an analytics service to access the sampling database instead of the original database to perform data analysis, thereby reducing processing overhead and network traffic.
Dependencies are inferred from the claim text.