Method, apparatus, and computer program product for locating data in large datasets
AssigneeInternational Business Machines Corporation
InventorMasakazu Shirota
Life of this patent
Estimating the term…
Abstract
To analyze a data set having a one-to-many relation, the number of simultaneous occurrences of data in which two data elements are coexistent is obtained for all combinations of two data elements. A dependence ratio of one data element upon the other data element is calculated from the numbers of simultaneous occurrences. The data elements are grouped based upon the numbers of occurrences of individual data elements and the dependence ratios compared with the predetermined thresholds. Based on the number of occurrences of individual data elements and the dependence ratios, subordinate relations of data elements within the same group are specified and displayed to a user in the form of a tree or balloon figure.
Claims14
1, A data analysis system for analyzing data sets, comprising:
- a database for storing data;
- analysis processing means for analyzing a relation among data elements based on a number of occurrences, in stored data, of a first data element, and a number of simultaneous occurrences, in the stored data, of the first data element and a second data element;
- and
- output means for outputting results provided by the analysis processing means.
2, The data analysis system according to claim 1, wherein the analysis processing means groups data elements based on the number of simultaneous occurrences.
3, The data analysis system according to claim 2, wherein the analysis processing means groups the first data element and the second data element into the same group, when a ratio of the number of simultaneous occurrences to the number of occurrences of the first data element is greater than a predetermined value.
4, The data analysis system according to claim 1, wherein the output means represents dependencies among the data elements as a tree figure.
5, Apparatus for data analysis, comprising:
- analysis processing means for specifying pairs of keywords based on frequencies that keywords occur in a set of keywords of data stored in a database, and grouping the set of keywords into groups of keywords based on the specified pairs of keywords;
- and
- output means for outputting results provided by the analysis processing means.
6, The apparatus for data analysis according to claim 5, wherein the analysis processing means calculates a number of data samples that include both keywords of the pair of keywords, and groups both keywords into the same group if the calculated number of data samples that include both keywords meets a predetermined threshold.
7, The data analysis apparatus according to claim 6, wherein the analysis processing means defines the group to which one of the two keywords belongs, based on the group to which the other of the two keywords belongs, when the predetermined threshold is not met.
8, The data analysis apparatus according to claim 5, wherein the output means displays results provided by the analysis processing means in the form of a figure.
9, A display terminal comprising:
- an interface for requesting analysis of data;
- accepting means for accepting analysis results giving a relation based on a number of occurrences of two data elements of a plurality of data elements;
- and
- output means for displaying the relation as a figure, based on the analysis results.
10, The display terminal according to claim 9, wherein the output means displays subordinate relations of the data as a tree figure.
11, The display terminal according to claim 9, wherein the output means represents each data element in the form of a figure having a size based on the number of occurrences of the data element, and wherein the distance between two figures is based on the relation of two represented data elements.
12, A data analysis method, comprising the steps of:
- calculating dependence ratios of a plurality of data elements to be analyzed;
- grouping the data elements according to the dependence ratios;
- and
- outputting the grouped data elements.
13, The data analysis method according to claim 12, wherein the step of grouping includes the step of grouping a first data element and a second data element in the same group, if a ratio of the number of simultaneous occurrences of the first data element and the second data element to the number of occurrences of the first data element is greater than a predetermined value.
14, The data analysis method according to claim 13, wherein the step of grouping includes the step of specifying a subordinate relation between the first data element and the second data element, based on the ratio of the number of simultaneous occurrences to the number of occurrences of the first data element.