US 8,135,667 B2 · System, method, and computer-readable medium that facilitate in-database analytics with supervised data discretization

Abstract

A system, method, and computer-readable medium that facilitate in-database supervised discretisation mechanisms which improve data classification are provided. The disclosed mechanisms provide an efficient, automatic, and repeatable way to perform data discretisation without human intervention. Efficient processing of large and complex unknown data is provided that advantageously does not require the data being analyzed to be processed outside the database. The disclosed mechanisms may use an External Stored Procedure to avoid multiple joins of large tables and minimize the number of full table scans and, consequently, provide better performance than contemporary mechanisms. The disclosed system produces intermediate results in tables which may be conveyed to a visualization subsystem thereby providing users a better understanding of the data distribution in each category. Further, the disclosed system and method introduce a novel similarity-based solution to merge intervals when chi-square testing is not reliable and thereby improves the quality of the interval merge process.

Claims20

Dependent claims

A method of performing database analytics that facilitates data discretization, comprising:

receiving, by a processing module, a query including a selection applied on a first attribute of a base table and a second attribute of the base table, wherein the first attribute comprises numeric values and the second attribute comprises a plurality of discrete values;
generating, by the processing module, a contingency table that includes a respective row having an attribute for each numeric value of the first attribute of the base table and a plurality of corresponding attributes for each respective discrete value of the second attribute of the base table, wherein each of the plurality of corresponding attributes store a count of a number of occurrences of one of the discrete values of the second attribute that are identified in a common row of the base table with the numeric value of the first attribute of the base table;
identifying adjacent rows of the contingency table suitable to be merged based on an evaluation of similarities between the plurality of corresponding attributes of the contingency table;
and
assigning a discretized value to the numeric value of the contingency table.

2 more elements

A computer-readable medium having computer-executable instructions for execution by a processing system, the computer-executable instructions for performing database analytics that facilitate data discretization, the computer-executable instructions, when executed, cause the processing system to:

receive, by a processing module, a query including a selection applied on a first attribute of a base table and a second attribute of the base table, wherein the first attribute comprises numeric values and the second attribute comprises a plurality of discrete values;
generate, by the processing module, a contingency table that includes a respective row having an attribute for each numeric value of the first attribute of the base table and a plurality of corresponding attributes for each respective discrete value of the second attribute of the base table, wherein each of the plurality of corresponding attributes store a count of a number of occurrences of one of the discrete values of the second attribute that are identified in a common row of the base table with the numeric value of the first attribute of the base table;
identify, by the processing module, adjacent rows of the contingency table suitable to be merged based on an evaluation of similarities between the plurality of corresponding attributes of the contingency table;
and
assign, by the processing module, a discretized value to the numeric value of the contingency table.

2 more elements

A computer system having a database management system configured to perform database analytics that facilitates data discretization, comprising:

at least one storage medium on which the database management system and a base table is stored;
and
at least one processing module that receives a query including a selection applied on a first attribute of a base table and a second attribute of the base table, wherein the first attribute comprises numeric values and the second attribute comprises a plurality of discrete values, generates a contingency table that includes a respective row having an attribute for each numeric value of the first attribute of the base table and a plurality of corresponding attributes for each respective discrete value of the second attribute of the base table, wherein each of the plurality of corresponding attributes store a count of a number of occurrences of one of the discrete values of the second attribute that are identified in a common row of the base table with the numeric value of the first attribute of the base table, identifies adjacent rows of the contingency table suitable to be merged based on an evaluation of similarities between the plurality of corresponding attributes of the contingency table, and assigns a discretized value to the numeric value of the contingency table.

Dependencies are inferred from the claim text.

System, method, and computer-readable medium that facilitate in-database analytics with supervised data discretization

Life of this patent

System, method, and computer-readable medium that facilitate in-database analytics with supervised data discretization

Life of this patent

Abstract

Claims20