Method for apparatus for efficient mining of classification models from databases

Abstract

Method and apparatus for the construction of a classification model (for example a decision tree) from data in a large database. The database is stored in the form of data records on a storage medium for access by a computer. The computer executes a software component for building a classification model from the data in the database. The classification building component has a scheduler that directs the computer to sequentially access all records in the database one or more times as required for extracting the necessary sufficient statistics to construct the classification model. The method optimizes the construction of the classifier from the database by minimizing the number of database scans and making as much use of the computer's fast main memory (RAM) as possible. The method prescribes three different modes for scanning data and includes a scheduler component that decides which of these modes to employ and on what parts of the database. During the generation of a classification model from the database, the number of active data records is reduced on subsequent scans. As the number of records that must be accessed to construct the classifier is reduced, the scheduler switches from sequential access of all records in the database to indexed access of a subset of records within the database to complete the building of the classifier. As record subset sizes decrease to the point where a subset now fits in the computer's RAM, the scheduler ultimately switches to loading the data subset and performing all subsequent processing in-memory.

Dependencies are inferred from the claim text.