Relational database system and method for query processing using early aggregation

A relational database system comprising:

• a non-volatile memory;
• a volatile memory for temporarily storing a set of data records, the volatile memory having an amount of space available for query processing;
• a query processor coupled to the non-volatile and volatile memories to process a query of the data records according to at least one query parameter, the query processor being configured to partition the data records into multiple buckets for query processing using the available space in the volatile memory;
• and
• the query processor being configured to aggregate data records having like query parameters and to occasionally select ones of the data records for writing to the non-volatile memory to free up part of the available space to receive additional data records, the query processor selecting the data records for writing to the non-volatile memory from any one of the buckets.

A relational database system comprising:

• a volatile memory for temporarily storing a set of data records for query processing;
• a query processor coupled to the volatile memory to process a query of the data records, the query processor being configured to partition the data records into multiple partitions using a hash partitioning table, the hash partitioning table having entries that reference the partitions;
• and
• the query processor being configured to create a second hash table separate from the partitioning table that has entries that reference individual data records within the partitions, the separate hash table being used to lookup matching data records for aggregation.

A relational database system comprising:

• a non-volatile memory;
• a volatile memory for temporarily storing a set of data records, the volatile memory having an amount of available space segmented into multiple memory pages;
• a query processor coupled to the non-volatile and volatile memories to process a series of data records, the query processor being configured to partition the data records into multiple partitions and store the data records in one or more memory pages associated with the partitions;
• as a new data record is processed into a particular partition, the query processor being configured to alternately aggregate the new data record with a like data record already existing in the particular partition or store the data record separately within the particular partition;
• in an event that a memory page of the particular partition becomes full, the query processor assigns an additional memory page to the particular partition;
• and
• in an event that no free memory pages are left to assign, the query processor selects a memory page from any one of the partitions, writes the data records on the selected memory page to the non-volatile memory to empty the memory page, and moves the emptied memory page to the particular partition.

A relational database computer program embodied on a computer-readable medium comprising:

• partitioning code to partition data records into multiple partitions;
• aggregation code to aggregate within respective partitions like data records;
• and
• victim selection code to select at least one data record to write to non-volatile memory to free space in the partitions for future data records, the victim selection code selecting the data record according to how likely that data record will aggregate, if left in its respective partition, with the future data records.

A method for processing a query, comprising the following steps:

• partitioning data records into multiple partitions;
• aggregating within respective partitions like data records;
• and
• selecting at least one data record to write to non-volatile memory to free space in the partitions for future data records according to how likely that data record will aggregate, if left in its respective partition, with the future data records.

In a relational database system having multiple memory pages for holding data records that are separated into predefined partitions, a method for processing a query of the data records, comprising the following steps:

• partitioning a new data record into one of the partitions;
• if a like data record already exists in the partition, aggregating the new data record with the like data record and otherwise storing the data record separately within the partition;
• in an event that a memory page of the partition is full and cannot accept the new data entry, adding a next memory page to the partition;
• and
• in an event that no free memory pages are left to add, selecting a memory page from any one of the partitions according to a selection criteria that favors output of full memory pages over partially filled memory pages and that favors memory pages with low absorption rates and writing the data records on the selected memory page to non-volatile memory to empty the memory page.