Apparatus and method for read optimized bulk data storage

A method, comprising:

• appending new rows to an append only table, wherein appending new rows comprises:
• concurrently writing a first and a second set of records for data storage in the append only table, wherein concurrently writing is performed by independently operating writers, the writers including a first data writer and a second data writer, and wherein concurrently writing comprises:
• assigning a first memory region of the append only table to the first data writer;
• assigning a second memory region of the append only table to the second data writer;
• and
• writing the first set of records into the first memory region by the first data writer while writing the second set of records into the second memory region by the second data writer;
• maintaining data change information that is applicable to the append only table;
• receiving a read request to read data from the append only table;
• determining that the first memory region is visible to the read request;
• and
• producing read data in response to the read request, including:
• reading one or more tuples from the first memory region;
• reading the data change information;
• and
• updating or eliminating one or more of the tuples as specified by the data change information.

A computer readable storage medium, comprising executable instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:

• appending new rows to an append only table, wherein appending new rows comprises:
• concurrently writing a first and a second set of records for data storage in the append only table, wherein concurrently writing is performed by independently operating writers, the writers including a first data writer and a second data writer, and wherein concurrently writing comprises:
• assigning a first memory region of the append only table to the first data writer;
• assigning a second memory region of the append only table to the second data writer;
• and
• writing the first set of records into the first memory region by the first data writer while writing the second set of records into the second memory region by the second data writer;
• maintaining data change information that is applicable to the append only table;
• receiving a read request to read data from the append only table;
• determining that the first memory region is visible to the read request;
• and
• producing read data in response to the read request, including:
• reading one or more tuples from the first memory region;
• reading the data change information;
• and
• updating or eliminating one or more of the tuples as specified by the data change information.

A system, comprising one or more computing devices for operating a data warehouse, the one or more computing devices comprising an append only data store, a read only visibility manager, and a merge module;

• wherein the append only data store appends new rows to an append only table, wherein appending new rows comprises:
• concurrently writing a first and a second set of records for data storage in the append only table, wherein concurrently writing is performed by independently operating writers, the writers including a first data writer and a second data writer, and wherein concurrently writing comprises:
• assigning a first memory region of the append only table to the first data writer;
• assigning a second memory region of the append only table to the second data writer;
• and
• writing the first set of records into the first memory region by the first data writer while writing the second set of records into the second memory region by the second data writer;
• wherein the read only visibility manager is configured to control access to the append only data store, wherein controlling access comprises:
• maintaining data change information that is applicable to the append only table;
• and
• determining that a particular memory region is visible to a read request to read data from the append only table;
• and
• wherein the merge module is configured to produce read data in response to the read request, wherein producing read data comprises:
• reading one or more tuples from the particular memory region;
• reading the data change information;
• and
• updating or eliminating one or more of the tuples as specified by the data change information.