Optimized logging of data elements to a data storage device

In a computer system, a method for logging a stream of data produced by a data source to a hardware device by a data logger operating in a separate thread than the data source, the method comprising the steps of:

• retrieving from a set of empty buffers a first buffer pointing to a first block of memory;
• filling the first block of memory with data records from the stream of data;
• placing the first buffer in a queue of full buffers;
• retrieving from the set of empty buffers a second buffer pointing to a second block of memory;
• filling the second block of memory with data records from the stream of data;
• placing the second buffer in the queue of full buffers;
• retrieving the first and second buffers from the queue of full buffers;
• packaging the contents of the first and second buffers into a larger data block;
• passing the larger data block to a controller for logging the data stored in the first and second blocks of memory on the hardware device;
• and
• logging the data stored in the first and second blocks of memory on the hardware device.

In a computer system, a method for logging data produced by a data source to a hard disk drive by a data logger operating in a separate thread than the data source, the method comprising the steps of:

• retrieving from a set of empty buffers, by the data source, a first buffer pointing to a first page in memory;
• storing in the first page in memory, by the data source, data records to be logged to the hard disk drive;
• placing the first buffer in a queue of full buffers;
• retrieving from the set of empty buffers, by the data source, a second buffer pointing to a second page in memory;
• storing in the second page in memory, by the data source, data records to be logged to the hardware device;
• placing the second buffer in the queue of full buffers;
• retrieving the first and second buffers from the queue of full buffers;
• packaging the addresses contained in the first and second buffers in a parameter array;
• passing the parameter array to a hard drive controller;
• logging on the hard disk drive the data stored in the memory pages corresponding to the addresses contained in the parameter array;
• and
• returning the first and second buffers to the set of empty buffers.