US 6,651,074 B1 · Method and apparatus for storage and retrieval of very large databases using a direct pipe

Abstract

A method and apparatus for directly connecting very large data streams from an archive command into a backup data system using an “intelligent process.” An output stream-is piped into an intelligent pipe-reading process and distributed over a set of temporary data stores built from raw storage resources. A pipe interface process supervises backup of each filled data store, while the remaining output stream continues to be piped into another available data store. The backup system completes archiving of the datastream, keeping a catalog of the datastream storage locations. To retrieve the data, the intelligent process is run in reverse as a pipe-writing process, requesting data from the backup system. Retrieved data traverses the data stores from the backup system and are pumped into the pipe-writing process for delivery to the pipe output file identified by the retrieve or import command.

Claims17

Dependent claims

A method of directly connecting an output of a computer process to a storage device comprising the steps of:

creating a FIFO pipe fed by said output;
emptying said FIFO pipe into an intelligent process;
and
storing data from said intelligent process into said storage device,
wherein said intelligent process includes at least one FIFO data store and each of said at least one FIFO data stores conveys said data from said FIFO Dip to said storage device:
wherein said storing data includes,
reading data from said FIFO pipe,
selecting one of said at least one FIFO data stores to identify a selected data store,
marking said selected data store as busy, and
writing said data read from said FIFO pipe into said selected data store, wherein
if said selected data store is filled, and there is additional data in said FIFO pipe, then said storing includes,
selecting a next one of said at least one FIFO data stores,
writing said additional data in said FIFO pipe into said selected next one of said at least one FIFO data stores, and
writing said data from said selected data store into said storage device, and wherein
if said selected data store is filled, and there is no additional data in said FIFO pipe, then said storing includes
waiting until said FIFO pipe is closed;
and
writing said data from said selected data store into said storage device.

15 more elements

A method of directly reading data from a storage device into a computer process comprising the steps of:

activating a FIFO pipe having an input from an intelligent process and an output to a destination file,
wherein said intelligent process includes at least one FIFO data store and each of said at least one data stores conveys data from said storage device to said FIFO pipe;
reading said data into said intelligent process from said storage device;
selecting one of said at least one FIFO data stores to identify a selected data store,
marking said selected data store as busy, and
writing said data read from said storage device into said selected data store, wherein
if said selected data store is filled, and there is additional data to be read from said storage device, then said storing includes,
selecting a next one of said at least one FIFO data stores,
writing said additional data from said storage device into said selected next one of said at least one FIFO data stores, and
writing said data from said selected data store into said FIFO pipe, and wherein
if said selected data store is filled, and there is no additional data to be read from said storage device, then said storing includes
waiting until said data is read from said storage device, and writing said data from said selected data store into said FIFO pipe.

9 more elements

Dependencies are inferred from the claim text.

Method and apparatus for storage and retrieval of very large databases using a direct pipe

Life of this patent

Method and apparatus for storage and retrieval of very large databases using a direct pipe

Life of this patent

Abstract

Claims17