US 5,551,020 A · System for the compacting and logical linking of data blocks in files to optimize available physical storage

Abstract

The present invention is a method of processing and writing data to a database wherein the method comprises four broad steps: 1) manipulating data files into a more compact and efficient bit-encoded form and preparing the files to receive additional data to link the files; 2) linking the files with pointers to form an overall data structure; 3) determining the potential physical memory address for the files' data by optimizing available memory space for a given memory media wherein the files are partitioned into blocks of data which are sufficiently inclusive to permit retrieval of all required data with a single memory media read, yet small enough to allow all of the physical memory space to be fully utilized; and 4) generating reference tables to be interspersed with the data blocks wherein the reference tables track the physical location of related data, obviating the need for additional disk reads. The invention further comprises a system wherein said method is performed on a database contained on a first computer system so that the database can be written to a memory media of fixed size which can be used with a second computer system, such that memory access speed is enhanced, the density of information stored on the memory media is increased, yet the computational power needed to retrieve such information is reduced.

Claims20

Dependent claims

A method for compressing data files for storage on a memory media comprising the steps of:

a. processing the data within each file, wherein the step of processing comprises
(1) ordering the data;
(2) encoding the data;
(3) packing the encoded data to form data strings;
b. linking the files to form an overall data structure wherein the relative relationship between files is determined;
c. partitioning the data strings within each file to form data blocks and determine the physical location of said blocks on the memory media so that the available memory of the memory media is optimized, while the relative relationship between files is maintained;
d. generating reference data within each data block to relate the physical location of said blocks on the memory media;
and
e. writing the data blocks to the memory media.

6 more elements

A method for storing data, said data being organized into a first sequence of records on a first database and stored in a first binary form, wherein said data is stored in a second sequence on a second database such that the memory space required to store the second sequence of records is arranged in a manner which increases the speed at which the data can be retrieved, said method comprising the following steps:

a. ordering the data of each record of said first sequence of records into a predetermined sequence to form data segments wherein the data segments are arranged based on common characteristics of the data segments;
b. repeating the process of step (a) within each data segment such that each data segment is characterized by sub-segments arranged based on common characteristics of the data strings which comprise each segment.
c. encoding each data segment into a second binary form which requires less memory to store said data on said second database than was required by the data in said first binary form wherein the second binary form is such that each character is represented by a binary number smaller than would be required to represent the character in another binary form;
d. inserting a delineator at the beginning of each segment and sub-segment, wherein the value of the delineator allows the delineator to also functions as a pointer or contains information about the subsequent data segment to which the delineator has been appended;
e. compressing the encoded segments of each record into a continuous data string wherein the delineators separate individual data segments within each continuous data string;
f. appending an index table at the beginning of each continuous data string;
g. removing equivalent characters from each data segment within a data string and recording the equivalent characters in the index table;
h. recording in the index table the length of a data segment in a data string when the data string is comprised of equivalent length data segments;
i. linking the data strings by generating and appending additional data to each data string which relates the individual data strings;
j. appending additional empty data places to each string for receiving location dependent data;
k. partitioning the data strings into blocks of data based on the physical characteristics of said second database such that said second database is optimally filled wherein the physical characteristics of the database are utilized to determine the break points within data strings and the location of each block in said second database such that blocks which have related data are physically located adjacent one another in said second database;
l. generating location dependent data to fill the empty data places of each data string wherein said location dependent data is generated based on the physical location on said second database to which each block is assigned;
and
m. writing the blocks of data to their predetermined locations on said second database.

11 more elements

A system for compressing and retrieving data comprising:

a. a first data storage medium wherein said first storage medium is transportable;
b. a first large database containing data records in a first binary form, wherein said first database requires more memory for its storage than is available on said first data storage medium;
c. a first computer system configured to receive said first data storage medium, said first computer system further comprising:
(1) a second data storage medium, of greater memory capacity than said first storage medium, wherein said second storage medium contains said first large database;
(2) wherein said first computer system is utilized to
(a) order the data of each record of said first database into a predetermined sequence to form data segments wherein the data segments are arranged based on common characteristics of the data segments;
(b) repeat the order process within each data segment such that each data segment is characterized by sub-segments arranged based on common characteristics of the data strings which comprise each segment;
(c) encode each data segment into a second binary form which requires less memory to store said data than was required by said data in said first binary form wherein the second binary form is such that each character is represented by a binary number smaller than would be required to represent the character in another binary form;
(d) insert a delineator at the beginning of each segment and sub-segment, wherein the value of each delineator allows the delineator to also functions as a pointer or contain information about the subsequent data segment to which the delineator has been appended;
(e) compress the encoded segments of each record into a continuous data string wherein the delineators separate individual data segments within each continuous data string;
(f) append an index table at the beginning of each continuous data string;
(g) remove equivalent characters from the beginning of each data segment within a data string and record the equivalent character in the index table;
(h) record in the index table of each data string the length of a data segment in a data string when the data string is comprised of equivalent length data segments;
(i) link the data strings by generating and appending additional data to each data string which relates the individual data strings;
(j) append additional empty data places to each string for receiving location dependent data;
(k) partition the data strings into blocks of data based on the physical characteristics of said first data storage medium such that said first data storage medium is optimally filled wherein the physical characteristics of said first storage medium are utilized to determine the break points within data strings and the location of each block in said first storage medium such that blocks which have related data are physically located adjacent one another on said first storage medium;
(l) generate location dependent data to fill the empty data places of each data string wherein said location dependent data is generated based on the assigned physical location of each block on said first storage medium;
and
(m) write the blocks of data to their predetermined locations on said first storage medium;
d. a second computer system configured to receive said first data storage medium and retrieve data from said first storage medium, wherein said second computer has a smaller computing capacity than said first computer system.

17 more elements

Dependencies are inferred from the claim text.

System for the compacting and logical linking of data blocks in files to optimize available physical storage

Life of this patent

System for the compacting and logical linking of data blocks in files to optimize available physical storage

Life of this patent

Abstract

Claims20