# US 2012/0254253 A1 · Disk-Resident Streaming Dictionary

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0254253 A1
- Document ID: US20120254253A1
- Status: Published application
- Kind: A1
- Application number: 13/476,826
- Filed: 2012-05-21
- Published: 2012-10-04
- Earliest claimed: 2007-06-08
- Assignees: Massachusetts Institute of Technology; The Research Foundation of State University of New York; Rutgers University
- Inventors: Bradley C. Kuszmaul; Michael A. Bender; Martin Farach-Colton
- Web page: https://patentgrep.com/patent/US20120254253A1

## CPC

- G06F 11/1471: involving logging of persistent data for recovery
- G06F 16/2246: {Trees, e.g. B+trees}
- G06F 16/2272: {Management thereof}
- G06F 2201/82: Solving problems relating to consistency
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method, apparatus and computer program product for storing data in a disk storage system is presented. A dictionary data structure is defined and stored on the disk storage system. Key-value pairs can be inserted and deleted into the dictionary data structure, with full transactional semantics, at a rate that is faster than one insertion per disk-head movement. Keys can be looked up with only a logarithmic number of transfers, even for keys that have been recently inserted or deleted. Queries can be performed on ranges of key-value pairs, including recently inserted or deleted pairs, at a constant fraction of the bandwidth of the disk. The dictionary employs indirect logging for physical block logging.

## Claims

### Claim 1

A method organizing data in a disk storage system comprising:
organizing a plurality of key-value pairs, wherein a total ordering is applied upon the respective keys via placing respective key-value pairs in nodes structured as a tree, wherein the tree comprises at least one leaf node and at least one non-leaf node, wherein the leaf node includes at least one respective key-value pair, and wherein the non-leaf nodes comprise a buffer, a child pointer sequence and at least one pivot key from a pivot key sequence, wherein the leaf node comprises a root of a subtree;
wherein the buffer contains space for at least one of: an insert command, a delete command, and a different type of command, wherein the insert command is associated with a particular key-value pair, and the delete command is associated with a particular key, each child pointer from the child pointer sequence identifying a child node comprising a root of a child subtree;
wherein, for each child pointer occurring after a primary child pointer in the child pointer sequence, there is a corresponding pivot key ordered before a pivot key associated with any subsequent child pointer occurring in the child pointer sequence, wherein the keys stored in a respective subtree referenced by a child pointer are ordered greater than any pivot key associated with the child pointer or any previous child pointer, wherein the keys stored in the subtree are ordered less than or equal to the pivot key associated with any subsequent child pointer;
wherein a first non-leaf node of the tree includes a first pivot key associated with a child pointer referencing a root of a first subtree, a second pivot key associated with a child pointer referencing a root of a second subtree and a first buffer, where the first pivot key is ordered in the first non-leaf node before the second pivot key according to the pivot key sequence;
wherein the root of the first subtree includes a third pivot key and (ii) a second buffer, wherein the pivot key sequence orders the third pivot key between the first pivot key and the second pivot key;
wherein the first subtree includes a first leaf node for storing at least one key-value pair having a key corresponding to a position in the pivot key sequence occurring between the first pivot key and the third pivot key and a second leaf node for storing at least one key-value pair having a key corresponding to a position in the pivot key sequence occurring between the third pivot key and the second pivot key;
wherein the first buffer includes an insert command identifying the key and a value of a first key-value pair currently in the first leaf node;
wherein the second buffer includes a delete command identifying the key of the first key-value pair;
upon detecting the first buffer has reached a threshold capacity, pushing the insert command from the first buffer to the second buffer of the first subtree; and
upon detecting the insert command in the second buffer, removing the delete command from the second buffer based on the insert command being newer than the delete command.

### Claim 2 (depends on 1)

The method of claim 1 further comprising:
in which to look up a key in a subtree rooted at a node, the buffer of the node is searched for via a command containing the key,
returning the key-value pair of the command if the command containing the key is found and the command containing the key is an insert command,
returning an indication that the key is not found if the command containing the key is a delete command; and
upon returning the indication that the key is not found, utilizing the respective pivot keys of the node to determine an appropriate child node, and the key is then looked up in the subtree rooted at the child node.

### Claim 3 (depends on 2)

The method of claim 2, further comprising in which upon inserting a command into the buffer of the root node of a subtree, if the buffer overflows, then a child node which is the root of a child subtree is chosen, and some commands are pushed from the buffer of the root of the subtree into the child subtree, removing the commands. from the buffer of the root of the subtree,
where to push a command into a non-leaf node inserts the command into the buffer of the node.

### Claim 4 (depends on 3)

The method of claim 3 further comprising in which to insert a key-value pair into a subtree, a command to insert the key-value pair is pushed into the root node of the subtree.

### Claim 5 (depends on 4)

The method of claim 4 further comprising to push an insert command into a leaf node, the corresponding key-value pair is inserted into the set of key-value pairs of the leaf node, removing any previously extant key-value pairs that have the same key from the set.

### Claim 6 (depends on 5)

The method of claim 5 further comprising in which to delete a key-value pair from a subtree, a delete command identifying the key is pushed into the root node of the subtree.

### Claim 7 (depends on 6)

The method of claim 6 further comprising to push a delete command into a leaf node, any key-value pair with a matching key is removed from the set of key-value pairs of the leaf node.

### Claim 8 (depends on 1)

The method of claim 1, further comprising in which upon inserting a command into the buffer of the root node of a subtree, if the buffer overflows, then a child node which is the root of a child subtree is chosen, and some commands are pushed from the buffer of the root of the subtree into the child subtree, removing the commands from the buffer of the root of the subtree,
where to push a command into a non-leaf node inserts the command into the buffer of the node.

### Claim 9 (depends on 8)

The method of claim 8 further comprising in which to insert a key-value pair into a subtree, a command to insert the key-value pair is pushed into the soot node of the subtree.

### Claim 10 (depends on 9)

The method of claim 9 further comprising to push an insert command into a leaf node, the corresponding key-value pair is inserted into the set of key-value pairs of the leaf node, removing any previously extant key-value pairs that have the same key from the set.

### Claim 11 (depends on 8)

The method of claim 8 further comprising in which to delete a key-value pair from a subtree, a delete command identifying the key is pushed into the root node of the subtree.

### Claim 12 (depends on 11)

The method of claim 11 further comprising to push a delete command into a leaf node, any key-value pair with a matching key is removed from the set of key-value pairs of the leaf node.

### Claim 13

A non-transitory computer readable medium having computer readable code thereon for organizing data in a disk storage system comprising:
instructions for organizing a plurality of key-value pairs, wherein a total ordering is applied upon the respective keys via placing respective key-value pairs in nodes structured as a tree, wherein the tree comprises at least one leaf node and at least one non-leaf node, wherein the leaf node includes at least one respective key-value pair, and wherein the non-leaf nodes comprise a buffer, a child pointer sequence and at least one pivot key from a pivot key sequence, wherein the leaf node comprises a root of a subtree;
wherein the buffer contains space for at least one of: an insert command, a delete command, and a different type of command, wherein the insert command is associated with a particular key-value pair, and the delete command is associated with a particular key, each child pointer from the child pointer sequence identifying a child node comprising a root of a child subtree;
wherein, for each child pointeroccurring after a primary child pointer in the child pointer sequence, there is a corresponding pivot key ordered before a pivot key associated with any subsequent child pointer occurring in the child pointer sequence, wherein the keys stored in a respective subtree referenced by a child pointer are ordered greater than any pivot key associated with the child pointer or any previous child pointer, wherein the keys stored in the subtree are ordered less than or equal to the pivot key associated with any subsequent child pointer;
wherein a first non-leaf node of the tree includes a first pivot key associated with a child pointer referencing a root of a first subtree, a second pivot key associated with a child pointer referencing a root of a second subtree and a first buffer, where the first pivot key is ordered in the first non-leaf node before the second pivot key according to the pivot key sequence;
wherein the root of the first subtree includes a third pivot key and a second buffer, wherein the pivot key sequence orders the third pivot key between the first pivot key and the second pivot key;
wherein the first subtree includes a first leaf node for storing at least one key-value pair having a key corresponding to a position in the pivot key sequence occurring between the first pivot key and the third pivot key and a second leaf node for storing at least one key-value pair having a key corresponding to a position in the pivot key sequence occurring between the third pivot key and the second pivot key;
wherein the first buffer includes an insert command identifying the key and a value of a first key-value pair currently in the first leaf node;
wherein the second buffer includes a delete command identifying the key of the first key-value pair;
instructions for pushing the insert command from the first buffer to the second buffer of the first subtree upon detecting the first buffer has reached a threshold capacity; and
instructions for removing the delete command from the second buffer, upon detecting the insert command has entered the second buffer, based on the insert command being newer than the delete command.

### Claim 14 (depends on 13)

The computer readable medium of claim 13 further comprising instructions for in which to look up a key in a subtree rooted at a node, the buffer of the node is searched for via a command containing the key,
returning the key-value pair of the command if the command containing the key is found and the command containing the key is an insert command,
returning an indication that the key is not found if the command containing the key is a delete command; and
upon returning the indication that the key is not found, utilizing the respective pivot keys of the node to determine an appropriate child node, and the key is then looked up in the subtree rooted at the child node.

### Claim 15 (depends on 14)

The computer readable medium of claim 14, further comprising instructions for in which upon inserting a command into the buffer of the root node of a subtree, if the buffer overflows, then a child node which is the root of a child subtree is chosen, and some commands are pushed from the buffer of the root of the subtree into the child subtree, removing the commands from the buffer of the root of the subtree,
where to push a command into a non-leaf node inserts the command into the buffer of the node.

### Claim 16 (depends on 15)

The computer readable medium of claim 15 further comprising instructions for in which to insert a key-value pair into a subtree, a command to insert the key-value pair is pushed into the root node of the subtree.

### Claim 17 (depends on 16)

The computer readable medium of claim 16 further comprising instructions for to push an insert command into a leaf node, the corresponding key-value pair is inserted into the set of key-value pairs of the leaf node, removing any previously extant key-value pairs that have the same key from the set.

### Claim 18 (depends on 17)

The computer readable medium of claim 17 further comprising instructions for in which to delete a key-value pair from a subtree, a delete command identifying the key is pushed into the root node of the subtree.

### Claim 19 (depends on 18)

The computer readable medium of claim 18 further comprising instructions for to push a delete command into a leaf node, any key-value pair with a matching key is removed from the set of key-value pairs of the leaf node.

### Claim 20 (depends on 13)

The computer readable medium of claim 13, further comprising instructions for in which upon inserting a command into the buffer of the root node of a subtree, if the buffer over-flows, then a child node which is the root of a child subtree is chosen, and some commands are pushed from the buffer of the root of the subtree into the child subtree, removing the commands from the buffer of the root of the subtree,
where to push a command into a non-leaf node inserts the command into the buffer of the node.

### Claim 21 (depends on 20)

The computer readable medium of claim 20 further comprising instructions for in which to insert a key-value pair into a subtree, a command to insert the key-value pair is pushed into the root node of the subtree.

### Claim 22 (depends on 21)

The computer readable medium of claim 21 further comprising instructions for to push an insert command into a leaf node, the corresponding key-value pair is inserted into the set of key-value pairs of the leaf node, removing any previously extant key-value pairs that have the same key from the set.

### Claim 23 (depends on 20)

The computer readable medium of claim 20 further comprising instructions for in which to delete a key-value pair from a subtree, a delete command identifying the key is pushed into the root node of the subtree.

### Claim 24 (depends on 23)

The computer readable medium of claim 23 further comprising instructions for to push a delete command into a leaf node, any key-value pair with a matching key is removed from the set of key-value pairs of the leaf node.

### Claim 25

A computer system comprising a processor; a main memory; a disk; and wherein the memory is encoded with an application for storing data on the disk, that when performed on the processor, provides a process for processing information, the process causing the computer system to perform the operations of organizing data in a disk storage system comprising:
organizing a plurality of key-value pairs, wherein a total ordering is applied upon the respective keys via placing respective key-value pairs in nodes structured as a tree, wherein the tree comprises at least one leaf node and at least one non-leaf node, wherein the leaf node includes at least one respective key-value pair, and wherein the non-leaf nodes comprise a buffer, a child pointer sequence and at least one pivot key from a pivot key sequence, wherein the leaf node comprises a root of a subtree;
wherein the buffer contains space for at least one of: an insert command, a delete command, and a different type of command, wherein the insert command is associated with a particular key-value pair, and the delete command is associated with a particular key, each child pointer from the child pointer sequence identifying a child node comprising a root of a child subtree;
wherein, for each child pointer occurring after a primary child pointer in the child pointer sequence, there is a corresponding pivot key ordered before a pivot key associated with any subsequent child pointer occurring in the child pointer sequence, wherein the keys stored in a respective subtree referenced by a child pointer are ordered greater than any pivot key associated with the child pointer or any previous child pointer, wherein the keys stored in the subtree are ordered less than or equal to the pivot key associated with any subsequent child pointer;
wherein a first non-leaf node of the tree includes a first pivot key associated with a child pointer referencing a root of a first subtree, a second pivot key associated with a child pointer referencing a root of a second subtree and a first buffer, where the first pivot key is ordered in the first non-leaf node before the second pivot key according to the pivot key sequence;
wherein the root of the first subtree includes a third pivot key and a second buffer, wherein the pivot key sequence orders the third pivot key between the first pivot key and the second pivot key;
wherein the first subtree includes a first leaf node for storing at least one key-value pair having a key corresponding to a position in the pivot key sequence occurring between the first pivot key and the third pivot key and a second leaf node for storing at least one key-value pair having a key corresponding to a position in the pivot key sequence occurring between the third pivot key and the second pivot key;
wherein the first buffer includes an insert command identifying the key and a value of a first key-value pair currently in the first leaf node;
wherein the second buffer includes a delete command identifying the key of the first key-value pair;
upon detecting the first buffer has reached a threshold capacity, pushing the insert command from the first buffer to the second buffer of the first subtree; and
upon detecting the insert command in the second buffer, removing the delete command from the second buffer based on the insert command being newer than the delete command.

### Claim 26 (depends on 25)

The computer system of claim 25 further comprising in which to look up a key in a subtree rooted at a node, the buffer of the node is searched for via a command containing the key,
returning the key-value pair of the command if the command containing the key is found and the command containing the key is an insert command,
returning an indication that the key is not found if the command containing the key is a delete command; and
upon returning the indication that the key is not found, utilizing the respective pivot keys of the node to determine an appropriate child node, and the key is then looked up in the subtree rooted at the child node.

### Claim 27 (depends on 26)

The computer system of claim 26, further comprising in which upon inserting a command into the buffer of the root node of a subtree, if the buffer overflows, then a child node which is the root of a child subtree is chosen, and some commands are pushed from the buffer of the root of the subtree into the child subtree, removing the commands from the buffer of the root of the subtree,
where to push a command into a non-leaf node inserts the command into the buffer of the node.

### Claim 28 (depends on 27)

The computer system of claim 27 further comprising in which to insert a key-value pair into a subtree, a command to insert the key-value pair is pushed into the root node of the subtree.

### Claim 29 (depends on 28)

The computer system of claim 28 further comprising to push an insert command into a leaf node, the corresponding key-value pair is inserted into the set of key-value pairs of the leaf node, removing any previously extant key-value pairs that have the same key from the set.

### Claim 30 (depends on 29)

The computer system of claim 29 further comprising in which to delete a key-value pair from a subtree, a delete command identifying the key is pushed into the root node of the subtree.

### Claim 31 (depends on 30)

The computer system of claim 30 further comprising to push a delete command into a leaf node, any key-value pair with a matching key is removed from the set of key-value pairs of the leaf node.

### Claim 32 (depends on 25)

The computer system of claim 25, further comprising in which upon inserting a command into the buffer of the root node of a subtree, if the buffer overflows, then a child node which is the root of a child subtree is chosen, and some commands are pushed from the buffer of the root of the subtree into the child subtree, removing the commands from the buffer of the root of the subtree,
where to push a command into a non-leaf node inserts the command into the buffer of the node.

### Claim 33 (depends on 32)

The computer system of claim 32 further comprising in which to insert a key-value pair into a subtree, a command to insert the key-value pair is pushed into the root node of the subtree.

### Claim 34 (depends on 33)

The computer system of claim 33 further comprising to push an insert command into a leaf node, the corresponding key-value pair is inserted into the set of key-value pairs of the leaf node, removing any previously extant key-value pairs that have the same key from the set.

### Claim 35 (depends on 32)

The computer system of claim 32 further comprising in which to delete a key-value pair from a subtree, a delete command identifying the key is pushed into the root node of the subtree.

### Claim 36 (depends on 35)

The computer system of claim 35 further comprising to push a delete command into a leaf node, any key-value pair with a matching key is removed from the set of key-value pairs of the leaf node.

### Claim 37 (depends on 8)

The method as in claim 8, wherein inserting the command includes:
inserting a first command into the buffer of the root node of the subtree, the buffer of the root node of the subtree already containing a second command; and
as the some commands are pushed from the buffer of the root of the subtree: removing the first command from the buffer of the root node of the subtree; inserting the first command into the buffer of the node; and keeping the second command in the buffer of the root node of the subtree.
