# US 2024/0202163 A1 · System and method for random-access manipulation of compacted data files

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202163 A1
- Document ID: US20240202163A1
- Status: Published application
- Kind: A1
- Application number: 18/412,439
- Filed: 2024-01-12
- Published: 2024-06-20
- Earliest claimed: 2017-10-30
- Assignees: Atombeam Technologies Inc.
- Applicants: AtomBeam Technologies Inc.
- Inventors: Joshua Cooper; Charles Yeomans
- Web page: https://patentgrep.com/patent/US20240202163A1

## CPC

- G06F 16/1752: {based on file chunks}
- G06F 3/0608: {Saving storage space on storage systems}
- G06F 3/0641: {De-duplication techniques}
- G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}
- G06F 21/606: {by securing the transmission between two devices or processes}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method for random-access manipulation of compacted data files, utilizing a reference codebook, a random-access engine, a data deconstruction engine, and a data deconstruction engine. The system may receive a data query pertaining to a data read or data write request, wherein the data file to be read from or written to is a compacted data file. A random-access engine may facilitate data manipulation processes by transforming the codebook into a hierarchical representation and then traversing the representation scanning for specific codewords associated with a data query request. In an embodiment, an estimator module is present and configured to utilize cardinality estimation to determine a starting codeword to begin searching the compacted data file for the data associated with the data query. The random-access engine may encode the data to be written, insert the encoded data into a compacted data file, and update the codebook as needed.

## Claims

### Claim 1

A system for random-access manipulation of a compacted data file, comprising:
a computing device comprising a memory, a processor, and a non-volatile data storage device;
the random access engine comprising a first plurality of programming instructions stored in the memory and operating on the processor, wherein the first plurality of programming instructions, when operating on the processor, cause the computing device to:
receive a data search query, the data search query being directed to the compacted data file, wherein the compacted file is compacted using a reference codebook;
organize the reference codebook into a hierarchical representation;
traverse the hierarchical representation until a start codeword corresponding to the beginning of the data search query is identified; and
send the start codeword and a plurality of immediately following codewords from the compacted data file to a decoder.

### Claim 2 (depends on 1)

The system of claim 1, further comprising a deconstruction engine comprising a second plurality of programming instructions stored in the memory and operable on the processor, wherein the second plurality of programming instructions, when operating on the processor, cause the computing device to:
deconstruct a data stream into a plurality of data sourceblocks;
create the compacted data file by compacting the plurality of data sourceblocks using the reference codebook; and
send the reference codebook to the random access engine.

### Claim 3 (depends on 1)

The system of claim 1, further comprising an estimator module comprising a third plurality of programming instructions stored in the memory and operable on the processor, wherein the third plurality of programming instructions, when operating on the processor, cause the computing device to:
estimate a first starting bit location in the compacted data file;
refine the first starting bit location by:
determining a plurality of codeword boundaries by performing distinct value estimation on the compacted data file, wherein the distinct value estimates correspond to a codeword boundary; and
determining whether a bit sequence starting at the first starting bit location corresponds to a codeword boundary of the plurality of codeword boundaries and, if not, traversing the hierarchical representation until a codeword boundary is located at a new starting bit.

### Claim 4 (depends on 3)

The system of claim 3, wherein the estimator module sends the codeword boundary estimate to the random access engine.

### Claim 5 (depends on 4)

The system of claim 4, wherein the random access engine uses the codeword boundary estimate as an initial point to begin traversing the hierarchical representation.

### Claim 6

A method for random-access manipulation of a compacted data file, comprising the steps of:
receiving a data search query, the data search query being directed to the compacted data file, wherein the compacted file is compacted using a reference codebook;
organizing the reference codebook into a hierarchical representation;
traversing the hierarchical representation until a start codeword corresponding to the beginning of the data search query is identified; and
sending the start codeword and a plurality of immediately following codewords from the compacted data file to a decoder.

### Claim 7 (depends on 6)

The method of claim 6, further comprising the steps of:
deconstructing a data stream into a plurality of data sourceblocks;
creating the compacted data file by compacting the plurality of data sourceblocks using the reference codebook; and
sending the reference codebook to the random access engine.

### Claim 8 (depends on 6)

The method of claim 6, further comprising the steps of:
estimating, using an estimator module, a first starting bit location in the compacted data file; and
refining the first starting bit location by:
determining a plurality of codeword boundaries by performing distinct value estimation on the compacted data file, wherein the distinct value estimates correspond to a codeword boundary; and
determining whether a bit sequence starting at the first starting bit location corresponds to a codeword boundary of the plurality of codeword boundaries and, if not, traversing the hierarchical representation until a codeword boundary is located at a new starting bit.

### Claim 9 (depends on 8)

The method of claim 8, wherein the codeword boundary estimate is used as an initial point to begin traversing the hierarchical representation.
