# US 2024/0004894 A1 · Systems and methods for improved blockchain data indexing by decoupling compute and storage layers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004894 A1
- Document ID: US20240004894A1
- Status: Published application
- Kind: A1
- Application number: 17/810,040
- Filed: 2022-06-30
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: Coinbase, Inc.
- Applicants: Coinbase, Inc.
- Inventors: Jie ZHANG; Zhicong LIANG; Yaohua YANG; David LAI; Chaoqing LU; Jinghan XU; Xu MENG
- Web page: https://patentgrep.com/patent/US20240004894A1

## CPC

- G06F 16/258: {Data format conversion from or to a database}
- G06F 16/1794: {Details of file format conversion}
- G06F 16/27: Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F 16/273: {Asynchronous replication or reconciliation}
- G06F 16/275: {Synchronous replication}
- G06F 16/278: {Data partitioning, e.g. horizontal or vertical partitioning}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods for a unified approach that is compatible with all blockchains, protocols, etc. The systems and methods use a bifurcated indexing system with a dynamically selected application service. The systems and methods bifurcate the indexing process into a storage layer and a compute layer.

## Claims

### Claim 1

A system for improved blockchain data indexing by decoupling compute and storage layers, the system comprising:
a multi-layer data platform for indexing on-chain data;
a first dataset, wherein the first dataset is populated based on:
receiving first on-chain data from a blockchain node of a blockchain network, wherein the first on-chain data comprises hexadecimal encoded data from a first block of the blockchain network;
transforming the first on-chain data to a first format, and wherein the first format comprises data types with field names identified by a respective integer;
storing the first on-chain data, in the first dataset at a blockchain-interface layer, wherein the first dataset comprises the hexadecimal encoded data, and wherein the first dataset uses the first format;
retrieving a first compute engine, wherein the first compute engine comprises a first workflow architecture, wherein the first workflow architecture comprises a first threshold for workflow throughout and a first threshold for a number of workflows;
a second dataset, wherein the second dataset is populated based on:
transforming the first on-chain data in the first format to a second format, wherein the second format comprises a columnar oriented format;
storing the first on-chain data, in the second dataset at a data lakehouse layer, wherein the second dataset comprises the first on-chain data and second on-chain data in the second format, and wherein the second on-chain data is from a second block on the blockchain network;
retrieving a second compute engine, wherein the second compute engine comprises a second workflow architecture, wherein the second workflow architecture comprises a second threshold for workflow throughout and a second threshold for the number of workflows, wherein the second threshold for workflow throughput is higher than the first threshold for workflow throughput, and wherein the second threshold for the number of workflows is lower than the first threshold for the number of workflows; and
a third dataset, wherein the third dataset is populated based on:
transforming the first on-chain data and the second on-chain data to a third format, wherein the third format is dynamically selected based on an application;
storing the first on-chain data and the second on-chain data, in the third dataset at an application service layer;
retrieving a third compute engine, wherein the third compute engine comprises the second workflow architecture.

### Claim 2

A method for improved blockchain data indexing by decoupling compute and storage layers, the method comprising:
receiving, at a blockchain-interface layer, first on-chain data from a blockchain node of a blockchain network, wherein the first on-chain data comprises hexadecimal encoded data from a first block of the blockchain network, wherein the blockchain-interface layer transforms the first on-chain data to a first format, using a first compute engine, for storage in a first dataset, and wherein the first format comprises data types with field names identified by a respective integer;
receiving, at a data lakehouse layer, the first on-chain data in the first format, wherein the data lakehouse layer transforms the first on-chain data to a second format, using a second compute engine, for storage in a second dataset, wherein the second format comprises a columnar oriented format, wherein the second dataset comprises the first on-chain data and second on-chain data, and wherein the second on-chain data is from a second block on the blockchain network;
determining an application characteristic for an application that performs blockchain operations using the first on-chain data or the second on-chain data;
receiving, at an application service layer, the first on-chain data and the second on-chain data in the second format, wherein the application service layer transforms, using a third compute engine, the first on-chain data and the second on-chain data to a third format for storage in a third dataset, and wherein the third format is dynamically selected based on the application characteristic; and
transmitting the first on-chain data and the second on-chain data in the third format to the application.

### Claim 3 (depends on 2)

The method of claim 2, wherein determining the application characteristic further comprises:
determining a data freshness requirement for the application; and
selecting the third format from a plurality of formats based on the third format corresponding to the data freshness requirement.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
receiving a request, from the application, for the first on-chain data and the second on-chain data; and
selecting between the blockchain node, the first dataset, the second dataset, and the third dataset for responding to the request based on the application characteristic.

### Claim 5 (depends on 2)

The method of claim 2, wherein the data lakehouse layer comprises a combination of a data lake with a data warehouse in a single data platform.

### Claim 6 (depends on 2)

The method of claim 2, wherein transforming the first on-chain data to the first format comprises:
receiving unstructured on-chain data;
determining a structuring condition based on the first format; and
applying the structuring condition to the unstructured on-chain data to generate structured on-chain data.

### Claim 7 (depends on 2)

The method of claim 2, wherein transforming the first on-chain data to the first format comprises:
receiving unstructured on-chain data;
parsing the unstructured on-chain data for an unstructured on-chain data characteristic; and
generating a semantic marker for the unstructured on-chain data characteristic, wherein the semantic marker is stored in the first dataset.

### Claim 8 (depends on 2)

The method of claim 2, wherein receiving the first on-chain data from the blockchain node comprises:
selecting the first block of the blockchain network;
querying the first block for available data matching a retrieval criterion; and
executing a retrieval operation to retrieve any available data matching the retrieval criterion.

### Claim 9 (depends on 2)

The method of claim 2, wherein:
the first compute engine comprises a first workflow architecture, wherein the first workflow architecture comprises a first threshold for workflow throughout and a first threshold for a number of workflows;
wherein the second compute engine comprises a second workflow architecture, wherein the second workflow architecture comprises a second threshold for workflow throughout and a second threshold for the number of workflows, wherein the second threshold for workflow throughput is higher than the first threshold for workflow throughput, and wherein the second threshold for the number of workflows is lower than the first threshold for the number of workflows; and
wherein the third compute engine comprises the second workflow architecture.

### Claim 10 (depends on 2)

The method of claim 2, wherein the first dataset maintains the first on-chain data as the hexadecimal encoded data while in the first format, and wherein the second dataset does not maintain the first on-chain data as the hexadecimal encoded data while in the second format.

### Claim 11 (depends on 2)

The method of claim 2, further comprising:
designating a first blockchain node of a plurality of blockchain nodes for a blockchain network as having a first node type; and
based on designating the first blockchain node of the plurality of blockchain nodes as having the first node type, establishing a session with the first blockchain node.

### Claim 12

A non-transitory computer-readable medium having instructions recorded thereon that when executed by one or more processors causes operations comprising:
receiving, at a blockchain-interface layer, first on-chain data from a blockchain node of a blockchain network, wherein the first on-chain data comprises hexadecimal encoded data from a first block of the blockchain network, wherein the blockchain-interface layer transforms the first on-chain data to a first format, using a first compute engine, for storage in a first dataset, and wherein the first format comprises data types with field names identified by a respective integer;
receiving, at a data lakehouse layer, the first on-chain data in the first format, wherein the data lakehouse layer transforms the first on-chain data to a second format, using a second compute engine, for storage in a second dataset, wherein the second format comprises a columnar oriented format, wherein the second dataset comprises the first on-chain data and second on-chain data, and wherein the second on-chain data is from a second block on the blockchain network;
determining an application characteristic for an application that performs blockchain operations using the first on-chain data or the second on-chain data;
receiving, at an application service layer, the first on-chain data and the second on-chain data in the second format, wherein the application service layer transforms, using a third compute engine, the first on-chain data and the second on-chain data to a third format for storage in a third dataset, and wherein the third format is dynamically selected based on the application characteristic; and
transmitting the first on-chain data and the second on-chain data in the third format to the application.

### Claim 13 (depends on 12)

The non-transitory, computer-readable medium of claim 12, wherein determining the application characteristic further comprises:
determining a data freshness requirement for the application; and
selecting the third format from a plurality of formats based on the third format corresponding to the data freshness requirement.

### Claim 14 (depends on 12)

The non-transitory, computer-readable medium of claim 12, wherein the instructions further cause an operation comprising:
receiving a request, from the application, for the first on-chain data and the second on-chain data; and
selecting between the blockchain node, the first dataset, the second dataset, and the third dataset for responding to the request based on the application characteristic.

### Claim 15 (depends on 12)

The non-transitory, computer-readable medium of claim 12, wherein the data lakehouse layer comprises a combination of a data lake with a data warehouse in a single data platform.

### Claim 16 (depends on 12)

The non-transitory, computer-readable medium of claim 12, wherein transforming the first on-chain data to the first format comprises:
receiving unstructured on-chain data;
determining a structuring condition based on the first format; and
applying the structuring condition to the unstructured on-chain data to generate structured on-chain data.

### Claim 17 (depends on 12)

The non-transitory, computer-readable medium of claim 12, wherein transforming the first on-chain data to the first format comprises:
receiving unstructured on-chain data;
parsing the unstructured on-chain data for an unstructured on-chain data characteristic; and
generating a semantic marker for the unstructured on-chain data characteristic, wherein the semantic marker is stored in the first dataset.

### Claim 18 (depends on 12)

The non-transitory, computer-readable medium of claim 12, wherein:
the first compute engine comprises a first workflow architecture, wherein the first workflow architecture comprises a first threshold for workflow throughout and a first threshold for a number of workflows;
wherein the second compute engine comprises a second workflow architecture, wherein the second workflow architecture comprises a second threshold for workflow throughout and a second threshold for the number of workflows, wherein the second threshold for workflow throughput is higher than the first threshold for workflow throughput, and wherein the second threshold for the number of workflows is lower than the first threshold for the number of workflows; and
wherein the third compute engine comprises the second workflow architecture.

### Claim 19 (depends on 12)

The non-transitory, computer-readable medium of claim 12, wherein the first dataset maintains the first on-chain data as the hexadecimal encoded data while in the first format, and wherein the second dataset does not maintain the first on-chain data as the hexadecimal encoded data while in the second format.

### Claim 20 (depends on 12)

The non-transitory, computer-readable medium of claim 12, wherein the instructions further cause an operation comprising:
designating a first blockchain node of a plurality of blockchain nodes for a blockchain network as having a first node type; and
based on designating the first blockchain node of the plurality of blockchain nodes as having the first node type, establishing a session with the first blockchain node.
