# US 2026/0037336 A1 · Big Data processing platform and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0037336 A1
- Document ID: US20260037336A1
- Status: Published application
- Kind: A1
- Application number: 19/286,313
- Filed: 2025-07-31
- Published: 2026-02-05
- Earliest claimed: 2024-08-01
- Inventors: Daniel Jaye; Sakthi Priyan Hari Krishnan; Sudheer Pendyala; Sundara Raghavan Sankaran
- Web page: https://patentgrep.com/patent/US20260037336A1

## CPC

- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 9/5066: {Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451)}
- G06F 9/5072: {Grid computing}
- G06F 16/221: {Column-oriented storage; Management thereof}
- G06F 16/2477: {Temporal data queries}
- G06F 2209/5017: Task decomposition
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A highly efficient, scalable, and adaptable framework suitable for modern Big Data processing challenges in cloud-based environments. The framework preferably leverages a scale-up before scale-out paradigm that prioritizes scaling up individual nodes (e.g., with high-core-count processors) before scaling out across multiple nodes. In addition, preferably the framework moves data to compute nodes, e.g., utilizing high-bandwidth cloud storage networks for efficient data transfer. Further, the framework preferably employs specialized primitives optimized for specific Big Data processing tasks, particularly for collations, sorts, and shuffles. Finally, framework utilizes optimized libraries for a small number of file formats.

## Claims

### Claim 1

A computing system operating in association with a cloud-based environment, comprising:
A compute layer implemented on hardware and software and configured with multi-executor orchestration to provide ingestion, transformation and collation of time-series event data, and;
a storage layer comprising a first data store, and a second data store, the first data store providing a system of record for harmonized data derived from the time-series event data, the harmonized data stored in an immutable, row-oriented data format, and the second data store providing an analytic environment and storing time-series event data in a column-oriented data format accessible utilizing columnar metadata; and
wherein multi-executor orchestration splits a workload into a series of tasks, assigns each task in the set of tasks to an executor of a set of executors, and receives and aggregates results from execution of the tasks by the set of executors.

### Claim 2 (depends on 1)

The computing system as described in claim 1, wherein the row-oriented data format is Avro, and wherein the column-oriented data format is Parquet.

### Claim 3 (depends on 1)

The computing system as described in claim 1, wherein the cloud-based environment comprises a set of compute nodes, and cloud storage, the set of compute nodes accessible via a set of network connections.

### Claim 4 (depends on 1)

The computing system as described in claim 1, wherein transformation and collation of time-series event data occurs on a first compute node of the set of compute nodes following on-demand transfer of time-series event data from the first data store to the at least one compute node.

### Claim 5 (depends on 4)

The computing system as described in claim 4, wherein the first compute node of the set of compute nodes is a host machine on which a given number of virtual cores are configurable for execution.

### Claim 6 (depends on 5)

The computing system as described in claim 5, wherein an executor executes on a virtual core of the host machine.

### Claim 7 (depends on 5)

The computing system as described in claim 5, wherein multi-executor orchestration scales up one or more virtual cores on the first compute node as necessary to execute the workload.

### Claim 8 (depends on 7)

The computing system as described in claim 7, wherein multi-executor orchestration scales out to a second compute node when the given number of virtual cores in the first compute node are not sufficient to execute the workload.

### Claim 9 (depends on 1)

The computing system as described in claim 1, wherein the multi-execution orchestration splits the workload across a swarm of serverless functions associated with the cloud-based environment.

### Claim 10 (depends on 1)

The computing system as described in claim 1, wherein time-series event data is stored in a hybrid row-columnar format.

### Claim 11 (depends on 10)

The computing system as described in claim 10, wherein the hybrid row-columnar format stores segmented embedded vectors.

### Claim 12 (depends on 2)

The computing system as described in claim 2, wherein the columnar metadata is configured to bypass Parquet file format scanning.

### Claim 13 (depends on 3)

The computing system as described in claim 3, wherein the cloud storage is co-located in a region with at least one node of the set of compute nodes.

### Claim 14

A method of processing in association with a cloud compute infrastructure with network connections to a cloud storage, comprising:
receiving and storing in cloud storage a dataset, wherein row-oriented data in the dataset is stored for access according to a given row-based file format, and wherein column-oriented data in the dataset is stored for access according to a given column-based file format;
in response to receipt of a request to process a workload, moving at least a portion of the dataset to one or more compute nodes within the cloud compute infrastructure via the network connections; and
processing the request by splitting the workload into a set of tasks, assigning each task in the set of tasks to an executor of a set of executors, and receiving and aggregating results from execution of the tasks by the set of executors;
wherein processing includes at least one executor applying a data processing primitive that is one of: a homogeneous collation primitive, a heterogeneous collation primitive for sort-merge joins, a presorted record merge primitive, and a geospatial processing primitive.

### Claim 15 (depends on 14)

The method as described in claim 14, wherein the given row-based file format is Avro, and the given column-based file format is Parquet.

### Claim 16 (depends on 14)

The method as described in claim 14 wherein the primitive is one of a set of primitives, the set of primitives are implemented across the one or more compute nodes using concurrent threads executing up to thousands of concurrent tasks per compute node.

### Claim 17 (depends on 14)

The method as described in claim 14, wherein data stored in the given row-based file format and given column-based file format are accessed according to requests that conform to a procedural object-oriented language.

### Claim 18 (depends on 17)

The method as described in claim 17, wherein the procedural object-oriented language is GoLang, and wherein GoLang goroutines provide threads consuming less than ten (10) kilobytes of memory per thread to enable concurrent processing of data streams from the cloud storage without thread-switching overhead.

### Claim 19 (depends on 14)

The method as described in claim 14, wherein the workload is processed on a swarm of serverless functions in the cloud compute infrastructure.

### Claim 20

A method associated with a cloud-based infrastructure comprising a plurality of compute nodes, cloud storage distinct from the plurality of compute nodes, and network connections between the plurality of compute nodes and the cloud storage, the method comprising:
receiving event data to be processed by a first processing stage in the plurality of compute nodes;
mapping operations on the event data using a plurality of executors of a set of executors;
In lieu of transferring data between processing stages in the plurality of compute nodes, writing intermediate shuffle data generated by the plurality of executors directly to cloud object storage organized by one or more partition keys; and
executing reduce operations by reading the intermediate shuffle data from the cloud object storage into a second processing stage distinct from the first processing stage;
wherein the cloud object storage serves as a persistent shuffle medium enabling job recovery without re-execution of completed mapping tasks.
