# US 2025/0094235 A1 · A dynamic data processing pipeline

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094235 A1
- Document ID: US20250094235A1
- Status: Published application
- Kind: A1
- Application number: 18/959,679
- Filed: 2024-11-26
- Published: 2025-03-20
- Earliest claimed: 2023-07-20
- Assignees: Zluri Technologies Private Limited
- Applicants: Zluri Technologies Private Limited
- Inventors: Chaithanya Yembari; Sethu Meenakshisundaram; Ritish Reddy; Chinmay Panda; Vanketesh Kumar; Deepanshu Agarwal; Tejaswini Jupudi
- Web page: https://patentgrep.com/patent/US20250094235A1

## CPC

- G06F 9/5044: {considering hardware capabilities}
- G06F 9/38: Concurrent instruction execution, e.g. pipeline or look ahead
- G06F 9/3838: {Dependency mechanisms, e.g. register scoreboarding}
- G06F 9/5038: {considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups)}
- G06F 9/544: {Buffers; Shared memory; Pipes}
- G06F 8/433: {Dependency analysis; Data or control flow analysis}
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method for dynamically generating a data processing pipeline is disclosed. A processor receives data including one or more data types from a data source. A set of sub-pipelines are created based on the one or more data types, wherein each sub-pipeline of the set of sub-pipelines includes one or more processing layers. Further, the one or more data types and volume of data assigned to each processing layer of the one or more processing layers is determined. Subsequently, the resource allocation to the one or more processing layers is done dynamically based on the one or more data types, the data source, and the volume of the data.

## Claims

### Claim 1

A method for dynamically generating a data processing pipeline, the method comprising:
receiving, by a processor, data including one or more data types from a data source;
creating, by the processor, a set of sub-pipelines based on the one or more data types, wherein each sub-pipeline of the set of sub-pipelines includes one or more processing layers;
determining, by the processor, the one or more data types and volume of data assigned to each processing layer of the one or more processing layers; and
dynamically allocating resources, by the processor, to the one or more processing layers based on the one or more data types, the data source, and the volume of the data, wherein the dynamically allocating resources comprises determining a resource type, based on at least one of, the one or more data types, the data source, and the volume of data.

### Claim 2 (depends on 1)

The method as claimed in claim 1, wherein an order of execution of data processing at the one or more processing layers includes at least one of a serial processing, a parallel processing, or a combination of both, wherein the order of execution of the data processing pipeline varies at different level of the one or more processing layers based on data dependency.

### Claim 3 (depends on 1)

The method as claimed in claim 1, wherein the dynamically allocating resources comprises allocation of different resource type at different processing layers within a sub-pipeline of the set of sub-pipelines.

### Claim 4 (depends on 1)

The method as claimed in claim 1, further comprising assigning the resource type based on one or more of a processor speed, number of Central Processing Unit (CPU) cores, Random Access Memory (RAM) size, storage type, and size, cache size, and network bandwidth.

### Claim 5 (depends on 1)

The method as claimed in claim 1, wherein the data is received from at least one of a Single Sign On (SSO) platform, application integration platform, and browser agent.

### Claim 6 (depends on 1)

The method as claimed in claim 1, further comprising prior to receiving the data, accessing a configuration file for generating the data processing pipeline, wherein data processing operation of each sub-pipeline is defined in the configuration file including information related to the data source, the one or more processing layers, and a dependency between one or more processing layers and error handling.

### Claim 7 (depends on 1)

The method as claimed in claim 1, wherein the one or more data types include at least one of an application state data and a time interval-based application data.

### Claim 8 (depends on 1)

The method as claimed in claim 1, comprises generating a validation script to validate one or more aspects of the data processing pipeline, wherein the one or more aspects include data input, data output, data processing steps, and performance of the data processing pipeline.

### Claim 9 (depends on 1)

The method as claimed in claim 1, wherein the data processing pipeline includes a self-recovery mechanism in case a failure is detected, wherein a state of the data processing is routinely captured at predefined intervals, providing recovery and continuation of the data processing operation in case of the failure.

### Claim 10 (depends on 1)

The method as claimed in claim 1, wherein allocation of resources occurs on the fly based on the one or more data types and volume of the data.

### Claim 11 (depends on 1)

The method as claimed in claim 1, wherein the data is processed as a single indivisible task at the one or more processing layers.

### Claim 12 (depends on 1)

The method as claimed in claim 1, wherein an auto scale function is used to manage the resources at different processing layers.

### Claim 13 (depends on 1)

The method as claimed in claim 1, wherein the data processing pipeline includes an auto resume mechanism for error handling.

### Claim 14

A system for dynamically generating a data processing pipeline, the system comprising:
a memory; and
a processor coupled to the memory, wherein the processor is configured to execute program instructions stored in the memory for:
receiving data including one or more data types from a data source;
creating a set of sub-pipelines based on the one or more data types, wherein each sub-pipeline of the set of sub-pipelines includes one or more processing layers;
determining the one or more data types and volume of data assigned for processing to each processing layer of the one or more processing layers; and
dynamically allocating resources to the one or more processing layers based on the one or more data types, the data source, and the volume of the data, wherein the dynamically allocating resources comprises determining a resource type, based on at least one of, the one or more data types, the data source, and the volume of data.

### Claim 15 (depends on 14)

The system as claimed in claim 14, further comprises assigning different resource types to a processing layer of the one or more processing layers.

### Claim 16

A computer programmable product having embodied thereon a computer program for dynamically generating a data processing pipeline, the computer programmable product storing instructions for:
receiving data including one or more data types from a data source;
creating a set of sub-pipelines based on the one or more data types, wherein each sub-pipeline of the set of sub-pipelines includes one or more processing layers;
determining the one or more data types and volume of data assigned for processing to each processing layer of the one or more processing layers; and
dynamically allocating resources to the one or more processing layers based on the one or more data types, the data source, and the volume of the data, wherein the dynamically allocating resources comprises determining a resource type, based on at least one of, the one or more data types, the data source, and the volume of data.
