Data flow graph-driven analytics platform using data processing units having hardware accelerators
AssigneeFungible, Inc.
InventorsRajan Goyal, Satyanarayana Lakshmipathi Billa
Life of this patent
Estimating the term…
Abstract
A data flow graph-driven analytics platform is described in which highly-programmable data stream processing devices, referred to generally herein as data processing units (DPUs), operate to provide a scalable, fast and efficient analytics processing architecture. In general, the DPUs are specialized data-centric processors architected for efficiently applying data manipulation operations (e.g., regular expression operations to match patterns, filtering operations, data retrieval, compression/decompression and encryption/decryption) to streams of data units, such as packet flows having network packets, a set of storage packets being retrieved from or written to storage or other data units.
Claims31
A device comprising:
- an analytics interface to receive a request specifying at least one analytical operation to be performed on data;
- a query compiler to generate, based on the analytical operation, a data flow graph for configuring at least one data processing unit (DPU) to execute the analytical operation, wherein the DPU comprises an integrated circuit having hardware-based accelerators configured for processing the data as one or more streams of data units, and wherein the data flow graph comprises a data structure having one or more graph nodes connected by one or more directional arcs, each arc representing a stream of data units to be processed or produced by the DPU, and each of the graph nodes represents a set of data stream processing operations to be performed by the DPU to process or produce the data streams;
- and
- a query execution controller configured to communicate the data flow graph to the DPU to configure the DPU to perform the analytical operation on the data.
A method comprising:
- receiving, with a control node, a request specifying at least one analytical operation to be performed on data;
- selecting at least one data processing unit (DPU) having hardware-based accelerators configured for processing the data as one or more streams of data units;
- generating, based on the analytical operation, a data flow graph for configuring the DPU to execute the analytical operation, wherein generating the data flow graph comprises generating the data flow graph as a data structure having one or more graph nodes connected by directional arcs, each arc representing a stream of data units to be processed or produced by the DPU, and each of the graph nodes represents a set of data stream processing operations to be performed by the DPU to process or produce the data streams;
- and
- communicating the data flow graph to the DPU to configure the DPU to perform the analytical operation on the data.
A device comprising:
- a data processing unit (DPU) comprising an integrated circuit having programmable processor cores and hardware-based accelerators configured for processing streams of data units;
- and
- a controller to receive a data flow graph and configure the DPU to apply an analytical operation, wherein the data flow graph has one or more graph nodes connected by one or more directional arcs, each arc representing a stream of data units to be processed or produced by the DPU, and each of the graph nodes represents a set of data stream processing operations to be performed by the DPU to process the data streams to apply the analytical operation.
A system comprising:
- a data source storing data;
- a plurality of data processing units (DPUs), each DPU having hardware-based accelerators configured for processing streams of data units;
- a control node comprising:
- an analytics interface to receive a request specifying at least one analytical operation to be performed on the data;
- a query compiler to generate a data flow graph for configuring the DPUs to execute the analytical operation, wherein the data flow graph comprises a data structure having one or more graph nodes connected by one or more directional arcs, each arc representing a stream of data units to be processed or produced by one of the DPUs, and each of the graph nodes represents a set of data stream processing operations to be performed by one of the DPUs to process or produce the data streams;
- and
- a query execution controller configured to communicate the data flow graph to the DPUs to configure the DPUs to perform the analytical operation on the data.
Dependencies are inferred from the claim text.