# US 10,122,806 B1 · Distributed analytics platform > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,122,806 B1 - Document ID: US10122806B1 - Status: Granted - Kind: B1 - Application number: 14/986,276 - Filed: 2015-12-31 - Granted: 2018-11-06 - Earliest claimed: 2015-04-06 - Assignees: EMC IP Holding Company LLC - Applicants: EMC Corporation - Inventors: Patricia Gomes Soares Florissi; Orna Berry - Web page: https://patentgrep.com/patent/US10122806B1 ## CPC - H04L 67/535: {Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34)} - G06F 16/00: Information retrieval; Database structures therefor; File system structures therefor - G06F 16/25: Integrating or interfacing systems involving database management systems - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - H04W 4/70: Services for machine-to-machine communication [M2M] or machine type communication [MTC] - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract An apparatus comprises an analytics platform having a plurality of distributed processing nodes. Data from a data source is received at a first one of the processing nodes. One or more analytics operations are performed on the data at the first processing node, and the data is updated at the first processing node based on results of the one or more analytics operations. The updated data is transmitted to another one of the processing nodes along a data path between the data source and a data destination. The performing, updating and transmitting are repeated for the other processing node and for one or more additional distributed processing nodes of the analytics platform along the data path. The analytics platform thereby performs distributed analytics processing on the data over multiple distributed processing nodes as the data moves through the data path from the data source to the data destination. ## Claims ### Claim 1 A method comprising: receiving data from a first data source at a first one of a plurality of distributed processing nodes of an analytics platform; performing one or more analytics operations on the data at the first processing node; updating the data at the first processing node based on results of the one or more analytics operations; transmitting the updated data to another one of the processing nodes along a first data path between the first data source and a first data destination; and repeating the performing, updating and transmitting for the other processing node and for one or more additional distributed processing nodes of the analytics platform along the first data path; the analytics platform thereby performing distributed analytics processing on the data over multiple ones of the distributed processing nodes as the data moves through the first data path from the first data source to the first data destination; wherein the method is implemented by at least one processing device comprising a processor coupled to a memory; wherein each of at least a subset of the distributed processing nodes is a part of multiple distinct data paths and performs distinct analytics operations for different ones of the data paths such that a role of a given one of the processing nodes in the analytics platform varies based at least in part on the particular data path over which data is received for processing; wherein the first processing node is part of the subset of the distributed processing nodes and is a part of at least two distinct data paths including the first data path between the first data source and the first data destination and at least a second data path between a second data source and a second data destination; wherein the first processing node performs a first type of analytics operation for a first role when receiving data on the first data path; wherein the first processing node performs a second type of analytics operation for a second role when receiving data on the second data path; wherein the first and second roles comprise respective distinct sets of one or more analytics-related processing tasks; wherein the first role and the second role of the first processing node are defined by a distributed service of a software-defined function of the analytics platform; and wherein updating the data based on results of the one or more analytics operations comprises two or more of: adding metadata to the data; modifying existing metadata of the data; and transforming the data into a different format. ### Claim 2 (depends on 1) The method of claim 1 wherein the analytics platform is configured to determine the first data path as at least a portion of an inverse spanning tree. ### Claim 3 (depends on 1) The method of claim 1 wherein the distributed processing nodes of the analytics platform comprise a plurality of distributed processing node clusters associated with respective data zones, each of the clusters being configured to perform analytics operations utilizing local data resources locally accessible within its corresponding data zone. ### Claim 4 (depends on 3) The method of claim 3 wherein the plurality of distributed processing node clusters comprise respective Yet Another Resource Negotiator (YARN) clusters. ### Claim 5 (depends on 1) The method of claim 1 wherein the plurality of distributed processing nodes are arranged in multiple distinct abstraction levels with each such level implementing a different type of analytics functionality. ### Claim 6 (depends on 1) The method of claim 1 wherein the data comprises IoT data from a data source comprising one or more IoT devices. ### Claim 7 (depends on 1) The method of claim 1 wherein the analytics platform is configured to optimize configuration of the first data path for one or more specified parameters relating to transmission of the data from the data source to the data destination including one or more of speed, bandwidth, cost and compliance. ### Claim 8 (depends on 1) The method of claim 1 wherein the one or more analytics operations performed by a given one of the processing nodes include at least one of: a user-defined function; and a function selected from a catalog of available functions. ### Claim 9 (depends on 1) The method of claim 1 wherein updating the data based on results of the one or more analytics operations comprises at least one of: combining the data with data from one or more other data paths; replacing at least a portion of the data with data from one or more other data paths; and modifying at least a portion of the data based at least in part on the data from one or more other data paths. ### Claim 10 (depends on 1) The method of claim 1 further comprising providing intermediate analytics results based at least in part on intermediate analytics operations performed at respective intermediate ones of the processing nodes that are part of the first data path between the data source and the data destination. ### Claim 11 (depends on 1) The method of claim 1 wherein a given analytics operation performed at a particular one of the processing nodes utilizes local information available to that processing node within a corresponding local data zone. ### Claim 12 (depends on 1) The method of claim 1 wherein the method is performed at least in part in a worldwide data node. ### Claim 13 A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes said at least one processing device: to receive data from a first data source at a first one of a plurality of distributed processing nodes of an analytics platform; to perform one or more analytics operations on the data at the first processing node; to update the data at the first processing node based on results of the one or more analytics operations; to transmit the updated data to another one of the processing nodes along a first data path between the first data source and a first data destination; and to repeat the performing, updating and transmitting for the other processing node and for one or more additional distributed processing nodes of the analytics platform along the first data path; the analytics platform thereby performing distributed analytics processing on the data over multiple ones of the distributed processing nodes as the data moves through the first data path from the first data source to the first data destination; wherein each of at least a subset of the distributed processing nodes is a part of multiple distinct data paths and performs distinct analytics operations for different ones of the data paths such that a role of a given one of the processing nodes in the analytics platform varies based at least in part on the particular data path over which data is received for processing; wherein the first processing node is part of the subset of the distributed processing nodes and is a part of at least two distinct data paths including the first data path between the first data source and the first data destination and at least a second data path between a second data source and a second data destination; wherein the first processing node performs a first type of analytics operation for a first role when receiving data on the first data path; wherein the first processing node performs a second type of analytics operation for a second role when receiving data on the second data path; wherein the first and second roles comprise respective distinct sets of one or more analytics-related processing tasks; wherein the first role and the second role of the first processing node are defined by a distributed service of a software-defined function of the analytics platform; and wherein updating the data based on results of the one or more analytics operations comprises two or more of: adding metadata to the data; modifying existing metadata of the data; and transforming the data into a different format. ### Claim 14 (depends on 13) The computer program product of claim 13 wherein the plurality of distributed processing nodes are arranged in multiple distinct abstraction levels with each such level implementing a different type of analytics functionality. ### Claim 15 (depends on 13) The computer program product of claim 13 wherein a given analytics operation performed at a particular one of the processing nodes utilizes local information available to that processing node within a corresponding local data zone. ### Claim 16 An apparatus comprising: a first processing node of an analytics platform; the first processing node being one of a plurality of distributed processing nodes of the analytics platform and being configured for communication with other ones of the distributed processing nodes over one or more networks; the first processing node being further configured: to receive data from a first data source; to perform one or more analytics operations on the data; to update the data based on results of the one or more analytics operations; and to transmit the updated data to another one of the processing nodes along a first data path between the first data source and a first data destination; wherein the performing, updating and transmitting are repeated for the other processing node and for one or more additional distributed processing nodes of the analytics platform along the first data path; the analytics platform thereby performing distributed analytics processing on the data over multiple ones of the distributed processing nodes as the data moves through the first data path from the first data source to the first data destination; wherein each of at least a subset of the distributed processing nodes is a part of multiple distinct data paths and performs distinct analytics operations for different ones of the data paths such that a role of a given one of the processing nodes in the analytics platform varies based at least in part on the particular data path over which data is received for processing; wherein the first processing node is part of the subset of the distributed processing nodes and is a part of at least two distinct data paths including the first data path between the first data source and the first data destination and at least a second data path between a second data source and a second data destination; wherein the first processing node performs a first type of analytics operation for a first role when receiving data on the first data path; wherein the first processing node performs a second type of analytics operation for a second role when receiving data on the second data path; wherein the first and second roles comprise respective distinct sets of one or more analytics-related processing tasks; wherein the first role and the second role of the first processing node are defined by a distributed service of a software-defined function of the analytics platform; and wherein updating the data based on results of the one or more analytics operations comprises two or more of: adding metadata to the data; modifying existing metadata of the data; and transforming the data into a different format. ### Claim 17 (depends on 16) An information processing system comprising the apparatus of claim 16. ### Claim 18 (depends on 16) The apparatus of claim 16 wherein a given analytics operation performed at a particular one of the processing nodes utilizes local information available to that processing node within a corresponding local data zone. ### Claim 19 (depends on 16) The apparatus of claim 16 wherein the plurality of distributed processing nodes are arranged in multiple distinct abstraction levels with each such level implementing a different type of analytics functionality.