# US 10,374,968 B1 · Data-driven automation mechanism for analytics workload distribution > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,374,968 B1 - Document ID: US10374968B1 - Status: Granted - Kind: B1 - Application number: 15/395,340 - Filed: 2016-12-30 - Granted: 2019-08-06 - Earliest claimed: 2016-12-30 - Assignees: EMC IP Holding Company LLC - Applicants: EMC IP Holding Company LLC - Inventors: Bryan Duerk; Natalie Corridan-Gregg; Patricia Gomes Soares Florissi - Web page: https://patentgrep.com/patent/US10374968B1 ## CPC - H04L 67/1004: Server selection for load balancing - G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals} - G06F 9/5044: {considering hardware capabilities} - H04L 41/0893: Assignment of logical groups to network elements - H04L 47/70: Admission control; Resource allocation - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - G06F 2209/5017: Task decomposition - H04L 41/0895: Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An apparatus in one embodiment comprises at least one processing device having a processor coupled to a memory. The processing device implements a first workload distribution node configured to communicate with multiple distributed data processing clusters over at least one network. The workload distribution node is further configured to receive a data processing request, to identify particular ones of the distributed data processing clusters that are suitable for handling at least a portion of the data processing request, and to assign the data tasks to one or more of the distributed data processing clusters. Results of performance of the data tasks from the one or more assigned distributed data processing clusters are received by the first workload distribution node and aggregated into a response that is returned to a source of the data processing request. The source of the data processing request in some embodiments is another workload distribution node. ## Claims ### Claim 1 A method comprising: receiving a data processing request in a first workload distribution node configured to communicate with a plurality of distributed data processing clusters over at least one network; identifying particular ones of the plurality of distributed data processing clusters that are suitable for handling at least a portion of the data processing request; separating the data processing request into a plurality of data tasks; providing each of the data tasks to one or more of the identified distributed data processing clusters; receiving for each of the data tasks an indication from one or more of the distributed data processing clusters of its ability to perform the data task; assigning the data tasks to one or more of the distributed data processing clusters responsive to the received indications; receiving results of performance of the data tasks from the one or more assigned distributed data processing clusters; and aggregating the results into a response that is returned to a source of the data processing request; wherein the source of the data processing request comprises another workload distribution node and further wherein the data processing request comprises a given data task of a higher-level data processing request separated into a plurality of data tasks by the other workload distribution node for handling by the first workload distribution node and one or more additional workload distribution nodes; wherein the first workload distribution node comprises an analytics workload distribution node and the given data task of the higher-level data processing request comprises a request to process at least a portion of an analytics workload using at least a subset of the plurality of distributed data processing clusters; wherein the data tasks are assigned and the corresponding results are aggregated in a manner that ensures satisfaction of one or more privacy policies of the one or more distributed data processing clusters; wherein the method is performed by at least one processing device comprising a processor coupled to a memory; and wherein said at least one processing device implements the first workload distribution node. ### Claim 2 (depends on 1) The method of claim 1 wherein the data processing request comprises a plurality of fields including two or more of a distribution type field, a distribution category field and a success type field, and wherein the distribution type field denotes a particular distribution type including one or more of universal distribution, geographic distribution and custom distribution, the distribution category field denotes a particular distribution category including a designated set of distributed data processing clusters of a particular distribution type, and the success type field denotes a particular success type including one or more of all distributed data processing clusters required, at least one cluster required, first cluster to bid, lowest cost cluster and best effort among multiple clusters. ### Claim 3 (depends on 1) The method of claim 1 wherein at least a subset of the plurality of distributed data processing clusters are associated with respective distinct data centers. ### Claim 4 (depends on 1) The method of claim 1 further comprising validating the results of performance of the data tasks from the assigned distributed data processing clusters prior to aggregating the results into the response that is returned to the source of the data processing request. ### Claim 5 (depends on 1) The method of claim 1 wherein the workload distribution node comprises at least a portion of a WWH node. ### Claim 6 (depends on 1) The method of claim 1 wherein each of the workload distribution nodes is associated with a different set of one or more distributed data processing clusters. ### Claim 7 (depends on 1) The method of claim 1 wherein the first workload distribution node provides at least a portion of the received data processing request to another workload distribution node as another data processing request. ### Claim 8 (depends on 1) The method of claim 1 wherein the first workload distribution node provides one or more of the data tasks to another workload distribution node for possible assignment to one or more additional distributed data processing clusters. ### Claim 9 (depends on 8) The method of claim 8 wherein the first workload distribution node is associated with an on-premises data center of an enterprise and the other workload distribution node is associated with one or more off-premises data centers of the enterprise. ### Claim 10 (depends on 1) The method of claim 1 wherein the data processing request includes an indication that only a subset of the distributed data processing clusters of a particular type can be assigned to perform its associated data tasks. ### Claim 11 (depends on 1) The method of claim 1 wherein the indications received by the first workload distribution node of the ability of the one or more distributed data processing clusters to perform the data tasks include respective estimates from one or more of the distributed data processing clusters of costs associated with performing the data tasks. ### Claim 12 A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes said at least one processing device to implement a first workload distribution node configured to communicate with a plurality of distributed data processing clusters over at least one network, the first workload distribution node being further configured: to receive a data processing request; to identify particular ones of the plurality of distributed data processing clusters that are suitable for handling at least a portion of the data processing request; to separate the data processing request into a plurality of data tasks; to provide each of the data tasks to one or more of the identified distributed data processing clusters; to receive for each of the data tasks an indication from one or more of the distributed data processing clusters of its ability to perform the data task; to assign the data tasks to one or more of the distributed data processing clusters responsive to the received indications; to receive results of performance of the data tasks from the one or more assigned distributed data processing clusters; and to aggregate the results into a response that is returned to a source of the data processing request; wherein the source of the data processing request comprises another workload distribution node and further wherein the data processing request comprises a given data task of a higher-level data processing request separated into a plurality of data tasks by the other workload distribution node for handling by the first workload distribution node and one or more additional workload distribution nodes; and wherein the first workload distribution node comprises an analytics workload distribution node and the given data task of the higher-level data processing request comprises a request to process at least a portion of an analytics workload using at least a subset of the plurality of distributed data processing clusters; wherein the data tasks are assigned and the corresponding results are aggregated in a manner that ensures satisfaction of one or more privacy policies of the one or more distributed data processing clusters. ### Claim 13 (depends on 12) The computer program product of claim 12 wherein the data tasks are assigned and the corresponding results are aggregated in a manner that further ensures satisfaction of one or more specified policies of the one or more distributed data processing clusters relating to at least one of security, governance, risk and compliance. ### Claim 14 (depends on 12) The computer program product of claim 12 wherein the data processing request comprises a plurality of fields including two or more of a distribution type field, a distribution category field and a success type field, and wherein the distribution type field denotes a particular distribution type including one or more of universal distribution, geographic distribution and custom distribution, the distribution category field denotes a particular distribution category including a designated set of distributed data processing clusters of a particular distribution type, and the success type field denotes a particular success type including one or more of all distributed data processing clusters required, at least one cluster required, first cluster to bid, lowest cost cluster and best effort among multiple clusters. ### Claim 15 (depends on 12) The computer program product of claim 12 wherein the first workload distribution node provides one or more of the data tasks to another workload distribution node for possible assignment to one or more additional distributed data processing clusters. ### Claim 16 (depends on 15) The computer program product of claim 15 wherein the first workload distribution node is associated with an on-premises data center of an enterprise and the other workload distribution node is associated with one or more off-premises data centers of the enterprise. ### Claim 17 An apparatus comprising: at least one processing device having a processor coupled to a memory; wherein said at least one processing device implements a first workload distribution node configured to communicate with a plurality of distributed data processing clusters over at least one network; the workload distribution node being further configured: to receive a data processing request; to identify particular ones of the plurality of distributed data processing clusters that are suitable for handling at least a portion of the data processing request; to separate the data processing request into a plurality of data tasks; to provide each of the data tasks to one or more of the identified distributed data processing clusters; to receive for each of the data tasks an indication from one or more of the distributed data processing clusters of its ability to perform the data task; to assign the data tasks to one or more of the distributed data processing clusters responsive to the received indications; to receive results of performance of the data tasks from the one or more assigned distributed data processing clusters; and to aggregate the results into a response that is returned to a source of the data processing request; wherein the source of the data processing request comprises another workload distribution node and further wherein the data processing request comprises a given data task of a higher-level data processing request separated into a plurality of data tasks by the other workload distribution node for handling by the first workload distribution node and one or more additional workload distribution nodes; wherein the first workload distribution node comprises an analytics workload distribution node and the given data task of the higher-level data processing request comprises a request to process at least a portion of an analytics workload using at least a subset of the plurality of distributed data processing clusters; and wherein the data tasks are assigned and the corresponding results are aggregated in a manner that ensures satisfaction of one or more privacy policies of the one or more distributed data processing clusters. ### Claim 18 (depends on 17) The apparatus of claim 17 wherein the data tasks are assigned and the corresponding results are aggregated in a manner that further ensures satisfaction of one or more specified policies of the one or more distributed data processing clusters relating to at least one of security, governance, risk and compliance. ### Claim 19 (depends on 17) The apparatus of claim 17 wherein the data processing request comprises a plurality of fields including two or more of a distribution type field, a distribution category field and a success type field, and wherein the distribution type field denotes a particular distribution type including one or more of universal distribution, geographic distribution and custom distribution, the distribution category field denotes a particular distribution category including a designated set of distributed data processing clusters of a particular distribution type, and the success type field denotes a particular success type including one or more of all distributed data processing clusters required, at least one cluster required, first cluster to bid, lowest cost cluster and best effort among multiple clusters. ### Claim 20 (depends on 17) The apparatus of claim 17 wherein the first workload distribution node provides one or more of the data tasks to another workload distribution node for possible assignment to one or more additional distributed data processing clusters.