# US 2019/0317949 A1 · Analytics platform for scalable distributed computations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0317949 A1
- Document ID: US20190317949A1
- Status: Published application
- Kind: A1
- Application number: 16/272,517
- Filed: 2019-02-11
- Published: 2019-10-17
- Earliest claimed: 2015-04-06
- Assignees: EMC IP Holding Company LLC
- Applicants: EMC IP Holding Company LLC
- Inventors: Patricia Gomes Soares Florissi; Ofri Masad
- Web page: https://patentgrep.com/patent/US20190317949A1

## CPC

- G06F 16/2471: {Distributed queries}
- G06F 16/285: {Clustering or classification}
- G06F 16/2477: {Temporal data queries}
- G06F 16/27: Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F 16/951: Indexing; Web crawling techniques
- G06F 16/2246: {Trees, e.g. B+trees}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An analytics platform is configured to receive a request to execute a distributed analytics workflow specification comprising multiple distributed computations to be performed across a plurality of data processing clusters associated with respective data zones. For each of a plurality of analytic services to be executed in accordance with the distributed analytics workflow specification, the platform determines if execution of the analytic service is authorized for the request, and for each of a plurality of datasets to be operated on by one or more of the analytic services in accordance with the distributed analytics workflow specification, the platform determines if execution of the one or more analytic services on the dataset is authorized by its corresponding data zone for the request. Responsive to each of the analytic services and each of the datasets being authorized for the request, the platform initiates execution of the distributed computations of the distributed analytics workflow specification across the data processing clusters.

## Claims

### Claim 1

A method comprising:
receiving a request to execute a distributed analytics workflow specification comprising multiple distributed computations to be performed across a plurality of data processing clusters associated with respective data zones;
for each of a plurality of analytic services to be executed in accordance with the distributed analytics workflow specification, determining if execution of the analytic service is authorized for the request;
for each of a plurality of datasets to be operated on by one or more of the analytic services in accordance with the distributed analytics workflow specification, determining if execution of the one or more analytic services on the dataset is authorized by its corresponding data zone for the request; and
responsive to each of the analytic services and each of the datasets being authorized for the request, initiating execution of the distributed computations of the distributed analytics workflow specification across the data processing clusters;
wherein the method is performed by an analytics platform comprising at least one processing device comprising a processor coupled to a memory.

### Claim 2 (depends on 1)

The method of claim 1 wherein each of at least a subset of the data processing clusters comprises a plurality of processing nodes interconnected by a network.

### Claim 3 (depends on 1)

The method of claim 1 wherein the analytic services are identified in the distributed analytics workflow specification and comprise analytic services that were previously registered by respective owning entities with one or more analytic service libraries of the analytics platform.

### Claim 4 (depends on 1)

The method of claim 1 wherein the datasets are identified in the distributed analytics workflow specification and comprise datasets that were previously registered by respective owning entities with one or more dataset libraries of the analytics platform.

### Claim 5 (depends on 1)

The method of claim 1 wherein the data zones are identified in the distributed analytics workflow specification and comprise data zones that were previously registered by respective owning entities with one or more data zone libraries of the analytics platform.

### Claim 6 (depends on 1)

The method of claim 1 wherein a given one of the data zones comprises a data lake that includes one or more of the datasets.

### Claim 7 (depends on 1)

The method of claim 1 wherein the analytics platform comprises an insight exchange platform accessible to client devices via a web-based interface.

### Claim 8 (depends on 7)

The method of claim 7 wherein receiving the request comprises receiving at least a portion of the request in the insight exchange platform from a particular one of the client devices via the web-based interface.

### Claim 9 (depends on 7)

The method of claim 7 wherein a given owning entity registering at least one of an analytic service, a dataset and a data zone with the insight exchange platform is provided with one or more specified credits responsive to successful completion of the distributed computations in accordance with the distributed analytics workflow specification.

### Claim 10 (depends on 1)

The method of claim 1 wherein the analytic services are executed on the datasets without the datasets leaving their respective data zones.

### Claim 11 (depends on 1)

The method of claim 1 wherein a first one of data processing clusters is implemented in a first cloud of a first type provided by a first cloud service provider, and at least a second one of the data processing clusters is implemented in a second cloud of a second type different than the first type, provided by a second cloud service provider different than the first cloud service provider.

### Claim 12 (depends on 11)

The method of claim 11 wherein the first one of data processing clusters implemented in the first cloud utilizes a first local data structure configured to support a first computational framework and wherein at least the second one of the data processing clusters implemented in the second cloud utilizes a second local data structure different than the first local data structure and configured to support a second computational framework different than the first computational framework.

### Claim 13 (depends on 12)

The method of claim 12 wherein the first computational framework comprises a batch framework and the second computational framework comprises a streaming framework.

### Claim 14 (depends on 1)

The method of claim 1 wherein performing the distributed computations across the data processing clusters associated with respective data zones comprises utilizing local processing results of at least a subset of the distributed computations from respective ones of the data processing clusters to generate global processing results, each of at least a subset of the data processing clusters being configured to process data from a dataset of the corresponding data zone using one or more local computations of that data processing cluster to generate at least a portion of the local processing results, at least one of the data processing clusters being configured to apply one or more global computations to one or more of the local processing results to generate at least a portion of the global processing results.

### Claim 15

A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes said at least one processing device:
to receive a request to execute a distributed analytics workflow specification comprising multiple distributed computations to be performed across a plurality of data processing clusters associated with respective data zones;
for each of a plurality of analytic services to be executed in accordance with the distributed analytics workflow specification, to determine if execution of the analytic service is authorized for the request;
for each of a plurality of datasets to be operated on by one or more of the analytic services in accordance with the distributed analytics workflow specification, to determine if execution of the one or more analytic services on the dataset is authorized by its corresponding data zone for the request; and
responsive to each of the analytic services and each of the datasets being authorized for the request, to initiate execution of the distributed computations of the distributed analytics workflow specification across the data processing clusters.

### Claim 16 (depends on 15)

The computer program product of claim 15 wherein said at least one processing device implements an analytics platform and wherein the analytic services are identified in the distributed analytics workflow specification and comprise analytic services that were previously registered by respective owning entities with one or more analytic service libraries of the analytics platform.

### Claim 17 (depends on 15)

The computer program product of claim 15 wherein said at least one processing device implements an analytics platform and wherein the datasets are identified in the distributed analytics workflow specification and comprise datasets that were previously registered by respective owning entities with one or more dataset libraries of the analytics platform.

### Claim 18

An apparatus comprising:
at least one processing device having a processor coupled to a memory;
wherein said at least one processing device is configured:
to receive a request to execute a distributed analytics workflow specification comprising multiple distributed computations to be performed across a plurality of data processing clusters associated with respective data zones;
for each of a plurality of analytic services to be executed in accordance with the distributed analytics workflow specification, to determine if execution of the analytic service is authorized for the request;
for each of a plurality of datasets to be operated on by one or more of the analytic services in accordance with the distributed analytics workflow specification, to determine if execution of the one or more analytic services on the dataset is authorized by its corresponding data zone for the request; and
responsive to each of the analytic services and each of the datasets being authorized for the request, to initiate execution of the distributed computations of the distributed analytics workflow specification across the data processing clusters.

### Claim 19 (depends on 18)

The apparatus of claim 18 wherein said at least one processing device implements an analytics platform and wherein the analytic services are identified in the distributed analytics workflow specification and comprise analytic services that were previously registered by respective owning entities with one or more analytic service libraries of the analytics platform.

### Claim 20 (depends on 18)

The apparatus of claim 18 wherein said at least one processing device implements an analytics platform and wherein the datasets are identified in the distributed analytics workflow specification and comprise datasets that were previously registered by respective owning entities with one or more dataset libraries of the analytics platform.
