# US 2019/0171632 A1 · Data ingestion into a polystore

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171632 A1
- Document ID: US20190171632A1
- Status: Published application
- Kind: A1
- Application number: 16/209,397
- Filed: 2018-12-04
- Published: 2019-06-06
- Earliest claimed: 2017-12-04
- Assignees: The Regents of the University of California
- Applicants: The Regents of the University of California
- Inventors: Amarnath Gupta; Subhasis Dasgupta; Margaret Roberts
- Web page: https://patentgrep.com/patent/US20190171632A1

## CPC

- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06F 16/24542: {Plan optimisation}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method may include determining, based on a data analysis plan that includes analytical operations performed on at least a portion of the dataset being ingested into and/or stored in the polystore, an optimization for the ingestion of the dataset into the polystore. The ingestion of the dataset includes storing, in a first database management system and/or a second database management system in the polystore, the portion of the dataset. The analysis plan may be modified, based on the optimization, by adding, removing, and/or reordering one or more analytical operations included in the data analysis plan. During the ingestion of the dataset, the optimization may be applied by routing, based on the optimization, the portion of the dataset for storage at the first database management system and/or the second database management system. Related systems and articles of manufacture are also provided.

## Claims

### Claim 1

A system, comprising:
at least one data processor; and
at least one memory storing instructions which, when executed by the at least one data processor, result in operations comprising:
determining, based at least on a data analysis plan, at least one optimization for an ingestion of a dataset into a polystore that includes a first database management system and a second database management system, the data analysis plan including one or more analytical operations performed on at least a portion of the dataset being ingested into the polystore and/or stored in the polystore, and the ingestion of the dataset into the polystore including storing, in the first database management system and/or the second database management system, at least the portion of the dataset;
modifying, based on the at least one optimization determined for the ingestion of the dataset into the polystore, the data analysis plan, the data analysis plan being modified by at least adding, removing, and/or reordering the one or more analytical operations included in the data analysis plan; and
applying, during the ingestion of dataset into the polystore, the at least one optimization, the at least one optimization by at least routing, based on the at least one optimization, at least the portion of the dataset for storage at the first database management system and/or the second database management system.

### Claim 2 (depends on 1)

The system of claim 1, further comprising:
processing at least the portion of the dataset to conform to a constraint associated with the first database management system and/or the second database management system.

### Claim 3 (depends on 2)

The system of claim 2, wherein the processing includes combining or partitioning chunks of data from the dataset in response to the chunks of data failing to meet the size constraint and/or a quantity constraint associated with the first database management system and/or the second database management system.

### Claim 4 (depends on 2)

The system of claim 2, wherein the processing includes buffering at least the portion of the dataset in response to the portion of the dataset being output from one or more data sources at a rate that exceeds an ingestion rate of the first database management system and/or the second database management system.

### Claim 5 (depends on 1)

The system of claim 1, wherein the at least one optimization include storing at least the portion of the dataset at the first database management system and not the second database management system.

### Claim 6 (depends on 1)

The system of claim 1, wherein the at least one optimization includes precomputing, based on at least the portion of the dataset, derived data, and wherein the at least one optimization further includes storing the derived data at the first database management system and/or the second database management system.

### Claim 7 (depends on 6)

The system of claim 6, wherein the derived data is generated by performing a first analytical operation in the data analysis plan, and wherein the data analysis plan is modified by at least removing, from the data analysis plan, the first analytical operation.

### Claim 8 (depends on 7)

The system of claim 7, wherein the data analysis plan is further modified by at least adding, to the data analysis plan, a second analytical operation to retrieve, from the first database management system and/or the second database management system, at least a portion of the derived data.

### Claim 9 (depends on 1)

The system of claim 1, wherein the at least one optimization is determined based on a data model associated with the first database management system and/or the second database management system.

### Claim 10

(canceled)

### Claim 11 (depends on 9)

The system of claim 9, wherein the at least on optimization includes storing at least the portion of the dataset in the first database management system and/or the second database management system based at least on the portion of the dataset having a same data model as the first database management system and/or the second database management system.

### Claim 12 (depends on 1)

The system of claim 1, wherein the at least one optimization is determined based at least on a rate at which the first database management system and/or the second database management system is able to ingest the dataset.

### Claim 13 (depends on 1)

The system of claim 1, wherein the at least one optimization is determined based at least on the first database management system and/or the second database management system supporting a first analytical operation but not a second analytical operation in the data analysis plan.

### Claim 14 (depends on 1)

The system of claim 1, wherein the at least one optimization is determined based at least on a resource that is available at the first database management system and/or the second database management system for performing the one or more analytical operations in the data analysis plan.

### Claim 15

(canceled)

### Claim 16

A computer-implemented method, comprising:
determining, based at least on a data analysis plan, at least one optimization for an ingestion of a dataset into a polystore that includes a first database management system and a second database management system, the data analysis plan including one or more analytical operations performed on at least a portion of the dataset being ingested into the polystore and/or stored in the polystore, and the ingestion of the dataset into the polystore including storing, in the first database management system and/or the second database management system, at least the portion of the dataset;
modifying, based on the at least one optimization determined for the ingestion of the dataset into the polystore, the data analysis plan, the data analysis plan being modified by at least adding, removing, and/or reordering the one or more analytical operations included in the data analysis plan; and
applying, during the ingestion of dataset into the polystore, the at least one optimization, the at least one optimization by at least routing, based on the at least one optimization, at least the portion of the dataset for storage at the first database management system and/or the second database management system.

### Claim 17

(canceled)

### Claim 18

(canceled)

### Claim 19

(canceled)

### Claim 20 (depends on 16)

The method of claim 16, wherein the at least one optimization include storing at least the portion of the dataset at the first database management system and not the second database management system.

### Claim 21 (depends on 16)

The method of claim 16, wherein the at least one optimization includes precomputing, based on at least the portion of the dataset, derived data, and wherein the at least one optimization further includes storing the derived data at the first database management system and/or the second database management system.

### Claim 22 (depends on 21)

The method of claim 21, wherein the derived data is generated by performing a first analytical operation in the data analysis plan, and wherein the data analysis plan is modified by at least removing, from the data analysis plan, the first analytical operation.

### Claim 23 (depends on 22)

The method of claim 22, wherein the data analysis plan is further modified by at least adding, to the data analysis plan, a second analytical operation to retrieve, from the first database management system and/or the second database management system, at least a portion of the derived data.

### Claim 24

(canceled)

### Claim 25

(canceled)

### Claim 26 (depends on 16)

The method of claim 16, wherein the at least on optimization includes storing at least the portion of the dataset in the first database management system and/or the second database management system based at least on the portion of the dataset having a same data model as the first database management system and/or the second database management system.

### Claim 27

(canceled)

### Claim 28

(canceled)

### Claim 29

(canceled)

### Claim 30

(canceled)

### Claim 31

A non-transitory computer-readable medium storing instructions, which when executed by at least one data processor, result in operations comprising:
determining, based at least on a data analysis plan, at least one optimization for an ingestion of a dataset into a polystore that includes a first database management system and a second database management system, the data analysis plan including one or more analytical operations performed on at least a portion of the dataset being ingested into the polystore and/or stored in the polystore, and the ingestion of the dataset into the polystore including storing, in the first database management system and/or the second database management system, at least the portion of the dataset;
modifying, based on the at least one optimization determined for the ingestion of the dataset into the polystore, the data analysis plan, the data analysis plan being modified by at least adding, removing, and/or reordering the one or more analytical operations included in the data analysis plan; and
applying, during the ingestion of dataset into the polystore, the at least one optimization, the at least one optimization by at least routing, based on the at least one optimization, at least the portion of the dataset for storage at the first database management system and/or the second database management system.

### Claim 32

(canceled)

### Claim 33

(canceled)
