# US 10,503,718 B2 · Parallel transfers of electronic data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,718 B2
- Document ID: US10503718B2
- Status: Granted
- Kind: B2
- Application number: 15/203,451
- Filed: 2016-07-06
- Published: 2018-01-11
- Granted: 2019-12-10
- Earliest claimed: 2016-07-06
- Assignees: Micro Focus LLC
- Applicants: Hewlett Packard Enterprise Development LP
- Inventors: Jeffrey LeFevre; Maria Castellanos; Meichun Hsu; Rui Liu
- Web page: https://patentgrep.com/patent/US10503718B2

## CPC

- G06F 16/2282: {Tablespace storage structures; Management thereof}
- G06F 16/2365: {Ensuring data consistency and integrity}
- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Examples disclosed herein relate to parallel transfers of electronic data. Some examples disclosed herein may include executing, by a processing node of a computation engine, a computation task among a plurality of computation tasks generated by the computation engine for transferring an electronic data table to a target table. The computation task, when executed by the processing node, may cause the processing node to transfer a segment of the electronic data table to a staging table, update a task status table upon completing the transfer of the segment to the staging table, in response to determining that the plurality of computation tasks have completed, update a last committer table with a task identifier associated with the computation task, and in response to determining that the last committer table includes the task identifier associated with the computation task, transfer the staging table to the target table.

## Claims

### Claim 1

A method for parallel transfers of electronic data, the method comprising:
receiving an electronic data table and an identification of a target table in a database;
partitioning, by a processor, the received electronic data table into segments and table;
causing, by the processor, a plurality of processing nodes to perform parallel transfers of the segments of the electronic data table to a staging table, wherein the staging table is a table where all of the segments are stored prior to being transferred together to the identified target table in the database;
upon completing the transfer of one of the segments to the staging table, checking, by a processing node of the processing nodes that completed the transfer of the segment, statuses in a task status table to determine whether the transfers of all of the segments have completed;
in response to determining that the transfers of all of the segments have completed, updating a last committer table with a task identifier associated with the transfer of a last segment; and
in response to determining that the last committer table includes the task identifier associated with the computation task of transferring the last segment, transferring the staging table to the target table.

### Claim 2 (depends on 1)

The method of claim 1, comprising:
upon completing the transfer of one of the segments to the staging table, updating, by the processing node that performed the transfer of the segment, a status in the task status table to indicate that the transfer of the segment is complete.

### Claim 3 (depends on 1)

The method of claim 1, comprising:
receiving the identification of the target table via an application programming interface (API).

### Claim 4 (depends on 1)

The method of claim 1, comprising:
in response to receiving the electronic data table, generating, by the processor, the staging table, the task status table, and the last committer table.

### Claim 5 (depends on 1)

The method of claim 1, further comprising: causing, by the processor, one of the processing nodes to encode one of the segments in an Avro format before transferring the segment to the staging table.

### Claim 6 (depends on 1)

The method of claim 1, wherein transferring the staging table to the target table includes overwriting the target table with the staging table.

### Claim 7 (depends on 1)

The method of claim 1, wherein transferring the staging table to the target table includes appending the staging table to the target table.

### Claim 8 (depends on 1)

The method of claim 1, wherein transferring the staging table to the target table includes:
determining whether a number of errors in the staging table is less than a threshold number of errors; and
in response to determining that the number of errors in the staging table is less than the threshold number of errors, transferring the staging table to the target table.

### Claim 9

A system, comprising:
an electronic database to store a target table; and
a computation engine, including a plurality of processing nodes, to:
receive an electronic data table and an identification of the target table in the electronic database;
partition the received electronic data table into segments;
generate a plurality of computation tasks to transfer the segments to the identified target table;
cause the plurality of processing nodes of the computation engine to execute the computation tasks to transfer the segments to a staging table, wherein the staging table is a table where all of the segments are stored prior to being transferred together to the identified target table in the electronic database;
upon completing one of the computation tasks to transfer one of the segments, cause a processing node of the processing nodes to check statuses in a task status table to determine whether all of the computation tasks of transferring the segments have completed;
in response to determining that all of the computation tasks of transferring the segments have completed, update a last committer table with a task identifier associated with the computation task of transferring a last segment;
in response to determining that the last committer table includes the task identifier associated with the computation task of transferring the last segment, determine whether a number of errors in the staging table is less than a threshold number of errors; and
in response to determining that the number of errors in the staging table is less than the threshold number of errors, transfer the staging table to the identified target table.

### Claim 10 (depends on 9)

The system of claim 9, wherein the computation engine causes one of the processing nodes to determine whether the last committer table includes the task identifier associated with the computation task of transferring the last segment.

### Claim 11 (depends on 9)

The system of claim 9, wherein the computation engine causes the processing node to update the task status table to include the task identifier associated with the computation task of transferring the segment, a segment identifier, a number of errors that occurred while transferring the segment to the staging table, and a task completion status.

### Claim 12 (depends on 11)

The system of claim 11, wherein the computation engine causes the processing node to determine whether the number of errors in the staging table is less than the threshold number of errors by:
determining, based on the task status table, an aggregated number of errors that occurred during the transfers of all of the segments; and
comparing the aggregated number of errors to the threshold number of errors.

### Claim 13 (depends on 9)

The system of claim 9, wherein the computation engine causes one of the processing nodes to, in response to determining that the number of errors in the staging table is greater than or equal to the threshold number of errors:
abort the transfer of the staging table to the target table; and
update a final status table to indicate that the transfer of the staging table to the target table was unsuccessful.

### Claim 14 (depends on 9)

The system of claim 9, wherein to transfer the staging table to the target table, the computation engine is to overwrite the target table with the staging table or append the staging table to the target table.

### Claim 15

A non-transitory machine-readable storage medium storing instructions that, when executed by a processor, cause the processor to:
receive an electronic data table and an identification of a target table in a database;
partition the received electronic data table into segments;
generate, a plurality of computation tasks for transferring segments of the electronic data table to the target table;
cause a plurality of processing nodes to execute the plurality of computation tasks to transfer the segments to a staging table, wherein the staging table is a table where all of the segments are stored prior to being transferred together to the target table in the database;
upon completion of one of the computation tasks of transferring one of the segments to the staging table, cause a processing node of the processing nodes that completed the transfer of the segment to determine whether all of the computation tasks of transferring the segments have completed;
in response to determining that all of the computation tasks of transferring the segments have completed, update a last committer table with a task identifier associated with the computation task of transferring a last segment;
in response to determining that the last committer table includes the task identifier associated with the computation task of transferring the last segment, determine whether a number of errors in the staging table is less than a threshold number of errors; and
in response to determining that the number of errors in the staging table is less than the threshold number of errors, transfer the staging table to the target table and update a final status table to reflect the transfer of the staging table to the target table is complete.

### Claim 16 (depends on 15)

The non-transitory machine-readable storage medium of claim 15, wherein the instructions that cause the processor to transfer the staging table to the target table include instructions that cause the processor to identify the target table based on the identification of the target table.

### Claim 17 (depends on 15)

The non-transitory machine-readable storage medium of claim 15, wherein the instructions that cause the processor to transfer the staging table to the target table include instructions that cause the processor to overwrite the target table with the staging table.

### Claim 18 (depends on 15)

The non-transitory machine-readable storage medium of claim 15, wherein the instructions that cause the processor to transfer the staging table to the target table include instructions that cause the processor to append appending the staging table to the target table.

### Claim 19 (depends on 15)

The non-transitory machine-readable storage medium of claim 15, wherein the instructions that cause the processor to update the final status table include instructions that cause the processor to update the final status table to reflect a status of the transfer of the staging table to the target table.