# US 2018/0024858 A1 · Framework for authoring data loaders and data savers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0024858 A1
- Document ID: US20180024858A1
- Status: Published application
- Kind: A1
- Application number: 15/617,526
- Filed: 2017-06-08
- Published: 2018-01-25
- Earliest claimed: 2014-05-23
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Tong Wen; Parry Husbands; Samuel Weiss
- Web page: https://patentgrep.com/patent/US20180024858A1

## CPC

- G06F 9/46: Multiprogramming arrangements
- G06F 8/00: Arrangements for software engineering (testing or debugging G06F11/36; administrative, planning or organisation aspects of software project management G06Q10/06)
- G06F 9/4843: {by program, e.g. task dispatcher, supervisor, operating system}
- G06F 9/52: Program synchronisation; Mutual exclusion, e.g. by means of semaphores
- G06F 9/54: Interprogram communication
- G06F 8/36: Software reuse
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Implementing static loaders and savers for the transfer of local and distributed data containers to and from storage systems can be difficult because there are so many different configurations of output formats, data containers and storage systems. Described herein is an extensible componentized data transfer framework for performant and scalable authoring of data loaders and data savers. Abstracted local and distributed workflows drive selection of plug-ins that can be composed by the framework into particular local or distributed scenario loaders and savers. Reusability and code sparsity are maximized.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A computing device, comprising:
at least one processor; and
a memory connected to the at least one processor, wherein the at least one memory and the at least one processor are respectively configured to store and execute instructions for causing the computing device to perform operations, the operations comprising:
composing, by an extensible data transfer framework, a plurality of data transfer components into a data transfer executable, including:
dynamically creating a plurality of components; and
interfacing the plurality of components into a data transfer executable in which a control flow of the data transfer executable is dictated by the extensible data transfer framework instead of by the plurality of components.

### Claim 22 (depends on 21)

The computing device of claim 21, wherein the operations further comprise:
executing, with the extensible data transfer framework, the data transfer executable.

### Claim 23 (depends on 21)

The computing device of claim 21, wherein executing the data transfer executable includes:
coordinating, with the extensible data transfer framework, communications between a plurality of processes for a distributed data transfer.

### Claim 24 (depends on 21)

The computing device of claim 21, wherein executing the data transfer executable includes:
partitioning a distributed data container for the data transfer scenario into a sequence of blocks;
serializing each block of the sequence of blocks into a sequence of formatted blocks; and
writing at least a portion of the sequence of formatted blocks to a distributed storage system.

### Claim 25 (depends on 21)

The computing device of claim 21, wherein executing the data transfer executable includes:
in response to a determination that the serialization of each block of the sequence of blocks into the sequence of formatted blocks was unsuccessful, rolling back at least one partial write.

### Claim 26 (depends on 21)

The computing device of claim 21, wherein executing the data transfer executable includes:
employing a modifiable record in tracking workflow status for a plurality of processes for a data transfer scenario.

### Claim 27 (depends on 21)

The computing device of claim 21, wherein executing the data transfer executable includes:
performing at least one of a local or distributed data transfer operation with the data transfer executable.

### Claim 28 (depends on 21)

The computing device of claim 21, wherein the composing of the plurality of data transfer components into the data transfer executable further includes:
receiving parameters describing a data transfer scenario, the parameters indicating at least one of a location from which input data is to be loaded, a location to which output data to be saved, an input format for the input data, an output format into which the output data is to be converted, a type of a container into which the input data is to be loaded, or a direction of the data transfer scenario.

### Claim 29 (depends on 28)

The computing device of claim 28, wherein the composing of the plurality of data transfer components into the data transfer executable further includes:
providing, based on the received parameters, a modifiable record that indicates a workflow state for each of a plurality of processes.

### Claim 30 (depends on 21)

The computing device of claim 21, wherein the composing of the plurality of data transfer components into the data transfer executable further includes:
receiving at least one component via a strongly typed component application programming interfaces.

### Claim 31

A method, comprising:
composing, by a data transfer framework, a plurality of data transfer components into a data transfer executable, including:
receiving parameters identifying aspects of a particular data transfer scenario;
dynamically creating a plurality of components for the particular data transfer scenario; and
composing the plurality of components into a data transfer executable in which a control flow of the data transfer executable is dictated by the data transfer framework instead of by the plurality of components.

### Claim 32 (depends on 31)

The method of claim 31, wherein the received parameters indicate at least one of a location from which input data is to be loaded, a location to which output data to be saved, an input format for the input data, an output format into which the output data is to be converted, a type of a container into which the input data is to be loaded, or a direction of the data transfer scenario.

### Claim 33 (depends on 31)

The method of claim 31, further comprising:
executing, with the data transfer framework, the data transfer executable.

### Claim 34 (depends on 33)

The method of claim 33, wherein executing the data transfer executable includes:
coordinating, with the data transfer framework, communications between a plurality of processes for the particular data transfer scenario.

### Claim 35 (depends on 33)

The method of claim 33, wherein executing the data transfer executable includes:
in response to a determination that the serialization of each block of the sequence of blocks into the sequence of formatted blocks was unsuccessful, rolling back at least one partial write.

### Claim 36 (depends on 31)

The method of claim 31, wherein the particular data transfer scenario is for local data transfer.

### Claim 37 (depends on 31)

The method of claim 31, wherein the particular data transfer scenario is for distributed data transfer.

### Claim 38

A computing device, comprising:
at least one processor and
a memory connected to the at least one processor, wherein the at least one memory and the at least one processor are respectively configured to store and execute instructions for causing the computing device to perform operations, the operations comprising:
receiving parameters identifying aspects of a particular data transfer scenario;
creating a plurality of components for the particular data transfer scenario; and
dynamically composing the plurality of components into a data transfer executable in which a control flow of the data transfer executable is dictated by the data transfer framework instead of by the plurality of components.

### Claim 39 (depends on 38)

The computing device of claim 38, wherein the received parameters indicate a location from which input data is to be loaded, a location to which output data to be saved, an input format for the input data, an output format into which the output data is to be converted, a type of a container into which the input data is to be loaded, and a direction of the data transfer scenario.

### Claim 40 (depends on 38)

The computing device of claim 38, wherein the operations further comprise:
executing, with the extensible data transfer framework, the data transfer executable.
