Back to results
US 12,411,826 B2Granted
Location-constrained storage and analysis of large data sets
AssigneeAmazon Technologies, Inc.
InventorsMatt A Cember, Emma Lippman, Vinaykumar S. Patil
Life of this patent
Estimating the term…
Abstract
A constraint on a location at which a portion of a data set can be stored is determined based on input received via a programmatic interface. The portion of the data set is stored at a location selected in accordance with the constraint. An analysis operation, whose input includes the portion of the data set, is performed at a set of computing resources selected from a plurality of resources based at least in part on their location.
Claims20
A system, comprising:
- one or more computing devices;
- wherein the one or more computing devices include instructions that upon execution on or across the one or more computing devices:
- determine, based at least in part on input received via one or more programmatic interfaces of a cloud computing environment whose resources are distributed among a plurality of data centers in respective geographical regions, a first constraint on a location at which a first portion of a data set can be stored, wherein the first constraint is compliant with a first legal requirement applicable to the first portion of the data set, and wherein the data set is to be used as input for one or more analysis operations performed using computing resources of the cloud computing environment;
- store the first portion of the data set at a first set of persistent storage resources selected in accordance with the first constraint, wherein the first set of persistent storage resources is located at a first data center of the plurality of data centers;
- store location metadata pertaining to a plurality of portions of the data set at the cloud computing environment, wherein the location metadata indicates that
- a second portion of the data set is stored at a second set of persistent storage devices at a second data center of the plurality of data centers, wherein the first legal requirement is not applicable to the second portion of the data set;
- determine a second constraint on a location at which computations of a particular analysis operation of the one or more analysis operations can be performed;
- perform the computations of the particular analysis operation at the first data center in accordance with the second constraint, wherein to perform the computations of the particular analysis operation comprises to:
- obtain a replica of the second portion of the data set from the second data center to the first data center;
- and
- use at least the first portion and the replica of the second portion of the data set as input, at a set of computing resources of the cloud computing environment at the first data center, wherein the set of computing resources is selected in accordance with the second constraint;
- and
- provide, via the one or more programmatic interfaces, one or more audit records indicating (a) a location at which the first portion of the data set was stored and (b) a location at which the computations of the particular analysis operation were performed.
10 more elements
A computer-implemented method, comprising:
- determining, based at least in part on input received via one or more programmatic interfaces, a first constraint on a location at which a first portion of a data set can be stored in compliance with one or more regulations, wherein the data set is to be consumed as input by one or more analysis operations;
- storing the first portion of the data set at a first location selected in accordance with the first constraint;
- wherein a second portion of the data set is stored at a second location which differs from the first location, wherein the one or more regulations are not applicable to the second portion of the data set;
- and
- performing, at a set of computing resources, a particular analysis operation of the one or more analysis operations, at the first location in accordance with the first constraint, wherein performing the particular analysis operation comprises obtaining a replica of the second portion of the data set from the second location to the first location, wherein at least the first portion of the data set and the replica of the second portion of the data set are consumed as input by the particular analysis operation at the first location, and wherein the set of computing resources is selected in accordance with the one or more regulations.
2 more elements
One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors:
- determine, based at least in part on input received via one or more programmatic interfaces, a first constraint on a location at which a first portion of a data set can be stored, wherein the data set is to be consumed as input by one or more analysis operations;
- store the first portion of the data set at a first location selected in accordance with the first constraint;
- wherein a second portion of the data set is stored at a second location which differs from the first location, wherein the first constraint does not apply to the second portion of the data;
- and
- perform, at a set of computing resources, a particular analysis operation of the one or more analysis operations, at the first location in accordance with the first constraint, wherein performing the particular analysis operation comprises obtaining a replica of the second portion of the data set from the second location to the first location, wherein at least the first portion of the data set and the replica of the second portion of the data set are consumed as input by the particular analysis operation at the first location, and wherein the set of computing resources is selected in accordance with the one or more regulations.
2 more elements
Dependencies are inferred from the claim text.