US 2021/0232585 A1 · LARGE SCALE DATA INGESTION

Abstract

Disclosed are some implementations of systems, apparatus, methods and computer program products for performing data ingestion. A request received from a client device is processed, where the request has a payload including a plurality of data items pertaining to a data object. An identifier associated with the data object is generated. Information including the data items is obtained from the request. The information including the data items is stored in association with the identifier in a first data store, where the first data store is not a relational database. The data items and identifier are obtained from the first data store and stored in a second data store, where the second data store is a relational database. A query including the identifier is processed to obtain the identifier from the query. It is determined whether the identifier obtained from the query is in the first data store. One or more of the data items associated with the identifier are retrieved from the first data store according to a result of determining whether the identifier obtained from the query is in the first data store.

Claims21

Dependent claims

A system comprising:

a database system implemented using a server system including one or more hardware processors, the one or more hardware processors configurable to cause:
processing a request received from a client device, the request having a payload including a plurality of data items pertaining to a data object;
generating an identifier associated with the data object;
obtaining information including the data items from the request;
storing the information including the data items in association with the identifier in a first data store, the first data store being a non-relational database;
obtaining the data items and the identifier from the first data store;
storing the obtained data items in association with the identifier in a second data store, the second data store being a relational database;
processing a query including the identifier to obtain the identifier from the query;
determining whether the identifier obtained from the query is stored in the first data store;
and
retrieving one or more of the data items associated with the identifier from the first data store or the second data store according to a result of the determining whether the identifier obtained from the query is stored in the first data store.

8 more elements

A computer program product comprising a non-transitory computer-readable medium storing computer-readable program code capable of being executed by one or more processors, the program code comprising computer-readable instructions configurable to cause:

processing a request received from a client device, the request having a payload including a plurality of data items pertaining to a data object;
generating an identifier associated with the data object;
obtaining information including the data items from the request;
storing the information including the data items in association with the identifier in a first data store, the first data store being a non-relational database;
obtaining the data items and the identifier from the first data store;
storing the obtained data items in association with the identifier in a second data store, the second data store being a relational database;
processing a query including the identifier to obtain the identifier from the query;
determining whether the identifier obtained from the query is stored in the first data store;
and
retrieving one or more of the data items associated with the identifier from the first data store or the second data store according to a result of the determining whether the identifier obtained from the query is stored in the first data store.

7 more elements

Dependencies are inferred from the claim text.

Large scale data ingestion

Life of this patent

Large scale data ingestion

Life of this patent

Abstract

Claims21