# US 11,308,115 B2 · Method and system for persisting data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,115 B2
- Document ID: US11308115B2
- Status: Granted
- Kind: B2
- Application number: 16/088,149
- Filed: 2017-03-14
- Published: 2019-05-16
- Granted: 2022-04-19
- Earliest claimed: 2016-03-14
- Assignees: Kinaxis Inc.
- Applicants: Rubikloud Technologies Inc.
- Inventors: Frank Thomas
- Web page: https://patentgrep.com/patent/US11308115B2

## CPC

- G06F 16/254: {Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses}
- G06F 16/2228: {Indexing structures}
- G06F 16/25: Integrating or interfacing systems involving database management systems
- G06Q 10/06: Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q 10/067: Enterprise or organisation modelling
- G06Q 10/10: Office automation; Time management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A method and system for persisting data are provided. Batch data is periodically extracted via a computer system from at least one primary data source. Batch data is transformed via the computer system. The batch data is loaded, via the computer system, into a database. Real-time data is continuously received from at least one secondary data source. The received real-time data is indexed in an index. Business intelligence reports are generated from both the real-time data in the index and the batch data in the database.

## Claims

### Claim 1

A method for persisting and reconciling one or more data sources on a computer system, the method comprising:
periodically extracting batch data from at least one primary data source, the at least one primary data source not being real-time in nature;
transforming the batch data into a first format specified by a data schema for a database;
loading the transformed batch data into the database;
continuously receiving real-time data from at least one secondary data source;
pushing the real-time data onto a distributed message queue system;
indexing the received real-time data to generate an index;
storing the real-time data by key-value in a distributed key-value store for subsequent integration into the database;
periodically extracting the real-time data from the distributed key-value store and placing the extracted real-time data into an interim data store;
transforming the extracted real-time data stored in the interim data store into a second format specified by the data schema for the database; and
generating business intelligence reports comprising the real-time data in the index, the transformed real-time data in the database, the extracted batch data and the transformed batch data in the database.

### Claim 2 (depends on 1)

The method of claim 1, wherein transforming the batch data comprises at least one of discarding certain data, translating the data to account for differences in a representation of the data, determining the age of the data, sorting the data, joining the data where the data is from multiple sources, and aggregating the data.

### Claim 3 (depends on 1)

The method of claim 1, wherein the at least one primary data source does not publish or push the data immediately.

### Claim 4 (depends on 1)

The method of claim 1, further comprising analyzing the data in the database for generating the business intelligence reports, the analyzing comprising at least one of statistical analysis, machine learning techniques, and data mining.

### Claim 5 (depends on 1)

The method of claim 1, further comprising periodically reconciling the real-time data against the batch data.

### Claim 6 (depends on 1)

The method of claim 1, wherein the real-time data is acquired via at least one of REST requests and a user datagram protocol.

### Claim 7

A system for persisting and reconciling one or more data sources, the system comprising one or more processors and a data storage, the one or more processors configured to execute, or direct to be executed:
an extract-transform-load module to:
periodically extract batch data from at least one primary data source, the at least one primary data source not being real time in nature;
transform the batch data into a first format specified by a data schema for a database; and
load the transformed batch data into the database;
a real-time distributed processing module to:
continuously receive real-time data from at least one secondary data source;
push the real-time data onto a distributed message queue system;
index the received real-time data to generate an index;
store the real-time data by key-value in a distributed key-value store for subsequent integration into the database;
periodically extract the real-time data from the distributed key-value store and place the extracted real-time data into an interim data store; and
transform the extracted real-time data stored in the interim data store into a second format specified by the data schema for the database; and
at least one business intelligence tool to generate business intelligence reports comprising the real-time data in the index, the transformed real-time data in the database, the extracted batch data and the transformed batch data in the database.

### Claim 8 (depends on 7)

The system of claim 7, wherein transforming the batch data by the extract-transform-load module comprises transforming the batch data into a format specified by a data schema.

### Claim 9 (depends on 7)

The system of claim 7, wherein transforming the batch data by the extract-transform-load module comprises at least one of discarding certain data, translating the data to account for differences in a representation of the data, determining the age of the data, sorting the data, joining the data where the data is from multiple sources, and aggregating the data.

### Claim 10 (depends on 7)

The system of claim 7, wherein the at least one primary data source does not publish or push the data immediately.

### Claim 11 (depends on 7)

The system of claim 7, further comprising a data analysis module executable to analyze the data in the database for generating the business intelligence reports, the analysis comprising at least one of statistical analysis, machine learning techniques, and data mining.

### Claim 12 (depends on 7)

The system of claim 7, wherein the at least one business intelligence tool periodically reconciles the real-time data against the batch data.

### Claim 13 (depends on 7)

The system of claim 7, wherein the real-time data is acquired by the extract-transform-load module via at least one of REST requests and a user datagram protocol.