# US 2014/0156586 A1 · Big-fast data connector between in-memory database system and data warehouse system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0156586 A1 - Document ID: US20140156586A1 - Status: Published application - Kind: A1 - Application number: 13/756,272 - Filed: 2013-01-31 - Published: 2014-06-05 - Earliest claimed: 2012-12-04 - Assignees: Pivotal Software, Inc. - Applicants: Pivotal Software, Inc. - Inventors: Charles M. Black; Christopher T. Keene; Richard Lamb; Barrett T. Oglesby - Web page: https://patentgrep.com/patent/US20140156586A1 ## CPC - G06F 16/2219: {Large Object storage; Management thereof} - G06F 16/2255: {Hash tables} - G06F 16/283: {Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Embodiments of the present invention include systems and methods for insuring better query consistency between at least two different databases, where one faster database has more up-to-date information than another slower database, and wherein updates are typically applied to the faster database first and then to the slower database. In embodiments, the systems and methods also insure that a query to the slower database is not performed until a set of one or more updates from the faster database have been applied to that slower database. ## Claims ### Claim 1 An in-memory database system that supports fast ingest data transfer between the in-memory database system and a data warehouse system, the in-memory database system comprising: an in-memory database instance that receives and stores data; a listener, communicatively coupled to the in-memory database instance, that, responsive to receiving notification that a data event has occurred in the in-memory database instance, selects a queue to which the data event will be sent and sends the data event to the selected queue to be sent to the data warehouse system; and the queue system comprising a plurality of queues, each queue comprising a proxy table communicatively coupled to receive the data event from the listener and an associated write-behind listener that determines whether an update condition has been being satisfied for the queue and responsive to an update condition being satisfied, sends the data event to the data warehouse system. ### Claim 2 (depends on 1) The in-memory database system of claim 1 wherein the listener selects a queue by performing the steps comprising: hashing a primary key of the data event to obtain a hash code; and selecting the queue for the data event by taking a modulus of the hash code by the number of queues. ### Claim 3 (depends on 1) The in-memory database system of claim 1 wherein the in-memory database system is a distributed system comprising a plurality of in-memory database system instances and wherein: one of the in-memory database system instances comprises the listener and the queue system; and the listener is communicatively coupled to receive notifications from the other in-memory database system instances of the distributed system. ### Claim 4 (depends on 3) The in-memory database system of claim 3 wherein a plurality of the in-memory database system instances each comprises a listener and a queue system. ### Claim 5 A computer-implement method for facilitating data transfer between an in-memory database system and a data warehouse system comprising: receiving notification at a listener that a data event has been occurred at the in-memory database system; identifying a queue to which the data event should be sent; sending the data event to the identified queue to be sent to the data warehouse system; and responsive to an update condition being satisfied, sending the data event to the data warehouse system. ### Claim 6 (depends on 5) The computer-implement method of claim 5 wherein the step of sending the data event to the identified queue to be sent to the data warehouse system comprises: placing a copy of the data event into a proxy table; and responsive to a microbatch listener, which is associated with the proxy table and receives notification that the data event has been entered into the proxy table, determining that a batch condition has been satisfied, writing a microbatch which includes the data event, to the data warehouse. ### Claim 7 (depends on 6) The computer-implement method of claim 6 wherein the batch condition is user-defined. ### Claim 8 (depends on 5) The computer-implement method of claim 5 wherein the step of identifying a queue to which the data event should be sent comprises: hashing a primary key of the data event to obtain a hash code; and selecting the queue for the data event by taking a modulus of the hash code by the number of queues. ### Claim 9 (depends on 5) The computer-implement method of claim 5 wherein the update condition is based upon: time between updates, number of data events in the queue, or both. ### Claim 10 (depends on 5) The computer-implement method of claim 5 wherein the in-memory database system is part of a distributed system comprising a plurality of in-memory database system instances and wherein one of the in-memory database system instances comprises the listener and the queues and the listener is communicatively coupled to receive notifications from the other in-memory database system instances of the distributed system. ### Claim 11 (depends on 5) The computer-implement method of claim 5 wherein the in-memory database system is part of a distributed system comprising a plurality of in-memory database system instances and wherein at least some of the in-memory database system instances comprise a listener and one or more queues and wherein each listener is communicatively coupled to send data events to the one or more queues in the at least some of the in-memory database system instances that comprise one or more queues. ### Claim 12 (depends on 5) The computer-implement method of claim 5 wherein at least some of the one or more queues are nested to form queue tiers and wherein one or more additional factors are used to assign the data event to a nested queue. ### Claim 13 A fast ingest system for facilitating data transfer between an in-memory database system and a data warehouse system, the fast ingest system comprising: a listener that, responsive to receiving notification that a data event has occurred in the in-memory database system, selects a queue in a queue system comprising a plurality of queues to which the data event will be sent and sends the data event to the selected queue to be sent to the data warehouse system; and the queue system comprising a plurality of queues, each a queue comprising a proxy table communicatively coupled to receive the data event from the listener and an associated write-behind listener that determines whether an update condition has been being satisfied for the queue and responsive to an update condition being satisfied, writes the data event to the data warehouse system. ### Claim 14 (depends on 13) The fast ingest system of claim 13 wherein a queue comprises a set of queues, each queue having at least one update condition for determining when to write data events to the data warehouse. ### Claim 15 (depends on 14) The fast ingest system of claim 14 wherein at least some of the queues in the set of queues are nested to form queue tiers. ### Claim 16 (depends on 13) The fast ingest system of claim 13 wherein the listener selects a queue by performing the steps comprising: hashing a primary key of the data event to obtain a hash code; and selecting a queue for the data event by taking a modulus of the hash code by the number of queues. ### Claim 17 (depends on 13) The fast ingest system of claim 13 wherein the in-memory database system is part of a distributed system comprising a plurality of in-memory database system instances and wherein: one of the in-memory database system instances comprises the listener and the queue system; and the listener is communicatively coupled to receive notifications from the other in-memory database system instances of the distributed system. ### Claim 18 (depends on 17) The fast ingest system of claim 17 wherein a plurality of the in-memory database system instances each comprises a listener and a queue system. ### Claim 19 (depends on 18) The fast ingest system of claim 18 wherein at least one of the queue systems in the plurality of in-memory database system instances comprises a plurality of queues. ### Claim 20 (depends on 19) The fast ingest system of claim 19 wherein at least one of the plurality of queues comprises nested queues.