# US 10,120,895 B2 · Mirroring, in memory, data from disk to improve query performance

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,120,895 B2
- Document ID: US10120895B2
- Status: Granted
- Kind: B2
- Application number: 15/877,125
- Filed: 2018-01-22
- Published: 2018-05-24
- Granted: 2018-11-06
- Earliest claimed: 2013-09-21
- Assignees: Oracle International Corporation
- Applicants: Oracle International Corporation
- Inventors: Jesse Kamp; Amit Ganesh; Vineet Marwah; Vivekanandhan Raja; Tirthankar Lahiri; Allison L. Holloway; Sanket Hase; Shasank K. Chavan; Niloy Mukherjee; Teck Hua Lee; Michael J. Gleeson; Krishna Kunchithapadam
- Web page: https://patentgrep.com/patent/US10120895B2

## CPC

- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors}
- G06F 3/065: {Replication mechanisms}
- G06F 3/0665: {at area level, e.g. provisioning of virtual or logical volumes}
- G06F 3/0689: {Disk arrays, e.g. RAID, JBOD}
- G06F 11/2058: {using more than 2 mirrored copies}
- G06F 16/221: {Column-oriented storage; Management thereof}
- G06F 16/2453: Query optimisation
- G06F 16/275: {Synchronous replication}
- G06F 16/27: Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F 2201/80: Database-specific techniques
- G06F 2201/855: Details of asynchronous mirroring using a journal to transfer not-yet-mirrored changes
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data.

## Claims

### Claim 1

A method comprising:
maintaining, on persistent storage, a database that is accessible to a database server running on one or more computer systems;
wherein the database includes persistent-format data, stored on the persistent storage, in a persistent format;
converting, by the one or more computer systems, the persistent-format data to a mirror format to produce mirror-format data;
storing, by the one or more computer systems, the mirror-format data within volatile memory;
wherein the mirror format is different from the persistent format;
wherein, within the volatile memory, the mirror-format data is accessible to the database server;
wherein each data item in the mirror-format data is a copy of a corresponding data item in the persistent-format data;
receiving, by the database server, a request to execute a database command that requires a required set of data items; and
during execution of the database command:
the database server obtaining a first subset of data items, of the required set of data items, from the persistent-format data, and
the database server obtaining a second subset of data items, of the required set of data items, from the mirror-format data;
wherein, when the database server receives the request to execute the database command, a current version of each data item in the second subset of data items exists in both the persistent-format data and the mirror-format data;
wherein the first subset of data items is obtained from the persistent-format data based on the database server determining that obtaining the first subset of data items, for execution of the database command, from the persistent-format data would be more efficient than obtaining the first subset of data items, for execution of the database command, from the mirror-format data.

### Claim 2 (depends on 1)

The method of claim 1 wherein the second subset of data items is obtained from the mirror-format data based on the database server determining that obtaining the second subset of data items, for execution of the database command, from the mirror-format data would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data.

### Claim 3 (depends on 1)

The method of claim 1 further comprising:
maintaining, in the volatile memory, a cache of data items that are in the persistent format; and
wherein the database server determining whether obtaining the first subset of data items from the persistent-format data would be more efficient than obtaining the first subset of data items from the mirror-format data is based, at least in part, on whether the first subset of data items currently resides in the cache.

### Claim 4 (depends on 1)

The method of claim 1 wherein:
within the database, data items that belong to the persistent-format data are stored in a particular order; and
the method further comprises, within volatile memory, ordering data items that belong to the mirror-format data in a manner that reflects said particular order.

### Claim 5 (depends on 1)

The method of claim 1 wherein the mirror format is a column-major format.

### Claim 6 (depends on 1)

The method of claim 1 wherein the persistent format is a row-major format.

### Claim 7 (depends on 1)

The method of claim 1 further comprising maintaining the mirror-format data transactionally consistent with the persistent-format data.

### Claim 8 (depends on 7)

The method of claim 7 wherein maintaining the mirror-format data transactionally consistent with the persistent-format data comprises implicitly making particular changes to data items that belong to the mirror-format data, without directly making the particular changes to data items that belong to the mirror-format data, by recording the particular changes in one or more journals stored in the volatile memory.

### Claim 9 (depends on 8)

The method of claim 8 wherein the database server obtaining the second subset of data items, of the required set of data items, from the mirror-format data comprises:
determining whether the one or more journals include a version of a certain data item, of the second subset of data items, that is more recent than a copy of the certain data item in the mirror-format data; and
in response to determining that the one or more journals include a version of the certain data item that is more recently updated than the copy of the certain data item in the mirror-format data, retrieving the certain data item from the one or more journals.

### Claim 10 (depends on 8)

The method of claim 8 further comprising:
in response to a change, made as part of a transaction, to a data item that belongs to the persistent-format data, performing the steps of:
before the transaction is committed, storing a record of the change in a private journal that is only visible to the transaction; and
in response to the transaction committing, moving the record of the change to a global journal that is visible to other transactions.

### Claim 11 (depends on 10)

The method of claim 10 further comprising merging at least some changes indicated in the global journal into the mirror-format data.

### Claim 12 (depends on 11)

The method of claim 11 wherein the mirror format is a compressed format and the step of merging comprises:
decompressing at least a portion of the mirror-format data to produce uncompressed data;
updating the uncompressed data to produce updated uncompressed data that reflects said at least some changes; and
compressing the updated uncompressed data.

### Claim 13 (depends on 1)

The method of claim 1 wherein the mirror format is a compressed format.

### Claim 14

A method comprising:
maintaining, on persistent storage, a database that is accessible to a database server running on one or more computer systems;
wherein the database includes persistent-format data, stored on the persistent storage, in a persistent format;
converting, by the one or more computer systems, the persistent-format data to a mirror format to produce mirror-format data;
storing, by the one or more computer systems, the mirror-format data within volatile memory;
wherein the mirror format is different from the persistent format;
wherein, within the volatile memory, the mirror-format data is accessible to the database server;
wherein each data item in the mirror-format data is a copy of a corresponding data item in the persistent-format data;
receiving, by the database server, a request to execute a database command that requires a required set of data items that satisfy a condition; and
during execution of the database command:
the database server using the mirror-format data to determine which data items, from the persistent-format data, are condition-satisfying data items that satisfy the condition, and
the database server obtaining the condition-satisfying data items from the persistent-format data;
wherein the mirror-format data includes a column vector for a particular column;
wherein the condition comprises a particular condition on the particular column; and
wherein using the mirror-format data to determine which data items, from the persistent-format data, are condition-satisfying data items that satisfy the condition comprises identifying which values, within the column vector, satisfy the particular condition.

### Claim 15

A method comprising:
maintaining, on persistent storage, a database that is accessible to a database server running on one or more computer systems;
wherein the database includes persistent-format data, stored on the persistent storage, in a persistent format;
converting, by the one or more computer systems, the persistent-format data to a mirror format to produce mirror-format data;
storing, by the one or more computer systems, the mirror-format data within volatile memory;
wherein the mirror format is different from the persistent format;
wherein, within the volatile memory, the mirror-format data is accessible to the database server;
wherein each data item in the mirror-format data is a copy of a corresponding data item in the persistent-format data;
maintaining, in the volatile memory, a cache of data items that are in the persistent format;
receiving, by the database server, a request to execute a database command that requires a required set of data items; and
during execution of the database command:
determining whether a first subset of data items, of the required set of data items, currently exist in the mirror-format data,
responsive to determining that the first subset of data items do not currently exist in the mirror-format data, determining whether the first subset of data items currently exist in the cache of data items,
responsive to determining that the first subset of data items do not currently exist in the cache of data items, the database server obtaining the first subset of data items from the persistent storage, and
the database server obtaining a second subset of data items, of the required set of data items, from the mirror-format data;
wherein, when the database server receives the request to execute the database command, a current version of each data item in the second subset of data items exists in both the persistent-format data and the mirror-format data.

### Claim 16

A method comprising:
maintaining, on persistent storage, a database that is accessible to a database server running on one or more computer systems;
wherein the database includes persistent-format data, stored on the persistent storage, in a persistent format;
converting, by the one or more computer systems, the persistent-format data to a mirror format to produce mirror-format data;
storing, by the one or more computer systems, the mirror-format data within volatile memory;
wherein the mirror format is different from the persistent format;
wherein, within the volatile memory, the mirror-format data is accessible to the database server;
wherein each data item in the mirror-format data is a copy of a corresponding data item in the persistent-format data;
receiving, by the database server, a request to execute a database command that requires a required set of data items; and
during execution of the database command:
the database server obtaining a first subset of data items, of the required set of data items, from the persistent-format data, and
the database server obtaining a second subset of data items, of the required set of data items, from the mirror-format data based on the database server determining that obtaining the second subset of data items, for execution of the database command, from the mirror-format data would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data;
wherein, when the database server receives the request to execute the database command, a current version of each data item in the second subset of data items exists in both the persistent-format data and the mirror-format data.

### Claim 17 (depends on 16)

The method of claim 16 wherein determining that obtaining the second subset of data items, for execution of the database command, from the mirror-format data would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data comprises determining that the second subset of data items in mirror-format data is located within volatile memory of a database server instance executing the database command.

### Claim 18 (depends on 16)

The method of claim 16 wherein:
a first database server instance executes the database command; and
determining that obtaining the second subset of data items, for execution of the database command, from the mirror-format data would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data comprises:
determining that obtaining the second subset of data items, for execution of the database command, from mirror-format data located within volatile memory of a second database server instance, other than the first database server instance, would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data.

### Claim 19 (depends on 16)

The method of claim 16 further comprising:
maintaining, in the volatile memory, a cache of data items that are in the persistent format; and
the second subset of data items comprises data from one or more requested rows; and
determining that obtaining the second subset of data items, for execution of the database command, from the mirror-format data would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data comprises:
determining that at least one row of the one or more requested rows does not currently reside in the cache of data items.

### Claim 20

One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause:
maintaining, on persistent storage, a database that is accessible to a database server running on one or more computer systems;
wherein the database includes persistent-format data, stored on the persistent storage, in a persistent format;
converting the persistent-format data to a mirror format to produce mirror-format data;
storing the mirror-format data within volatile memory;
wherein the mirror format is different from the persistent format;
wherein, within the volatile memory, the mirror-format data is accessible to the database server;
wherein each data item in the mirror-format data is a copy of a corresponding data item in the persistent-format data;
receiving, by the database server, a request to execute a database command that requires a required set of data items; and
during execution of the database command:
the database server obtaining a first subset of data items, of the required set of data items, from the persistent-format data, and
the database server obtaining a second subset of data items, of the required set of data items, from the mirror-format data;
wherein, when the database server receives the request to execute the database command, a current version of each data item in the second subset of data items exists in both the persistent-format data and the mirror-format data;
wherein the first subset of data items is obtained from the persistent-format data based on the database server determining that obtaining the first subset of data items, for execution of the database command, from the persistent-format data would be more efficient than obtaining the first subset of data items, for execution of the database command, from the mirror-format data.

### Claim 21 (depends on 20)

The one or more non-transitory computer-readable media of claim 20 wherein the second subset of data items is obtained from the mirror-format data based on the database server determining that obtaining the second subset of data items, for execution of the database command, from the mirror-format data would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data.

### Claim 22 (depends on 20)

The one or more non-transitory computer-readable media of claim 20 wherein the instructions further comprise instructions which, when executed by one or more processors, cause:
maintaining, in the volatile memory, a cache of data items that are in the persistent format; and
wherein the database server determining whether obtaining the first subset of data items from the persistent-format data would be more efficient than obtaining the first subset of data items from the mirror-format data is based, at least in part, on whether the first subset of data items currently resides in the cache.

### Claim 23 (depends on 20)

The one or more non-transitory computer-readable media of claim 20 wherein:
within the database, data items that belong to the persistent-format data are stored in a particular order; and
the instructions further comprise instructions which, when executed by one or more processors, cause, within volatile memory, ordering data items that belong to the mirror-format data in a manner that reflects said particular order.

### Claim 24 (depends on 20)

The one or more non-transitory computer-readable media of claim 20 wherein the mirror format is a column-major format.

### Claim 25 (depends on 20)

The one or more non-transitory computer-readable media of claim 20 wherein the persistent format is a row-major format.

### Claim 26 (depends on 20)

The one or more non-transitory computer-readable media of claim 20, wherein the instructions further comprise instructions which, when executed by one or more processors, cause maintaining the mirror-format data transactionally consistent with the persistent-format data.

### Claim 27 (depends on 26)

The one or more non-transitory computer-readable media of claim 26 wherein maintaining the mirror-format data transactionally consistent with the persistent-format data comprises implicitly making particular changes to data items that belong to the mirror-format data, without directly making the particular changes to data items that belong to the mirror-format data, by recording the particular changes in one or more journals stored in the volatile memory.

### Claim 28 (depends on 27)

The one or more non-transitory computer-readable media of claim 27 wherein the database server obtaining the second subset of data items, of the required set of data items, from the mirror-format data comprises:
determining whether the one or more journals include a version of a certain data item, of the second subset of data items, that is more recent than a copy of the certain data item in the mirror-format data; and
in response to determining that the one or more journals include a version of the certain data item that is more recently updated than the copy of the certain data item in the mirror-format data, retrieving the certain data item from the one or more journals.

### Claim 29 (depends on 27)

The one or more non-transitory computer-readable media of claim 27 wherein the instructions further comprise instructions which, when executed by one or more processors, cause:
in response to a change, made as part of a transaction, to a data item that belongs to the persistent-format data, performing the steps of:
before the transaction is committed, storing a record of the change in a private journal that is only visible to the transaction; and
in response to the transaction committing, moving the record of the change to a global journal that is visible to other transactions.

### Claim 30 (depends on 29)

The one or more non-transitory computer-readable media of claim 29 wherein the instructions further comprise instructions which, when executed by one or more processors, cause merging at least some changes indicated in the global journal into the mirror-format data.

### Claim 31 (depends on 30)

The one or more non-transitory computer-readable media of claim 30 wherein the mirror format is a compressed format and the step of merging comprises:
decompressing at least a portion of the mirror-format data to produce uncompressed data;
updating the uncompressed data to produce updated uncompressed data that reflects said at least some changes; and
compressing the updated uncompressed data.

### Claim 32 (depends on 20)

The one or more non-transitory computer-readable media of claim 20 wherein the mirror format is a compressed format.

### Claim 33

One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause:
maintaining, on persistent storage, a database that is accessible to a database server running on one or more computer systems;
wherein the database includes persistent-format data, stored on the persistent storage, in a persistent format;
converting the persistent-format data to a mirror format to produce mirror-format data;
storing the mirror-format data within volatile memory;
wherein the mirror format is different from the persistent format;
wherein, within the volatile memory, the mirror-format data is accessible to the database server;
wherein each data item in the mirror-format data is a copy of a corresponding data item in the persistent-format data;
receiving, by the database server, a request to execute a database command that requires a required set of data items that satisfy a condition; and
during execution of the database command:
the database server using the mirror-format data to determine which data items, from the persistent-format data, are condition-satisfying data items that satisfy the condition, and
the database server obtaining the condition-satisfying data items from the persistent-format data;
wherein the mirror-format data includes a column vector for a particular column;
wherein the condition comprises a particular condition on the particular column; and
wherein using the mirror-format data to determine which data items, from the persistent-format data, are condition-satisfying data items that satisfy the condition comprises identifying which values, within the column vector, satisfy the particular condition.

### Claim 34

One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause:
maintaining, on persistent storage, a database that is accessible to a database server running on one or more computer systems;
wherein the database includes persistent-format data, stored on the persistent storage, in a persistent format;
converting the persistent-format data to a mirror format to produce mirror-format data;
storing the mirror-format data within volatile memory;
wherein the mirror format is different from the persistent format;
wherein, within the volatile memory, the mirror-format data is accessible to the database server;
wherein each data item in the mirror-format data is a copy of a corresponding data item in the persistent-format data;
maintaining, in the volatile memory, a cache of data items that are in the persistent format;
receiving, by the database server, a request to execute a database command that requires a required set of data items; and
during execution of the database command:
determining whether a first subset of data items, of the required set of data items, currently exist in the mirror-format data,
responsive to determining that the first subset of data items do not currently exist in the mirror-format data, determining whether the first subset of data items currently exist in the cache of data items,
responsive to determining that the first subset of data items do not currently exist in the cache of data items, the database server obtaining the first subset of data items from the persistent storage, and
the database server obtaining a second subset of data items, of the required set of data items, from the mirror-format data;
wherein, when the database server receives the request to execute the database command, a current version of each data item in the second subset of data items exists in both the persistent-format data and the mirror-format data.

### Claim 35

One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause:
maintaining, on persistent storage, a database that is accessible to a database server running on one or more computer systems;
wherein the database includes persistent-format data, stored on the persistent storage, in a persistent format;
converting the persistent-format data to a mirror format to produce mirror-format data;
storing the mirror-format data within volatile memory;
wherein the mirror format is different from the persistent format;
wherein, within the volatile memory, the mirror-format data is accessible to the database server;
wherein each data item in the mirror-format data is a copy of a corresponding data item in the persistent-format data;
receiving, by the database server, a request to execute a database command that requires a required set of data items; and
during execution of the database command:
the database server obtaining a first subset of data items, of the required set of data items, from the persistent-format data, and
the database server obtaining a second subset of data items, of the required set of data items, from the mirror-format data based on the database server determining that obtaining the second subset of data items, for execution of the database command, from the mirror-format data would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data;
wherein, when the database server receives the request to execute the database command, a current version of each data item in the second subset of data items exists in both the persistent-format data and the mirror-format data.

### Claim 36 (depends on 35)

The one or more non-transitory computer-readable media of claim 35 wherein determining that obtaining the second subset of data items, for execution of the database command, from the mirror-format data would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data comprises determining that the second subset of data items in mirror-format data is located within volatile memory of a database server instance executing the database command.

### Claim 37 (depends on 35)

The one or more non-transitory computer-readable media of claim 35 wherein:
a first database server instance executes the database command; and
determining that obtaining the second subset of data items, for execution of the database command, from the mirror-format data would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data comprises:
determining that obtaining the second subset of data items, for execution of the database command, from mirror-format data located within volatile memory of a second database server instance, other than the first database server instance, would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data.

### Claim 38 (depends on 35)

The one or more non-transitory computer-readable media of claim 35 wherein the instructions further comprise instructions which, when executed by one or more processors, cause:
maintaining, in the volatile memory, a cache of data items that are in the persistent format; and
the second subset of data items comprises data from one or more requested rows; and
determining that obtaining the second subset of data items, for execution of the database command, from the mirror-format data would be more efficient than obtaining the second subset of data items, for execution of the database command, from the persistent-format data comprises:
determining that at least one row of the one or more requested rows does not currently reside in the cache of data items.