Asynchronous queries on secondary data cores in a distributed computing system
AssigneeVMware, Inc.
InventorsRamsés V. Morales, Mirza Mohsin Beg, Shashank Pedamallu
Life of this patent
Estimating the term…
Abstract
The present disclosure relates generally to techniques for processing asynchronous queries across multiple data cores including secondary data cores stored in the secondary storage system in a distributed computing system. Secondary data cores are recovered in the data plane from secondary storage in order to be accessible for searching. Using this technique, asynchronous queries are run in parallel allowing the client to dynamically manage the queries and receive notifications when results are available.
Claims33
A method for querying timestamped data items stored across a plurality of data cores and a plurality of secondary data cores, the plurality of data cores stored at a data plane of a distributed computing system and the plurality of secondary data cores stored at a secondary storage system external to the distributed computing system, the method comprising:
- at a management plane of the distributed computing system:
- processing a received search request for one or more data items, wherein the search request includes a specified time range;
- determining whether one or more secondary data cores of the plurality of secondary data cores contain one or more data items corresponding to the specified time range;
- in accordance with a determination that one or more secondary data cores of the plurality of secondary data cores contain one or more data items corresponding to the specified time range:
- determining whether an amount of available storage in the data plane satisfies one or more predetermined criteria required to recover at least one secondary data core of the one or more secondary data cores in the data plane;
- in accordance with a determination that an amount of available storage in the data plane satisfies one or more predetermined criteria required to recover at least one secondary data core of the one or more secondary data cores in the data plane:
- causing recovery of the at least one secondary data core in the data plane;
- and
- executing the search request using the at least one recovered secondary data core in the data plane.
A method for querying timestamped data items stored across a plurality of data cores and a plurality of secondary data cores, the plurality of data cores stored at a data plane of a distributed computing system and the plurality of secondary data cores stored at a secondary storage system external to the distributed computing system, the method comprising:
- at a management plane of the distributed computing system:
- processing a received search request for one or more data items, wherein the search request includes a specified time range;
- determining whether a portion of one or more secondary data cores contains one or more data items relevant to the search request;
- in accordance with a determination that a portion of one or more secondary data cores contains one or more data items corresponding to the specified time range, changing a state of the one or more secondary data cores from a first state in which the one or more secondary data cores are not accessible from the data plane to a second state in which the one or more secondary data cores are accessible from the data plane and retrieving, via an in-memory file system of the data plane, the one or more data items from the portion of the one or more secondary data cores from the secondary storage system while the one or more secondary data cores are in the second state;
- and
- executing the search request using the one or more data items secondary data cores.
A non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of a distributed computing system, the one or more programs including instructions for:
- at a management plane of the distributed computing system:
- processing a received search request for one or more data items, wherein the search request includes a specified time range;
- determining whether one or more secondary data cores of the plurality of secondary data cores contain one or more data items corresponding to the specified time range;
- in accordance with a determination that one or more secondary data cores of the plurality of secondary data cores contain one or more data items corresponding to the specified time range:
- determining whether an amount of available storage in the data plane satisfies one or more predetermined criteria required to recover at least one secondary data core of the one or more secondary data cores in the data plane;
- in accordance with a determination that an amount of available storage in the data plane satisfies one or more predetermined criteria required to recover at least one secondary data core of the one or more secondary data cores in the data plane:
- causing recovery of the at least one secondary data core in the data plane;
- and
- executing the search request using the at least one recovered secondary data core in the data plane.
A distributed computing system comprising:
- one or more processors and
- a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
- at a management plane of the distributed computing system:
- processing a received search request for one or more data items, wherein the search request includes a specified time range;
- determining whether one or more secondary data cores of the plurality of secondary data cores contain one or more data items corresponding to the specified time range;
- in accordance with a determination that one or more secondary data cores of the plurality of secondary data cores contain one or more data items corresponding to the specified time range:
- determining whether an amount of available storage in the data plane satisfies one or more predetermined criteria required to recover at least one secondary data core of the one or more secondary data cores in the data plane;
- in accordance with a determination that an amount of available storage in the data plane satisfies one or more predetermined criteria required to recover at least one secondary data core of the one or more secondary data cores in the data plane:
- causing recovery of the at least one secondary data core in the data plane;
- and
- executing the search request using the at least one recovered secondary data core in the data plane.
A non-transitory computer-readable storage medium comprising one or more programs configured to be executed by one or more processors of a distributed computing system, the one or more programs including instructions for:
- at a management plane of the distributed computing system:
- processing a received search request for one or more data items located in a plurality of data cores stored at a data plane of the distributed computing system and a plurality of secondary data cores stored external to the data plane, wherein the search request includes a specified time range;
- determining whether one or more secondary data cores of the plurality of secondary data cores contains one or more data items relevant to the search request;
- in accordance with a determination that the one or more secondary data cores contain one or more data items relevant to the search request, replacing one or more data cores of the plurality of data cores with the one or more secondary data cores;
- and
- executing the search request using the one or more secondary data cores.
A distributed computing system comprising:
- one or more processors and
- a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
- at a management plane of the distributed computing system:
- processing a received search request for one or more data items located in a plurality of data cores stored in a data plane of the distributed computing system and a plurality of secondary data cores stored external to the data plane, wherein the search request includes a specified time range;
- determining whether the plurality of secondary data cores contains one or more data items relevant to the search request;
- in accordance with a determination that a secondary data core of the plurality of secondary data cores contains one or more of the data items relevant to the search request, transferring, the secondary data core from the secondary storage system to an extra data plane container of the data plane that is shared by one or more other tenants;
- and
- executing the search request using the portion of the one or more secondary data cores.
Dependencies are inferred from the claim text.