Back to results
US 11,030,262 B2Granted
Recyclable private memory heaps for dynamic search indexes
AssigneeVerizon Media Inc.
InventorsEdward Bortnikov, Guy Gueta, Pranav Sharma
Life of this patent
Estimating the term…
Abstract
In one embodiment, a search engine may generate and store a plurality of search index segments such that each of the search index segments is stored in a corresponding one of a plurality of heaps of memory. The plurality of search index segments may include inverted index segments mapping content to documents containing the content. A garbage collection module may release one or more heaps of the memory.
Claims25
A method, comprising:
- generating and storing in memory, by a search engine, a first search index segment of a search index, the first search index segment being stored in a first heap of the memory, the first search index segment including one or more first inverted index segments mapping content to a first set of documents, the search index being an inverted index;
- generating and storing in the memory, by the search engine, a second search index segment of the search index, the second search index segment being stored in a second heap of the memory concurrent with the first heap of the memory, the second search index segment including one or more second inverted index segments mapping content to a second set of documents, the second set of documents being more recently processed by the search engine than the first set of documents;
- updating the second search index segment until a threshold corresponding to a number of documents or an amount of memory has been reached;
- responsive to determining that the threshold corresponding to the number of documents or the amount of memory has been reached:
- cutting off a current generation of documents associated with the second heap of the memory by changing a status of the second search index segment being stored in the second heap of the memory from a read/write status to a read-only status;
- and
- initiating a next generation of documents by generating a third search index segment of the search index, wherein the third search index segment is assigned the read/write status and is stored in a new heap of the memory associated with the next generation of documents;
- and
- releasing, by a garbage collection module, a third heap of the memory, the third heap of the memory storing a fourth search index segment of the search index, the fourth search index segment generated by the search engine and including one or more third inverted index segments, the third heap of the memory being released based on at least one of a determination that a threshold of amount of memory has been consumed or a determination that a threshold number of search index segments are stored.
6 more elements
A system, comprising:
- one or more processors;
- and
- at least some memory, at least one of the one or more processors or the at least some memory being configured to:
- generate and store in memory, by a search engine, a first search index segment of a search index, the first search index segment being stored in a first heap of the memory, the first search index segment including one or more first inverted index segments mapping content to a first set of documents, the search index being an inverted index;
- generate and store in the memory, by the search engine, a second search index segment of the search index, the second search index segment being stored in a second heap of the memory concurrent with the first heap of the memory, the second search index segment including one or more second inverted index segments mapping content to a second set of documents, the second set of documents being more recently processed by the search engine than the first set of documents;
- update the second search index segment until a threshold corresponding to a number of documents or an amount of memory has been reached;
- responsive to determining that the threshold corresponding to the number of documents or the amount of memory has been reached:
- cut off a current generation of documents associated with the second heap of the memory by changing a status of the second search index segment being stored in the second heap of the memory from a read/write status to a read-only status;
- and
- initiate a next generation of documents by generating a third search index segment of the search index, wherein the third search index segment is assigned the read/write status and is stored in a new heap of the memory associated with the next generation of documents;
- and
- release, by a garbage collection module, a third heap of the memory, the third heap of the memory storing a fourth search index segment of the search index, the fourth search index segment generated by the search engine and including one or more third inverted index segments, the third heap of the memory being released based on a determination that a threshold of amount of memory has been consumed.
9 more elements
A non-transitory computer-readable storage medium storing thereon computer-readable storage medium, comprising:
- instructions configured to generate and store in memory, by a search engine, a first search index segment of a search index, the first search index segment being stored in a first heap of the memory, the first search index segment including one or more first inverted index segments mapping content to a first set of documents, the search index being an inverted index;
- instructions configured to generate and store in the memory, by the search engine, a second search index segment of the search index, the second search index segment being stored in a second heap of the memory concurrent with the first heap of the memory, the second search index segment including one or more second inverted index segments mapping content to a second set of documents, the second set of documents being more recently processed by the search engine than the first set of documents;
- instructions configured to update the second search index segment until a threshold corresponding to a number of documents or an amount of memory has been reached;
- instructions configured to responsive to determining that the threshold corresponding to the number of documents or the amount of memory has been reached:
- cut off a current generation of documents associated with the second heap of the memory by changing a status of the second search index segment being stored in the second heap of the memory from a read/write status to a read-only status;
- and
- initiate a next generation of documents by generating a third search index segment of the search index, wherein the third search index segment is assigned the read/write status and is stored in a new heap of the memory associated with the next generation of documents;
- and
- instructions configured to release a third heap of the memory, the third heap of the memory storing a fourth search index segment of the search index, the fourth search index segment generated by the search engine and including one or more third inverted index segments, the third heap of the memory being released based on a determination that a threshold number of search index segments are stored.
6 more elements
Dependencies are inferred from the claim text.