Back to results
US 8,898,138 B2Granted
Efficiently indexing and searching similar data
AssigneeEMC Corporation
InventorsWindsor W. Hsu, R. Hugo Patterson
Life of this patent
Estimating the term…
Abstract
Techniques for efficiently indexing and searching similar data are described herein. According to one embodiment, in response to a query for one or more terms received from a client, a query index is accessed to retrieve a list of one or more super files. Each super file is associated with a group of similar files. Each super file includes terms and/or sequences of terms obtained from the associated group of similar files. Thereafter, the super files representing groups of similar files are presented to the client, where each of the super files includes at least one of the queried terms. Other methods and apparatuses are also described.
Claims15
A computer implemented method, comprising:
- in response to a set of content files to be indexed, grouping the content files into one or more groups, each group including one or more content files having similar data;
- for each group, constructing a super file to contain one or more sequences of one or more terms extracted from the associated content files, including
- including in the super file one or more sequences of one or more terms obtained from a content file selected from a group of content files,
- for each of remaining content files in the group, determining whether each segment of a remaining content file exists in the super file, and
- appending a segment of the remaining content file to the super file if the segment does not exist in the super file, wherein the determining and appending comprises
- determining whether a current segment of a content file exists in the super file, the content file having a plurality of segments and each segment having a plurality of terms, and
- appending the current segment of the content file in the super file if the current segment of the content file does not exist in the super file;
- and
- indexing the super file for each group in a query index without indexing the associated content files, wherein the query index is used to retrieve one or more super files that contain one or more terms from a search query searching for content files that contain at least one of the one or more terms.
6 more elements
A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:
- in response to a set of content files to be indexed, grouping the content files into one or more groups, each group including one or more content files having similar data;
- for each group, constructing a super file to contain one or more sequences of one or more terms extracted from the associated content files, including
- including in the super file one or more sequences of one or more terms obtained from a content file selected from a group of content files,
- for each of remaining content files in the group, determining whether each segment of a remaining content file exists in the super file, and
- appending a segment of the remaining content file to the super file if the segment does not exist in the super file, wherein the determining and appending comprises
- determining whether a current segment of a content file exists in the super file, the content file having a plurality of segments and each segment having a plurality of terms, and
- appending the current segment of the content file in the super file if the current segment of the content file does not exist in the super file;
- and
- indexing the super file for each group in a query index without indexing the associated content files, wherein the query index is used to retrieve one or more super files that contain one or more terms from a search query searching for content files that contain at least one of the one or more terms.
6 more elements
A data processing system, comprising:
- a processor;
- and
- a memory coupled to the processor storing instructions, which when executed from the memory, cause the processor to
- in response to a set of content files to be indexed, group the content files into one or more groups, each group including one or more content files having similar data,
- for each group, construct a super file to contain one or more sequences of one or more terms extracted from the associated content files, including
- including in the super file one or more sequences of one or more terms obtained from a content file selected from a group of content files, and
- for each of remaining content files in the group, determining whether each segment of a remaining content file exists in the super file, and
- appending a segment of the remaining content file to the super file if the segment does not exist in the super file, wherein the determining and appending comprises
- determining whether a current segment of a content file exists in the super file, the content file having a plurality of segments and each segment having a plurality of terms, and
- appending the current segment of the content file in the super file if the current segment of the content file does not exist in the super file, and
- index the super file for each group in a query index without indexing the associated content files, wherein the query index is used to retrieve one or more super files that contain one or more terms from a search query searching for content files that contain at least one of the one or more terms.
8 more elements
Dependencies are inferred from the claim text.