A computer-implemented method for identifying a level of similarity between a user-provided data item and a data item within a set of data documents, the method comprising:
- clustering, by a reference map generator executing on a first computing device, in a two-dimensional metric space, a set of data documents selected according to at least one criterion, generating a semantic map;
- associating, using the semantic map, a coordinate pair with each data document in the set of data documents;
- generating, by a parser executing on the first computing device, an enumeration of terms occurring in the set of data documents;
- determining, by a representation generator executing on the first computing device, for each term in the enumeration, occurrence information including:
- (i) a number of data documents in which the term occurs, (ii) a number of occurrences of the term in each data document, and (iii) the coordinate pair associated with each data document in which the term occurs;
- generating, by the representation generator, for each term in the enumeration, a sparse distributed representation (SDR) using the occurrence information;
- storing, in an SDR database, each of the generated SDRs;
- receiving, by a query module executing on a second computing device and in communication with the first computing device, from a third computing device, a first term and a plurality of preference documents;
- generating, by the representation generator, a compound SDR using the plurality of preference documents;
- transmitting, by the query generator, the first term to a similarity engine executing on a fourth computing device;
- determining, by the similarity engine, a level of semantic similarity between a first SDR generated based on the first term and a second SDR of a second term, the second SDR retrieved from the SDR database;
- receiving, by the query module, from the similarity engine, the first term and the second term;
- transmitting, by the query module, to a full-text search system, a query for an identification of each of a subset of a second set of documents containing at least one term similar to at least one of the first term and the second term;
- transmitting, by the query module, to a full text search system, a query for an identification of each of a set of results documents similar to the first term;
- receiving, by the query module, the set of results documents from the full-text search system;
- generating, by the representation generator, an SDR for each of the documents identified in the set of results documents;
- determining, by a similarity engine executing on the second computing device, a level of semantic similarity between each SDR generated for the each of the set of results documents and the compound SDR;
- modifying, by a ranking module executing on the second computing device, an order of at least one document in the set of results documents, based on the determined level of semantic similarity;
- and
- providing, by the query module, to the third computing device, the identification of each of the set of search results documents in the modified order.