Document indexing, searching, and ranking with semantic intelligence

An apparatus comprising at least one processor and at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform a method for performing a search query associated with a plurality of search terms based at least in part on a plurality of documents, the method comprising:

• performing pre-processing of each document of the plurality of documents, wherein (1) each document of the plurality of documents comprises a plurality of words to generate an indexed representation of each document of the plurality of documents, and (2) the pre-processing of a document of the plurality of documents comprises:
• identifying one or more sentences in the document, wherein each sentence of the one or more sentences comprises one or more words from the plurality of words in the document;
• generating, for each n-gram of one or more n-grams associated with the document, one or more n-gram semantic scores based at least in part on one or more semantic proximity indicators for the corresponding n-gram, wherein (1) each n-gram of the one or more n-grams associated with the document comprises a combination of one or more words from the plurality of words in the document, (2) each semantic proximity indicator of the one or more semantic proximity indicators for an n-gram indicates a degree of semantic relatedness between a corresponding word of the one or more words in the n-gram and a corresponding candidate semantic label of one or more candidate semantic labels associated with the document, (3) each n-gram semantic score of the one or more n-gram semantic scores for a document is generated based at least in part on one or more frequency scores for the n-gram, and (4) the one or more frequency scores for a first one-gram of the one or more n-grams in the document are generated by (a) applying a logarithmic transformation to an occurrence frequency for the first one-gram in the document to generate a logarithmic occurrence frequency for the first one-gram in the document, and applying a modulo transformation to the logarithmic occurrence frequency to generate the one or more frequency scores for the first one-gram in the document;
• determining, based at least in part on each n-gram semantic score of the one or more n-gram semantic scores for an n-gram of the one or more n-grams, one or more semantic labels for each sentence of the one or more sentences in the document;
• and
• generating the indexed representation for the document of the plurality of documents based at least in part on the one or more semantic labels for each sentence of the one or more sentences in the document;
• processing the search query based at least in part on (1) the plurality of search terms, and (2) each indexed representation associated with a document of the plurality of documents, wherein the processing the search query comprises:
• providing the search query to a data storage unit, wherein (1) the data storage unit executes the search query to generate one or more search results for the search query, and (2) executing the search query comprises:
• generating, for each document of the plurality of documents, a relevance score in relation to the search query based at least in part on (1) one or more selected semantic labels for the document, (2) the one or more frequency scores, and (3) the one or more n-gram semantic scores;
• and
• generating the one or more search results based at least in part on the relevance scores for each document of the plurality of documents, wherein the one or more search results are ranked based at least in part on the relevance scores for each document of the plurality of documents;
• and
• transmitting the one or more search results to a computing device associated with the search query, wherein the one or more search results are displayed by the computing device.

A computer program product comprising a non-transitory computer readable medium storing computer readable instructions, the computer readable instructions configured, when executed by a processor, to cause the processor to perform a method for performing a search query associated with a plurality of search terms based at least in part on a plurality of documents, the method comprising:

• performing pre-processing of each document of the plurality of documents, wherein (1) each document comprises a plurality of words to generate an indexed representation of each document of the plurality of documents, and (2) the pre-processing of a document of the plurality of documents comprises:
• identifying one or more sentences in the document, wherein each sentence of the one or more sentences comprises one or more words from the plurality of words in the document;
• generating, for each n-gram of one or more n-grams associated with the document, one or more n-gram semantic scores based at least in part on one or more semantic proximity indicators for the corresponding n-gram, wherein (1) each n-gram of the one or more n-grams associated with the document comprises a combination of one or more words from the plurality of words in the document, (2) each semantic proximity indicator of the one or more semantic proximity indicators for an n-gram indicates a degree of semantic relatedness between a corresponding word of the one or more words in the n-gram and a corresponding candidate semantic label of one or more candidate semantic labels associated with the document, (3) each n-gram semantic score of the one or more n-gram semantic scores for a document is generated based at least in part on one or more frequency scores for the n-gram, and (4) the one or more frequency scores for a first one-gram of the one or more n-grams in the document are generated by (a) applying a logarithmic transformation to an occurrence frequency for the first one-gram in the document to generate a logarithmic occurrence frequency for the first one-gram in the document, and applying a modulo transformation to the logarithmic occurrence frequency to generate the one or more frequency scores for the first one-gram in the document;
• determining, based at least in part on each n-gram semantic score of the one or more n-gram semantic scores for an n-gram of the one or more n-grams, one or more semantic labels for each sentence of the one or more sentences in the document;
• and
• generating the indexed representation for the document of the plurality of documents based at least in part on the one or more semantic labels for each sentence of the one or more sentences in the document;
• processing the search query based at least in part on (1) the plurality of search terms, and (2) each indexed representation associated with a document of the plurality of documents, wherein the processing the search query comprises:
• providing the search query to a data storage unit, wherein (1) the data storage unit executes the search query to generate one or more search results for the search query, and (2) executing the search query comprises:
• generating, for each document of the plurality of documents, a relevance score in relation to the search query based at least in part on (1) one or more selected semantic labels for the document, (2) the one or more frequency scores, and (3) the one or more n-gram semantic scores;
• and
• generating the one or more search results based at least in part on the relevance scores for each document of the plurality of documents, wherein the one or more search results are ranked based at least in part on the relevance scores for each document of the plurality of documents;
• and
• transmitting the one or more search results to a computing device associated with the search query, wherein the one or more search results are displayed by the computing device.