# US 12,118,312 B2 · Systems and methods of natural language processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,312 B2 - Document ID: US12118312B2 - Status: Granted - Kind: B2 - Application number: 17/491,056 - Filed: 2021-09-30 - Published: 2023-04-06 - Granted: 2024-10-15 - Earliest claimed: 2021-09-30 - Assignees: Nasdaq, Inc. - Applicants: Nasdaq, Inc. - Inventors: Hyunsoo Jeong; Josep Puig Ruiz; Douglas Hamilton; Niharika Sharma - Web page: https://patentgrep.com/patent/US12118312B2 ## CPC - G06F 40/242: Dictionaries - G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06F 40/30: Semantic analysis - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Natural language processing techniques provide sentence level analysis on one or more topics that are associated with keywords. Indirect learning is used to expand the understanding of the keywords and associated topics. Semantic similarity is used on a sentence-level to assess whether a given sentence relates or mentions a particular topic. In some examples, additional keywords are suggested using filtering techniques in connection with graph embedding-based entity linking techniques. ## Claims ### Claim 1 A computer system for performing natural language processing, the computer system comprising: a processing system comprising instructions that, when executed by at least one hardware processor included in the processing system, cause the at least one hardware processor to perform operations comprising: obtaining target textual data; obtaining input data that includes at least one topic with a plurality of specified keywords for the at least one topic; for each corresponding one of the plurality of specified keywords for the at least one topic, performing a query to retrieve a descriptive text body that is associated with the corresponding one of the plurality of specified keywords, wherein the descriptive text body includes multiple different sentences; generating a first plurality of sentences from the target textual data; generating a second plurality of sentences from each of the descriptive text bodies retrieved in connection with each corresponding keyword; for each corresponding one of the first plurality of sentences, calculating a plurality of sentence similarity metrics between the corresponding one of the first plurality of sentences and each one of the sentences in the second plurality of sentences; and for each corresponding one of the first plurality of sentences, calculating a metric between the corresponding one of the first plurality of sentences and the at least one topic based on multiple ones of the calculated plurality of sentence similarity metrics for the corresponding one of the first plurality of sentences. ### Claim 2 (depends on 1) The computer system of claim 1, wherein the operations further comprise: generating a first plurality of vector sentences by transforming each sentence of the first plurality of sentences into a corresponding vector of numbers; and generating a second plurality of vector sentences by transforming each sentence of the second plurality of sentences into a corresponding vector of numbers, wherein the plurality of sentence similarity metrics between the corresponding one of the first plurality of sentences and each one of the sentences in the second plurality of sentences is calculated by the corresponding vector of numbers of each sentence. ### Claim 3 (depends on 2) The computer system of claim 2, wherein the plurality of sentence similarity metrics are calculated by using a Cosine similarity metric. ### Claim 4 (depends on 2) The computer system of claim 2, wherein the first and second plurality of vector sentences are generated by applying the first and second plurality of sentences to ROBERTa. ### Claim 5 (depends on 1) The computer system of claim 1, wherein the operations further comprise: ranking the plurality of sentence similarity metrics from most similar to least similar, wherein calculation of the metric between the corresponding one of the first plurality of sentences and the at least one topic is performed by averaging the multiple ones of the calculated plurality of sentence similarity metrics. ### Claim 6 (depends on 5) The computer system of claim 5, wherein the multiple ones of the calculated plurality of sentence similarity metrics is at least a top three of the ranked plurality of sentence similarity metrics. ### Claim 7 (depends on 1) The computer system of claim 1, wherein the operations further comprise: comparing a threshold to the calculated metric between the corresponding one of the first plurality of sentences and the at least one topic; and assigning a positive indicator for the at least one topic to the corresponding one of the first plurality of sentences based on the calculated metric meeting or exceeding the threshold. ### Claim 8 (depends on 1) The computer system of claim 1, wherein the operations further comprise: generating a report that includes each one of the first plurality of sentences and an indicator for whether or not each one of the at least one topics is determined to be present within each corresponding one of the first plurality of sentences. ### Claim 9 (depends on 1) The computer system of claim 1, wherein the operations further comprise: performing, for each one of the first plurality of sentences, a sentiment analysis process; and assigning a sentiment value to each of the first plurality of sentences. ### Claim 10 (depends on 1) The computer system of claim 1, wherein the operations further comprise: obtaining, by using a network graph, an initial dataset of keyword suggestions that is based on one of the keywords of the plurality of specified keywords for the at least one topic; performing a first filtering process to remove a first subset of keywords from the initial dataset of keyword suggestions to thereby obtain an intermediate dataset of suggested keywords; and performing a second filtering process, which is different from the first filtering process, on the intermediate dataset of suggested keywords to thereby obtain a dataset of relevant keyword suggestions for the one of the keywords of the plurality of specified keywords. ### Claim 11 (depends on 10) The computer system of claim 10, wherein the operations further comprise: adding at least one of the keywords from the dataset of relevant keyword suggestions to the plurality of specified keywords for the at least one topic. ### Claim 12 A method of performing natural language processing on a computer system, the method comprising: obtaining target textual data; obtaining input data that includes at least one topic with a plurality of specified keywords for the at least one topic; for each corresponding one of the plurality of specified keywords for the at least one topic, performing a query to retrieve an expanded body of text that is associated with the corresponding one of the plurality of specified keywords, wherein the expanded body of text includes multiple different sentences; generating a first plurality of sentences from the target textual data; generating a second plurality of sentences that is based on each of the expanded text bodies retrieved in connection with each corresponding keyword; for each corresponding one of the first plurality of sentences, calculating a plurality of sentence similarity metrics between the corresponding one of the first plurality of sentences and each one of the sentences in the second plurality of sentences; and for each corresponding one of the first plurality of sentences, calculating a metric between the corresponding one of the first plurality of sentences and the at least one topic based on multiple ones of the calculated plurality of sentence similarity metrics for the corresponding one of the first plurality of sentences. ### Claim 13 (depends on 12) The method of claim 12, further comprising: generating a first plurality of vector sentences by transforming each sentence of the first plurality of sentences into a corresponding vector of numbers; and generating a second plurality of vector sentences by transforming each sentence of the second plurality of sentences into a corresponding vector of numbers, wherein the plurality of sentence similarity metrics between the corresponding one of the first plurality of sentences and each one of the sentences in the second plurality of sentences is calculated by the corresponding vector of numbers of each sentence. ### Claim 14 (depends on 13) The method of claim 13, wherein the plurality of sentence similarity metrics are calculated by using a Cosine similarity metric. ### Claim 15 (depends on 13) The method of claim 13, wherein the first and second plurality of vector sentences are generated by applying the first and second plurality of sentences to ROBERTa. ### Claim 16 (depends on 12) The method of claim 12, further comprising: ranking the plurality of sentence similarity metrics from most similar to least similar, wherein calculation of the metric between the corresponding one of the first plurality of sentences and the at least one topic is performed by averaging the multiple ones of the calculated plurality of sentence similarity metrics. ### Claim 17 (depends on 16) The method of claim 16, wherein the multiple ones of the calculated plurality of sentence similarity metrics is at least a top three of the ranked plurality of sentence similarity metrics. ### Claim 18 A non-transitory computer readable storage medium storing instructions for use with a computer system, the computer system including at least one hardware processor, the stored instructions comprising instructions that are configured to cause at least on hardware processor to perform operations comprising: obtaining target textual data; obtaining input data that includes at least one topic with a plurality of specified keywords for the at least one topic; for each corresponding one of the plurality of specified keywords for the at least one topic, performing a query to retrieve an expanded body of text that is associated with the corresponding one of the plurality of specified keywords, wherein the expanded body of text includes multiple different sentences; generating a first plurality of sentences from the target textual data; generating a second plurality of sentences that is based on each of the expanded text bodies retrieved in connection with each corresponding keyword; for each corresponding one of the first plurality of sentences, calculating a plurality of sentence similarity metrics between the corresponding one of the first plurality of sentences and each one of the sentences in the second plurality of sentences; and for each corresponding one of the first plurality of sentences, calculating a metric between the corresponding one of the first plurality of sentences and the at least one topic based on multiple ones of the calculated plurality of sentence similarity metrics for the corresponding one of the first plurality of sentences. ### Claim 19 (depends on 18) The non-transitory computer readable storage medium of claim 18, wherein the operations further comprise: generating a first plurality of vector sentences by transforming each sentence of the first plurality of sentences into a corresponding vector of numbers; and generating a second plurality of vector sentences by transforming each sentence of the second plurality of sentences into a corresponding vector of numbers, wherein the plurality of sentence similarity metrics between the corresponding one of the first plurality of sentences and each one of the sentences in the second plurality of sentences is calculated by the corresponding vector of numbers of each sentence. ### Claim 20 (depends on 19) The non-transitory computer readable storage medium of claim 19, wherein the plurality of sentence similarity metrics are calculated by using a Cosine similarity metric.