Back to results
US 2025/0094477 A1Published application
Natural language inferencing and deep learning-based grouping of unlabeled text
AssigneeNasdaq, Inc.
InventorsHyunsoo JEONG, Yongheng ZHANG, Yu-Hsuan JOU
Life of this patent
Estimating the term…
Abstract
Systems and methods that provide improved (e.g., higher accuracy and time-efficient) aggregation of report data are described. Example embodiments effectively combine machine learning and deep learning in a processing pipeline to classify unlabeled source data to determine a reliable classification, or a labeling, of that source data. The classified or labeled source data can be used to generate reports that provide extensive insight into the aggregated source data.
Claims18
A system comprising:
- at least one memory comprising:
- a database of reports, each report comprising a plurality of report sections, wherein each report section of the plurality of report sections comprises at least one question and a corresponding response;
- a machine-learned model;
- and
- a deep-learning model;
- and
- at least one processor configured to perform operations comprising:
- receiving as input a plurality of predetermined report section titles and a list of questions extracted from the database of reports;
- generating, using the machine-learned model and the received input, an intermediate set of labeled questions by:
- assigning, to each question in the list of questions, a respective probability score associated with each title in the plurality of predetermined report section titles;
- and
- selecting, for each question in the list of questions, an associated predetermined title based on the assigned probability scores, wherein each labeled question in the intermediate set of labeled questions comprises a question from the list of questions and the selected predetermined title associated with the question;
- selecting at least a portion of the generated intermediate set of labeled questions to be provided as input to the deep learning model;
- using the selected at least portion of the generated intermediate set of labeled questions as input to the deep learning model,
- repeatedly processing respective labeled questions in the deep learning model until an accuracy threshold is satisfied for the respective labeled questions;
- generating a final set of labeled questions, wherein each labeled question in the final set satisfies the accuracy threshold;
- and
- outputting aggregated labeled questions of the final set of labeled questions based on the predetermined report section titles.
15 more elements
A method comprising:
- receiving as input, by at least one processor, a plurality of predetermined report section titles and a list of questions extracted from a database of reports stored in at least one memory, wherein in the database of reports, each report comprising a plurality of report sections, wherein each report section of the plurality of report sections comprises at least one question and a corresponding response;
- generating, by the at least one processor and using a machine-learned model and the received input, an intermediate set of labeled questions by:
- assigning, to each question in the list of questions, a respective probability score associated with each title in the plurality of predetermined report section titles;
- and
- selecting, for each question in the list of questions, an associated predetermined title based on the assigned probability scores, wherein each labeled question in the intermediate set of labeled questions comprises a question from the list of questions and the selected predetermined title associated with the question;
- selecting, by the at least one processor, at least a portion of the generated intermediate set of labeled questions to be provided as input to the deep learning model;
- using the selected at least portion of the generated intermediate set of labeled questions as input to a deep learning model, repeatedly processing respective labeled questions by the at least one processor in the deep learning model until an accuracy threshold is satisfied for the respective labeled questions;
- generating a final set of labeled questions by the at least one processor, wherein each labeled question in the final set satisfies the accuracy threshold;
- and
- outputting, by the at least one processor, aggregated labeled questions of the final set of labeled questions based on the predetermined report section titles.
7 more elements
A non-transitory computer readable storage medium storing instructions that, when executed by at least one processor of a computer system, causes the computer system to perform operations comprising:
- receiving as input a plurality of predetermined report section titles and a list of questions extracted from a database of reports stored in a memory, wherein in the database of reports, each report comprising a plurality of report sections, wherein each report section of the plurality of report sections comprises at least one question and a corresponding response;
- generating, using a machine-learned model and the received input, an intermediate set of labeled questions by:
- assigning, to each question in the list of questions, a respective probability score associated with each title in the plurality of predetermined report section titles;
- and
- selecting, for each question in the list of questions, an associated predetermined title based on the assigned probability scores, wherein each labeled question in the intermediate set of labeled questions comprises a question from the list of questions and the selected predetermined title associated with the question;
- selecting at least a portion of the generated intermediate set of labeled questions to be provided as input to the deep learning model;
- using the selected at least portion of the generated intermediate set of labeled questions as input to a deep learning model, repeatedly processing respective labeled questions in the deep learning model until an accuracy threshold is satisfied for the respective labeled questions;
- generating a final set of labeled questions, wherein each labeled question in the final set satisfies the accuracy threshold;
- and
- outputting aggregated labeled questions of the final set of labeled questions based on the predetermined report section titles.
7 more elements
Dependencies are inferred from the claim text.