Accelerated document categorization using machine learning

A computer-implemented method comprising:

• obtaining at least one document to be classified, the at least one document containing textual data;
• classifying, using a machine learning model, the at least one document into at least two classes, the machine learning model including an artificial neural network, ANN, and an attention mechanism and the machine learning model being trained using a training dataset comprising:
• a plurality of documents containing textual data, and
• information indicating which one of the at least two classes each one of the plurality of documents belongs to;
• determining, for each of the at least one document, a confidence value of the classifying, based on one or more outputs of one or more nodes comprised in the ANN, the one or more outputs being obtained from the classifying of the at least one document using the machine learning model,
• wherein the confidence value is determined based on:
• one or more activation levels of the one or more nodes in a final layer of the ANN,
• a maximum predictive probability of one of the at least two classes, and
• a predictive uncertainty output by the machine learning model;
• assigning, to each of the at least one document, based at least in part on the confidence value, one of at least two categories that are associated with different degrees of credibility of the classifying;
• filtering the at least one document to exclude a document of the at least one document from further processing based on a respective confidence value of the document satisfying an upper confidence threshold or a lower confidence threshold;
• and
• selecting one or more of the at least one document for further processing based on the filtering and one of the at least two categories assigned to each of the at least one document,
• wherein each document of the one or more of the at least one document selected for further processing has a confidence value that does not satisfy the upper confidence threshold and does not satisfy the lower confidence threshold
• providing, for display, the one or more documents of the at least one document with attention information that indicates significance, in the classifying, of one or more parts of each of the one or more documents of the at least one document provided for display,
• wherein the one or more parts of each document are provided with at least one word-level indicator to cause display of an indication of a significance of at least one word based on one or more word-level vectors included in the attention information, and
• wherein the one or more parts of each document are provided with at least one sentence-level indicator to cause display of an indication of a significance of at least one sentence based on one or more sentence-level vectors included in the attention information.

A computer-implemented method comprising:

• obtaining at least one document to be classified, the at least one document containing textual data;
• classifying, using a machine learning model, the at least one document into at least two classes, the machine learning model including an artificial neural network, ANN, and an attention mechanism and the machine learning model being trained using a training dataset comprising:
• a plurality of documents containing textual data, and
• information indicating which one of the at least two classes each one of the plurality of documents belongs to;
• identifying, for each of the at least one document, a confidence value associated with the classifying based on:
• one or more activation levels of one or more nodes in a final layer of the ANN,
• a maximum predictive probability of one of the at least two classes, and
• a predictive uncertainty output by the machine learning model;
• filtering the at least one document to exclude a document of the at least one document from further processing based on a respective confidence value of the document satisfying an upper confidence threshold or a lower confidence threshold;
• and
• providing for display, after the filtering and based on corresponding confidence values for one or more documents of the at least one document not satisfying the upper confidence threshold and not satisfying the lower confidence threshold, the one or more documents of the at least one document with attention information that indicates significance, in the classifying, of one or more parts of each of the one or more documents of the at least one document provided for display,
• wherein the one or more parts of each document are provided with at least one word-level indicator to cause display of an indication of a significance of at least one word based on one or more word-level vectors included in the attention information, and
• wherein the one or more parts of each document are provided with at least one sentence-level indicator to cause display of an indication of a significance of at least one sentence based on one or more sentence-level vectors included in the attention information.

A system comprising:

• at least one storage medium configured to store a machine learning model including an artificial neural network, ANN, and an attention mechanism, wherein the machine learning model is trained using a training dataset comprising:
• a plurality of documents containing textual data, and
• information indicating which one of at least two classes each one of the plurality of documents belongs to;
• and
• at least one processor in communication with the storage medium, the processor being configured to:
• obtain at least one document to be classified, the at least one document containing textual data;
• classify, using the machine learning model, the at least one document into the at least two classes;
• determine, for each of the at least one document, a confidence value of the classifying, based on one or more outputs of one or more nodes comprised in the ANN, the one or more outputs being obtained from the classifying of the at least one document using the machine learning model,
• wherein the confidence value is determined based on:
• one or more activation levels of the one or more nodes in a final layer of the ANN,
• a maximum predictive probability of one of the at least two classes, and
• a predictive uncertainty output by the machine learning model;
• assign, to each of the at least one document, based at least in part on the confidence value, one of at least two categories that are associated with different degrees of credibility of the classifying;
• filter the at least one document to exclude a document of the at least one document from further processing based on a respective confidence value of the document satisfying an upper confidence threshold or a lower confidence threshold;
• and
• provide, for output and after the filtering, one or more of the at least one document with:
• the one of the at least two categories assigned to each of the one or more of the at least one document, and
• attention information that indicates significance of one or more parts of each document provided for display in the classifying of the document using the machine learning model,
• wherein each document of the one or more of the at least one document provided for output has a confidence value that does not satisfy the upper confidence threshold and does not satisfy the lower confidence threshold,
• wherein the one or more parts of each document are provided with at least one word-level indicator to cause display of an indication of a significance of at least one word based on one or more word-level vectors included in the attention information, and
• wherein the one or more parts of each document are provided with at least one sentence-level indicator to cause display of an indication of a significance of at least one sentence based on one or more sentence-level vectors included in the attention information.

A non-transitory computer-readable medium comprising instructions that, when executed, configure one or more hardware processors to perform a method comprising:

• obtaining at least one document to be classified, the at least one document containing textual data;
• classifying, using a machine learning model, the at least one document into at least two classes, the machine learning model including an artificial neural network, ANN, and an attention mechanism and the machine learning model being trained using a training dataset comprising:
• a plurality of documents containing textual data, and
• information indicating which one of the at least two classes each one of the plurality of documents belongs to;
• determining, for each of the at least one document, a confidence value of the classifying, based on one or more outputs of one or more nodes comprised in the ANN, the one or more outputs being obtained from the classifying of the at least one document using the machine learning model,
• wherein the confidence value is determined based on:
• one or more activation levels of the one or more nodes in a final layer of the ANN,
• a maximum predictive probability of one of the at least two classes, and
• a predictive uncertainty output by the machine learning model;
• assigning, to each of the at least one document, based at least in part on the confidence value, one of at least two categories that are associated with different degrees of credibility of the classifying;
• filtering the at least one document to exclude a document of the at least one document from further processing based on a respective confidence value of the document satisfying an upper confidence threshold or a lower confidence threshold;
• and
• providing, for output and after the filtering, one or more of the at least one document with:
• the one of the at least two categories assigned to each of the one or more of the at least one document, and
• attention information that indicates significance of one or more parts of each document provided for display in the classifying of the document using the machine learning model,
• wherein each document of the one or more of the at least one document provided for output has a confidence value that does not satisfy the upper confidence threshold and does not satisfy the lower confidence threshold,
• wherein the one or more parts of each document are provided with at least one word-level indicator to cause display of an indication of a significance of at least one word based on one or more word-level vectors included in the attention information, and
• wherein the one or more parts of each document are provided with at least one sentence-level indicator to cause display of an indication of a significance of at least one sentence based on one or more sentence-level vectors included in the attention information.