Natural language processing and text analytics for audit testing with documentation prioritization and selection

A computer-implemented method comprising:

• receiving, by one or more processors, a plurality of documents corresponding to an audit request, the plurality of documents comprising a first set of one or more documents of a first file type and a second set of one or more documents of a second file type;
• applying, by the one or more processors, a plurality of text extraction and conversion techniques to the first and second sets of documents to generate a plurality of electronic documents;
• generating, by the one or more processors, a dataset comprising the plurality of electronic documents and labels assigned to a subset of the plurality of electronic documents, each label associated with at least one of a plurality of classifications corresponding to the audit request;
• training, by the one or more processors, based on supervised learning, using the dataset including the labels assigned to the subset of the plurality of electronic documents, a first machine-learning model to produce feature vectors comprising word embeddings from the electronic documents, the first machine-learning model configured to:
• map each word of a first plurality of words in the subset based on a predetermined audit word to generate a first plurality of feature vectors;
• and
• generate, from the first plurality of feature vectors, a respective similarity score for each word in the first plurality of words in the subset, each similarity score indicating a similarity of each word with a predetermined audit word;
• applying, by the one or more processors, the first machine-learning model to the dataset to generate a second plurality of feature vectors for a second plurality of words in the plurality of electronic documents;
• applying, by the one or more processors, a second machine-learning model to determine document weights of each electronic document of the plurality of electronic documents, the second machine-learning model configured to:
• obtain a respective document rank value for each of the plurality of electronic documents, each document rank value determined based on a number of associations between pairs of the plurality of electronic documents;
• receive the second plurality of feature vectors and their corresponding document rank values to assign a respective first-stage document weight to each electronic document;
• and
• receive, for each electronic document (i) the respective first-stage document weight and (ii) a respective third feature vector for each word within each electronic document a to assign a second-stage document weight to each electronic document based on the respective third feature vector and the respective first-stage document weight;
• and
• applying, by the one or more processors, the second machine-learning model to the second plurality of feature vectors and the respective document rank values to assign a third-stage document weight to each electronic document;
• and
• determining, by the one or more processors, based on each third-stage document weight and the audit request, a respective importance score indicating an audit likelihood for each of the plurality of documents.

A non-transitory computer-readable storage medium storing one or more programs that are configured to be executed by one or more processors of a computer system, the one or more programs including instructions for:

• receiving a plurality of documents corresponding to an audit request, the plurality of documents comprising a first set of one or more documents of a first file type and a second set of one or more documents of a second file type;
• applying a plurality of text extraction and conversion techniques to the first and second sets of documents to generate a plurality of electronic documents;
• generating a dataset comprising the plurality of electronic documents and labels assigned to a subset of the plurality of electronic documents, each label associated with at least one of a plurality of classifications corresponding to the audit request;
• training, based on supervised learning, using the dataset including the labels assigned to the subset of the plurality of electronic documents, a first machine-learning model to produce feature vectors comprising word embeddings from the electronic documents, the first machine-learning model configured to:
• map each word of a first plurality of words in the subset based on a predetermined audit word to generate a first plurality of feature vectors;
• and
• generate, from the first plurality of feature vectors, a respective similarity score for each word in the first plurality of words in the subset, each similarity score indicating a similarity of each word with a predetermined audit word;
• applying the first machine-learning model to the dataset to generate a second plurality of feature vectors for a second plurality of words in the plurality of electronic documents;
• applying a second machine-learning model to determine respective document weights of each electronic document of the plurality of electronic documents, the second machine-learning model configured to:
• obtain a respective document rank value for each of the plurality of electronic documents, each document rank value determined based on a number of associations between pairs of the plurality of electronic documents;
• receive the second plurality of feature vectors and their corresponding document rank values to assign a respective first-stage document weight to each electronic document;
• and
• receive, for each electronic document (i) the respective first-stage document weight and (ii) a respective third feature vector for each word within each electronic document to assign a second-stage document weight to each electronic document based on the respective third feature vector and the respective first-stage document weight;
• and
• applying the second machine-learning model to the second plurality of feature vectors and the respective document rank values to assign a third-stage document weight to each electronic document;
• and
• determining, based on each third-stage document weight and the audit request, a respective importance score indicating an audit likelihood for each of the plurality of documents.