# US 2019/0171944 A1 · Integrity evaluation of unstructured processes using artificial intelligence (ai) techniques > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0171944 A1 - Document ID: US20190171944A1 - Status: Published application - Kind: A1 - Application number: 15/833,974 - Filed: 2017-12-06 - Published: 2019-06-06 - Earliest claimed: 2017-12-06 - Assignees: Accenture Global Solutions Limited - Applicants: Accenture Global Solutions Limited - Inventors: Mingzhu LU - Web page: https://patentgrep.com/patent/US20190171944A1 ## CPC - G06F 16/313: {Selection or weighting of terms for indexing} - G06F 16/285: {Clustering or classification} - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning - G06N 3/09: Supervised learning - G06N 5/02: Knowledge representation; Symbolic representation - G06N 5/022: Knowledge engineering; Knowledge acquisition - G06N 5/025: Extracting rules from data - G06N 20/00: Machine learning - G06N 20/20: Ensemble learning - G06F 18/24: Classification techniques - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06Q 10/067: Enterprise or organisation modelling - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract A process integrity evaluation system ensures integrity of unstructured processes. The process integrity evaluation system handles structured, semi-structured, and unstructured data at massive and large scale. The system provides scalability, secure storage, indexing, knowledge storage, and visualizations of processes by information retrieval, natural language processing, cloud computing, large scale machine learning, knowledge discovery, and other artificial intelligence techniques. Self-provided data, systematically gathered data, and potentially related data from additional sources are incorporated in the process integrity evaluation system which provides the core capabilities of data integrity checking, entity extraction, entity resolution, entity categorization, entity relationship extraction, processes extraction and reconstruction based on knowledge storage, such as knowledge graphs, inference functions, and evaluation computations. After extracting and reconstructing unstructured processes successfully, machine learning functions compute an integrity assurance score, e.g., a similarity, between extracted documents and the internal records in addition to an evaluation result, which can ensure the integrity of the unstructured processes. ## Claims ### Claim 1 A process integrity evaluation system that evaluates an unstructured process comprising: at least one processor; a non-transitory computer readable medium storing machine-readable instructions that cause the at least one processor to: receive at least one input including data associated with the unstructured process; verify integrity of data obtained from the input, wherein the data is from data sources having data of different data formats and the data is one of historical data, real-time data and live streaming data; extract a plurality of entities and values of the plurality of entities from the at least one input; conduct entity resolution in a resolved format; categorize the plurality of entities into one or more entity categories; determine relationships among entities; store the entities and relationships between the entities to knowledge storage; extract responses by queries from the knowledge storage; extract and reconstruct process steps of the unstructured process from the knowledge storage; compute and obtain an integrity assurance score, the entity values and corresponding values stored in the data sources; determine that the unstructured process is valid upon the integrity assurance score being greater than an assurance score threshold; determine that one or more steps of the unstructured process are inaccurate with primary reasons in specific areas upon the integrity assurance score being less than the assurance score threshold; and initiate remediation when the unstructured process is determined to be inaccurate. ### Claim 2 (depends on 1) The process integrity evaluation system of claim 1, wherein the assurance score threshold is predetermined or optimized through machine learning or trained by models. ### Claim 3 (depends on 1) The process integrity evaluation system of claim 1, wherein the instructions to extract entities further comprise instructions that cause the processor to: parse and tokenize at least one document associated with the input; and recognize the entities, identify the entity values, and conduct entity resolutions. ### Claim 4 (depends on 3) The process integrity evaluation system of claim 3, wherein the instructions to extract the entities further comprise instructions that cause the processor to: employ artificial intelligence techniques that identify the entities and the entity values from an output obtained upon the parsing and tokenization of the at least one document. ### Claim 5 (depends on 1) The process integrity evaluation system of claim 1, wherein the instructions to categorize the entities into one or more entity categories further comprise instructions that cause the processor to: categorize the entities into one or more entity categories to differentiate between the one or more entity categories that include at least locations, names, dates, amounts and identifiers. ### Claim 6 (depends on 1) The process integrity evaluation system of claim 1, wherein the instructions for storing entities, entity values, and entity relationships further comprise instructions that cause the processor to: employ tensor factorization for predicting links between the entities. ### Claim 7 (depends on 1) The process integrity evaluation system of claim 1, wherein the instructions to extract and reconstruct process of the unstructured process from the knowledge storage further comprise instructions that cause the processor to: obtain responses to queries from one or more of the plurality of entities. ### Claim 8 (depends on 1) The process integrity evaluation system of claim 1, wherein the instructions to obtain an integrity assurance score for one or more of the values further comprise instructions that cause the processor to: calculate similarity between the entity values obtained from the input and corresponding values stored in the data sources; and compare the similarity to a similarity threshold. ### Claim 9 (depends on 1) The process integrity evaluation system of claim 1, further comprising instructions that cause the processor to: generate a report indicating that the unstructured process is accurate if the integrity assurance score is greater than the assurance score threshold, else, generate a report with the primary failure reasons in specific areas. ### Claim 10 (depends on 1) The process integrity evaluation system of claim 1, wherein the data sources include at least departmental data sources and system data sources. ### Claim 11 (depends on 1) The process integrity evaluation system of claim 1, wherein the data integrity verification comprises multiple sources check, cross-channel consolidation and outlier detection techniques. ### Claim 12 A method of automatically evaluating an unstructured process comprising: accessing inputs that pertain to the unstructured process; processing the inputs to obtain entities and attribute values of the entities; building respective data structures that store for each input, respective entities and respective attribute values; constructing knowledge storage corresponding to each of the inputs, the knowledge storage being constructed based on link predictions between the entities; calculating an integrity assurance score for one or more of the entity values extracted from the inputs, the integrity assurance score based on a similarity of the one or more entity values and corresponding values stored in data sources; comparing the integrity assurance score with an assurance score threshold; and determining that the unstructured process is accurate if the integrity assurance score is greater than the assurance score threshold, else generating a report with primary failure reasons in specific areas. ### Claim 13 (depends on 12) The method of claim 12, wherein the inputs include one or more of structured data, semi-structured data and unstructured data. ### Claim 14 (depends on 12) The method of claim 12, wherein processing the inputs to obtain the entities and values of the entities for each input further comprises: recognizing the entities, to identify the entity values; and conducting entity resolutions. ### Claim 15 (depends on 12) The method of claim 12, wherein constructing knowledge storage including the entities and the attribute values further comprises: constructing and storing the entities, entities values, and entity relationships in the knowledge storage by using artificial intelligence techniques for predicting links between the entities. ### Claim 16 (depends on 12) The method of claim 12, wherein the integrity assurance score calculation comprises employing one or more of document similarity, graph similarity, Jacob similarity matrix, and Cosine similarity matrix. ### Claim 17 A non-transitory device storage medium comprising machine-readable instructions that cause a processor to: receive at least one input including data associated with an unstructured process; verify integrity of data obtained from the input, wherein the data is from data sources having data of different data formats and the data is one of historical data, real-time data and live streaming data; extract a plurality of entities and values of the plurality of entities from the at least one input; conduct entity resolution in a resolved format; categorize the plurality of entities into one or more entity categories; determine relationships among entities; store the entities and relationships between the entities to knowledge storage; extract responses by queries from the knowledge storage; extract and reconstruct process steps of the unstructured process from the knowledge storage; compute and obtain an integrity assurance score, the entity values and corresponding values stored in the data sources; determine that the unstructured process is valid upon the integrity assurance score being greater than an assurance score threshold; determine that one or more steps of the unstructured process are inaccurate upon the integrity assurance score being less than the assurance score threshold with primary reasons in specific areas; and initiate remediation when the unstructured process is determined to be inaccurate. ### Claim 18 (depends on 17) The non-transitory device storage medium of claim 17, wherein the instructions for extracting the entities and the values of the entities further comprising machine-readable instructions that cause the processor to: parse and tokenize at least one document associated with the input; and recognize the entities, identify the entity values, and conduct entity resolutions. ### Claim 19 (depends on 17) The non-transitory device storage medium of claim 17, wherein the instructions for constructing knowledge storage including the entities and the entity values further comprising machine-readable instructions that cause the processor to: construct and store the entities, entities values, and the entity relationships in the knowledge storage by using artificial intelligence techniques for predicting links between the entities. ### Claim 20 (depends on 17) The non-transitory computer-readable storage medium of claim 17, wherein the instructions for obtaining the integrity assurance score further comprises instructions that cause the processor to: employ one or more of document similarity, graph similarity, Jacob similarity matrix, and Cosine similarity matrix to calculate the integrity assurance score.