# US 2025/0148209 A1 · IMPROVING ACCURACY OF Gen. AI DRIVEN DOCUMENT ANALYSIS

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148209 A1
- Document ID: US20250148209A1
- Status: Published application
- Kind: A1
- Application number: 18/604,219
- Filed: 2024-03-13
- Published: 2025-05-08
- Earliest claimed: 2023-11-08
- Assignees: Accenture Global Solutions Limited
- Applicants: Accenture Global Solutions Limited
- Inventors: Suraj Govind JADHAV; Ashwin RAMACHANDRAN; Krishna KUMMAMURU; Siddharth DAWAR; Manoj SHROFF
- Web page: https://patentgrep.com/patent/US20250148209A1

## CPC

- G06F 40/295: Named entity recognition
- G06F 40/247: Thesauruses; Synonyms
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 40/30: Semantic analysis
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An Artificial Intelligence (AI) & Generative AI-driven cross-domain document analysis system enables accurate and consistent narratives across a longitudinal timeline for an entity regarding communications in different operational aspects. The document analysis and insight system includes an Artificial Intelligence (AI) powered Search Interface (AIPS) and an Advanced Intelligent Knowledge Engine (AIKE). The AIPS is configured to pre-process documents from structured and unstructured data sources to generate data taxonomies and custom synonym files. The AIKE generates a preliminary evaluation of the various Large Language Models (LLMs) and uses the data taxonomies and custom synonym files to generate prompts that are configured to address limitations of the various LLMs to obtain accurate replies to user requirements.

## Claims

### Claim 1

An Artificial Intelligence (AI) based document analysis system, comprising:
at least one hardware processor; and
at least one non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to:
generate via pre-processing structured and unstructured data sources, a data taxonomy including entities extracted from the structured and unstructured data sources and a custom synonym file that lists domain-specific synonyms extracted from the structured and unstructured data sources;
select, from a plurality of Large Language Models (LLMs) one or more LLMs for generating responses to user queries, wherein the one or more LLMs are selected by an Artificial Intelligence (AI)-based model tester;
determine that a response to a received user requirement is to be generated;
retrieve using the data taxonomy, relevant documents for the received user requirement from one or more of the structured and unstructured data sources;
generate a prompt to the one or more LLMs from the relevant documents based at least on the received user requirement, wherein the prompt is generated to include less than a predetermined number of tokens arranged in complete sentences;
obtain a response to the prompt from each of the one or more LLMs;
receive user feedback to each of the responses from the one or more LLMs provided as replies to the received user requirement; and
further train the AI-based model tester on the user feedback for LLM selection.

### Claim 2 (depends on 1)

The AI-based document analysis system of claim 1, wherein the non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to further:
store in a cache, one of the plurality of responses receiving highest user approval as a reply to the received user requirement along with the received user requirement and a subset of the structured and unstructured data sources used for generating the prompt.

### Claim 3 (depends on 1)

The AI-based document analysis system of claim 1, wherein to generate the data taxonomy, the non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to further:
access text data from the structured and unstructured data sources; and
generate paragraph chunks from the text data.

### Claim 4 (depends on 3)

The AI-based document analysis system of claim 3, wherein to generate the data taxonomy, the non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to further:
extract entities and relationships between the entities from the paragraph chunks;
build knowledge graphs representing the entities and relationships between the entities; and
for each paragraph chunk:
seed the data taxonomy with facets and filters that are represented as nodes and sub-nodes in the knowledge graphs.

### Claim 5 (depends on 1)

The AI-based document analysis system of claim 1, wherein the non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to further:
train the AI-based model tester for LLM selection on features that include at least, token length, Application Programming Interface (API) wrapper availability, accuracy of responses, bias, false positives, and false negatives.

### Claim 6 (depends on 1)

The AI-based document analysis system of claim 1, wherein to generate the prompt, the non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to further:
verify that the response to the received user requirement is not already stored in a cache.

### Claim 7 (depends on 1)

The AI-based document analysis system of claim 1, wherein to generate the prompt, the non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to further:
identify and extract one or more entities from the received user requirement; and
extract required information for response generation from the unstructured and structured data sources with the entities extracted from the received user requirement.

### Claim 8 (depends on 7)

The AI-based document analysis system of claim 7, wherein to obtain the response to the prompt from each of the one or more LLMs the non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to further:
input along with the prompt, the required information extracted from the unstructured and structured data sources to the one or more LLMs.

### Claim 9 (depends on 1)

The AI-based document analysis system of claim 1, wherein to obtain the response to the prompt from each of the one or more LLMs the non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to further:
obtain the response to the prompt with a token length such that a total number of tokens of the prompt and the response are less than the predetermined number of tokens.

### Claim 10 (depends on 1)

The AI-based document analysis system of claim 1, wherein to receive the user feedback to the responses from each of the one or more LLMs the non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to further:
rank the responses from each of the one or more LLMs based at least on similarity with the received user requirement; and
display top k ranked responses for user selection, wherein k is a natural number and k begins with 1 and the top k ranked responses include a primary response from a best performing LLM of the one or more LLMs and other sample responses from other eligible LLMs of the one or more LLMs.

### Claim 11 (depends on 1)

The AI-based document analysis system of claim 1, wherein to further train the AI-based model tester the non-transitory processor-readable medium storing instructions executed by the at least one hardware processor to further:
provide inputs to the AI-based model tester, wherein the inputs carry differential weights favoring Subject Matter Expert (SME) knowledge and the user feedback.

### Claim 12

An Artificial Intelligence (AI) based document analysis method, comprising:
generating via pre-processing structured and unstructured data sources, a data taxonomy including entities extracted from the structured and unstructured data sources and a custom synonym file that lists domain-specific synonyms extracted from the structured and unstructured data sources;
selecting, from a plurality of Large Language Models (LLMs) one or more LLMs that can provide response to user queries, wherein the one or more LLMs are selected by an Artificial Intelligence (AI)-based model tester;
determining that a response to a received user requirement is to be generated;
retrieve using the data taxonomy, relevant paragraphs for the received user requirement from one or more of the structured and unstructured data sources;
generating a prompt to the one or more LLMs from the relevant documents based at least on the received user requirement, wherein the prompt is generated to include less than a predetermined number of tokens arranged in complete sentences;
obtaining a response to the prompt from each of the one or more LLMs;
receiving user feedback to a plurality of the responses from the one or more LLMs provided as replies to the received user requirement; and
further training the AI-based model tester on the user feedback for LLM selection.

### Claim 13 (depends on 12)

The AI-based document analysis method of claim 12, further comprising:
storing in a cache, one of the plurality of responses receiving highest user approval as a reply to the received user requirement along with the received user requirement and a subset of the structured and unstructured data sources used for generating the prompt.

### Claim 14 (depends on 12)

The AI-based document analysis method of claim 12, further comprising:
training the AI based model tester for LLM selection on features that include at least, token length, Application Programming Interface (API) wrapper availability, accuracy of responses, bias, false positives and false negatives.

### Claim 15 (depends on 12)

The AI-based document analysis method of claim 12, wherein generating the prompt further comprises:
extracting one or more entities from the received user requirement; and
extracting required information for response generation from the unstructured and structured data sources with the entities extracted from the received user requirement.

### Claim 16 (depends on 15)

The AI-based document analysis method of claim 15, wherein generating the prompt further comprises:
inputting along with the prompt, the required information extracted from the unstructured and structured data sources to the one or more LLMs.

### Claim 17 (depends on 12)

The AI-based document analysis method of claim 12, wherein further training the AI-based model tester comprises:
providing inputs to the AI-based model tester, wherein the inputs carry differential weights favoring Subject Matter Expert (SME) knowledge and the user feedback.

### Claim 18

A non-transitory processor-readable storage medium comprising machine-readable instructions that cause a processor to:
generate via pre-processing structured and unstructured data sources, a data taxonomy including entities extracted from the structured and unstructured data sources and a custom synonym file that lists domain-specific synonyms extracted from the structured and unstructured data sources;
select, from a plurality of Large Language Models (LLMs) one or more LLMs that can provide response to user queries, wherein the one or more LLMs are selected by an Artificial Intelligence (AI)-based model tester;
determine that a response to a received user requirement is to be generated;
retrieve using the data taxonomy, relevant paragraphs for the received user requirement from one or more of the structured and unstructured data sources;
generate a prompt to the one or more LLMs from the relevant documents based at least one the received user requirement, wherein the at least one prompt includes at least a chunk having less than a predetermined number of tokens arranged in complete sentences;
obtain a response to the prompt from each of the one or more LLMs;
receive user feedback to the responses obtained from the one or more LLMs provided; and
further train the AI-based model tester on the user feedback for LLM selection.

### Claim 19 (depends on 18)

The non-transitory processor-readable storage medium of claim 18, wherein the machine-readable instructions that cause a processor to:
rank the responses obtained from the one or more LLMs based at least on a semantic similarity with the received user requirement; and
provide top k ranked responses as replies to the received user requirement, wherein k is a natural number.

### Claim 20 (depends on 18)

The non-transitory processor-readable storage medium of claim 18, wherein the data taxonomy includes one or more knowledge graphs.
