# US 2021/0081608 A1 · Enabling Flexible Processing of Semantically-Annotated Documents

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0081608 A1
- Document ID: US20210081608A1
- Status: Published application
- Kind: A1
- Application number: 16/986,151
- Filed: 2020-08-05
- Published: 2021-03-18
- Earliest claimed: 2019-09-16
- Assignees: Docugami, Inc.
- Applicants: Docugami, Inc.
- Inventors: Andrew Begun; Steven DeRose; Taqi Jaffri; Luis Marti Orosa; Michael Palmer; Jean Paoli; Christina Pavlopoulou; Elena Pricoiu; Swagatika Sarangi; Marcin Sawicki; Manar Shehadeh; Michael Taron; Bhaven Toprani; Zubin Rustom Wadia; David Watson; Eric White; Joshua Yongshin Fan; Kush Gupta; Andrew Minh Hoang; Zhanlin Liu; Jerome George Paliakkara; Zhaofeng Wu; Yue Zhang; Xiaoquan Zhou
- Web page: https://patentgrep.com/patent/US20210081608A1

## CPC

- G06F 40/186: Templates
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06F 16/2457: with adaptation to user needs
- G06F 16/248: Presentation of query results
- G06F 16/93: Document management systems
- G06F 40/106: Display of layout of documents; Previewing
- G06F 40/117: Tagging; Marking up (details of markup languages G06F40/143); Designating a block; Setting of attributes (style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154)
- G06F 40/131: Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces
- G06F 40/169: Annotation, e.g. comment data or footnotes
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06F 40/295: Named entity recognition
- G06F 40/30: Semantic analysis
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/091: Active learning
- G06N 20/00: Machine learning
- G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
- G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
- G06V 30/418: Document matching, e.g. of document images
- G06F 40/205: Parsing
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Machine learning, artificial intelligence, and other computer-implemented methods are used to identify various semantically important chunks in documents, automatically label them with appropriate datatypes and semantic roles, and use this enhanced information to assist authors and to support downstream processes. Chunk locations, datatypes, and semantic roles can often be automatically determined from what is here called “context”, to wit, the combination of their formatting, structure, and content; those of adjacent or nearby content; overall patterns of occurrence in a document, and similarities of all these things across documents (mainly but not exclusively among documents in the same document set). Similarity is not limited to exact or fuzzy string or property comparisons, but may include similarity of natural language grammatical structure, ML (machine learning) techniques such as measuring similarity of word, chunk, and other embeddings, and the datatypes and semantic roles of previously-identified chunks.

## Claims

### Claim 1

A method implemented on a computer system executing instructions for assisting users to review a set of documents, the method comprising:
accessing a document set that contains a plurality of documents, wherein the document set also identifies chunks within the individual documents of the document set and also includes datatypes and semantic role labels for some of the chunks, wherein the semantic role labels are descriptive of the semantic roles played by the chunks within their respective documents;
deriving patterns (a) of occurrence of semantic roles played by chunks across the documents in the document set; and (b) of occurrence of counterpart chunks in different documents across the document set, wherein counterpart chunks play a same semantic role in different documents; and
based on the derived patterns of occurrences across the document set, automatically developing information about the content in one or more documents in the document set; and making the information available for use in a downstream process.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein deriving the patterns of occurrences comprises using machine learning and/or artificial intelligence to derive the patterns of occurrences.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the chunks in the document set comprise:
field chunks that contain content within the documents suitable for use as fields in document templates, wherein some of the field chunks are hierarchical and contain other chunks as sub-chunks;
structural chunks that contain content from structures within the layout of the documents, and the semantic role labels include semantic role labels for some of the structural chunks; and
chunks that contain images or video.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the information comprises content extracted from an individual document from the document set.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein the extracted content comprises one or more excerpts of chunks from the individual document that are counterparts to chunks in other documents.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the information comprises an indication of an occurrence or non-occurrence of a specific chunk in an individual document, and the specific chunk is a counterpart to chunks occurring in other documents from the document set.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the information comprises a summary of the individual document.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the information comprises content extracted from multiple documents in the document set.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the information comprises chunks extracted from multiple documents in the document set, and the information is organized according to which chunks are counterparts.

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, wherein the information comprises indications of anomalies in occurrences of counterpart chunks across the documents in the document set.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein the anomalies comprise absence of counterpart chunks in individual documents, and the information is made available in a format that facilitates navigation to the individual documents in which the counterpart chunks are absent.

### Claim 12 (depends on 10)

The computer-implemented method of claim 10, wherein the anomalies comprise absence of counterpart chunks in individual documents, and the information is made available in a format that summarizes the absence of counterpart chunks in the individual documents.

### Claim 13 (depends on 8)

The computer-implemented method of claim 8, wherein the downstream process is implemented by a software application, and the information is made available in a format suitable for use by the software application.

### Claim 14 (depends on 13)

The computer-implemented method of claim 13 wherein the information further comprises:
a description of one or more processes executable by the software application for retrieving the content extracted from the multiple documents in the document set.

### Claim 15 (depends on 8)

The computer-implemented method of claim 8, wherein the downstream process comprises verifying compliance of the content in chunks that play semantic roles for which the documents are subject to predefined requirements or policies.

### Claim 16 (depends on 1)

The computer-implemented method of claim 1, wherein the downstream process comprises generating a report in a format understandable by humans.

### Claim 17 (depends on 16)

The computer-implemented method of claim 16, further comprising:
receiving a user selection of a chunk in one of the documents in the document set;
wherein, in response to the user selection, the report comprises occurrence or non-occurrence of counterpart chunks for the user-selected chunk.

### Claim 18 (depends on 17)

The computer-implemented method of claim 17, further comprising:
in response to the report missing some counterpart chunks, receiving a user selection of one of the missing counterpart chunks and, in response to the user selection, updating the report to add the missing counterpart chunks.

### Claim 19

A non-transitory computer-readable storage medium storing executable computer program instructions for assisting users to review a set of documents, the instructions executable by a computer system and causing the computer system to perform a method comprising:
accessing a document set that contains a plurality of documents, wherein the document set also identifies chunks within the individual documents of the document set and also includes datatypes and semantic role labels for some of the chunks, wherein the semantic role labels are descriptive of the semantic roles played by the chunks within their respective documents;
deriving patterns (a) of occurrence of semantic roles played by chunks across the documents in the document set; and (b) of occurrence of counterpart chunks in different documents across the document set, wherein counterpart chunks play a same semantic role in different documents;
based on the derived patterns of occurrences across the document set, automatically developing information about the content in one or more documents in the document set; and making the information available for use in a downstream process.

### Claim 20

A computer system for assisting users to review a set of documents, the computer system comprising:
a storage medium for receiving and storing a document set that contains a plurality of documents, wherein the document set also identifies chunks within the individual documents of the document set and also includes datatypes and semantic role labels for some of the chunks, wherein the semantic role labels are descriptive of the semantic roles played by the chunks within their respective documents; and
a processor system having access to the storage medium and executing an application program for assisting users to review a set of documents, wherein the processor system executing the application program:
derives patterns (a) of occurrence of semantic roles played by chunks across the documents in the document set; and (b) of occurrence of counterpart chunks in different documents across the document set, wherein counterpart chunks play a same semantic role in different documents; and
based on the derived patterns of occurrences across the document set, automatically develops information about the content in one or more documents in the document set; and makes the information available for use in a downstream process.
