# US 2022/0035993 A1 · Automated document highlighting in a digital management platform > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0035993 A1 - Document ID: US20220035993A1 - Status: Published application - Kind: A1 - Application number: 16/942,673 - Filed: 2020-07-29 - Published: 2022-02-03 - Earliest claimed: 2020-07-29 - Assignees: DocuSign, Inc. - Applicants: DocuSign, Inc. - Inventors: Mangesh Bhandarkar; Shrinivas Kiran Kaza; Taiwo Raphael Alabi; Ashwath Mohan; Kim Cuong Phung - Web page: https://patentgrep.com/patent/US20220035993A1 ## CPC - G06F 40/166: Editing, e.g. inserting or deleting - G06F 40/169: Annotation, e.g. comment data or footnotes - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/40: Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor - G06F 40/106: Display of layout of documents; Previewing - G06F 40/216: using statistical methods - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06F 40/295: Named entity recognition - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G06N 20/20: Ensemble learning - G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text - G06N 3/045: Combinations of networks - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A highlighting engine modifies a target document by identifying and highlighting a set of text passages. The highlighting engine receives a training set of data including documents that each include a set of highlighted text passages. The highlighting engine trains a machine learned model using the training set of data. The trained machine learned model, when applied to subsequent identified candidate sets of text passages within the target document, identifies the set of text passages to highlight. The highlighting engine modifies the target document with the highlighted set of text passages and provides the modified target document for display via an interface. The highlighted set of text passages enable a user to quickly read and understand the target document. ## Claims ### Claim 1 A method of automatically highlighting text passages within a target document comprising: accessing a training set of data comprising documents each with a set of highlighted text passages; training a machine learned model using the training set of data, the machine learned model configured to identify a set of text passages of a document to highlight; identifying a first candidate set of text passages of the target document by: converting text of the target document into word vectors, generating a matrix representation of the target document, each row of the matrix representation corresponding to a text passage of the target document, the values of each row corresponding to the word vectors, and identifying a threshold portion of the text passages of the target document that are most relevant to the target document using an unweighted graph generated based on the generated matrix representation; identifying a second candidate set of text passages within the target document based on characteristics of text of the second candidate set of text passages; identifying a third candidate set of text passages within the target document based on feedback from a user; identifying a target set of text passages to highlight by applying the machine learned model to the first candidate set of text passages, the second candidate set of text passages, and the third candidate set of text passages; and modifying the target document by highlighting the target set of text passages. ### Claim 2 (depends on 1) The method of claim 1, further comprising: in response to accessing the target document, identifying characters within the target document and a character location for each identified character in the target document, and wherein modifying the target document by highlighting the target set of text passages comprises highlighting the characters of the target set of text passages at the character locations corresponding to the characters. ### Claim 3 (depends on 2) The method of claim 2, wherein identifying characters of the target document and the character location for each identified character comprises assigning a sentence location value for every character in a sentence, wherein the sentence location value is unique for each sentence, and wherein highlighting the target set of text passages is based on the corresponding sentence location values. ### Claim 4 (depends on 1) The method of claim 1, further comprising: determining a document type of the target document by applying a second machine learned model to the target document, the second machine learned model configured to identify the document type of the target document based on content of the target document, wherein one or more of the first candidate set of text passages, the second candidate set of text passages, and the third candidate set of text passages are identified based additionally on the determined document type of the target document. ### Claim 5 (depends on 1) The method of claim 1, wherein the values of each row corresponding to the word vectors include an average word vector value of the word vectors in each text passage of the target document. ### Claim 6 (depends on 1) The method of claim 1, wherein applying the machine learned model to the first candidate set of text passages, the second candidate set of text passages, and the third candidate set of text passages comprises performing an information extraction operation on the first candidate set of text passages, the second candidate set of text passages, and the third candidate set of text passage to identify words corresponding to one or more of a name, an organization, and a date, and wherein the target set of text passages are identified based at least in part on the identified words. ### Claim 7 (depends on 1) The method of claim 1, further comprising: comparing the text of the target document to a catalog of previously identified sets of text passages; identifying a fourth candidate set of text passages within the target document based on the comparison; and identifying the target set of text passages to highlight by additionally applying the machine learned model to the fourth candidate set of text passages. ### Claim 8 (depends on 7) The method of claim 7, wherein the previously identified sets of text passages include text passages identified by the user. ### Claim 9 (depends on 7) The method of claim 7, wherein the previously identified sets of text passages include text passages within other documents accessible to, created by, or signed by the user. ### Claim 10 (depends on 1) The method of claim 1, wherein the characteristics of text of the second candidate set of text passages include one of: a font, a size, a position, or a typographical emphasis. ### Claim 11 (depends on 1) The method of claim 1, wherein the target document is accessed by: receiving the target document to be highlighted from a user of a document management system; generating the target document to be highlighted within the document management system; accessing the target document to be highlighted from storage of the document management system; or accessing the target document to be highlighted from a document system different than the document management system. ### Claim 12 A non-transitory computer readable storage medium comprising computer executable instructions for automatically highlighting text passages within a target document that when executed by one or more processors causes the one or more processors to perform operations comprising: accessing a training set of data comprising documents each with a set of highlighted text passages; training a machine learned model using the training set of data, the machine learned model configured to identify a set of text passages of a document to highlight; identifying a first candidate set of text passages of the target document by: converting text of the target document into word vectors, generating a matrix representation of the target document, each row of the matrix representation corresponding to a text passage of the target document, the values of each row corresponding to the word vectors, identifying a threshold portion of the text passages of the target document that are most relevant to the target document using an unweighted graph generated based on the generated matrix representation; identifying a second candidate set of text passages within the target document based on characteristics of text of the second candidate set of text passages; identifying a third candidate set of text passages within the target document based on feedback from a user; identifying a target set of text passages to highlight by applying the machine learned model to the first candidate set of text passages, the second candidate set of text passages, and the third candidate set of text passages; and modifying the target document by highlighting the target set of text passages. ### Claim 13 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the instructions further cause the processors to perform operations comprising: in response to accessing the target document, identifying characters within the target document and a character location for each identified character in the target document, and wherein modifying the target document by highlighting the target set of text passages comprises highlighting the characters of the target set of text passages at the character locations corresponding to the characters. ### Claim 14 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the instructions further cause the processors to perform operations comprising: determining a document type of the target document by applying a second machine learned model to the target document, the second machine learned model configured to identify the document type of the target document based on content of the target document, wherein one or more of the first candidate set of text passages, the second candidate set of text passages, and the third candidate set of text passages are identified based additionally on the determined document type of the target document. ### Claim 15 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein applying the machine learned model to the first candidate set of text passages, the second candidate set of text passages, and the third candidate set of text passages comprises performing an information extraction operation on the first candidate set of text passages, the second candidate set of text passages, and the third candidate set of text passage to identify words corresponding to one or more of a name, an organization, and a date, and wherein the target set of text passages are identified based at least in part on the identified words. ### Claim 16 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the instructions further cause the processors to perform operations comprising: comparing the text of the target document to a catalog of previously identified sets of text passages; identifying a fourth candidate set of text passages within the target document based on the comparison; and identifying the target set of text passages to highlight by additionally applying the machine learned model to the fourth candidate set of text passages. ### Claim 17 (depends on 16) The non-transitory computer readable storage medium of claim 16, wherein the previously identified sets of text passages include text passages identified by the user. ### Claim 18 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the target document is accessed by: receiving the target document to be highlighted from a user of a document management system; generating the target document to be highlighted within the document management system; accessing the target document to be highlighted from storage of the document management system; or accessing the target document to be highlighted from a document system different than the document management system. ### Claim 19 A computer system for automatically highlighting text passages within a target document comprising: one or more computer processors; and a non-transitory computer readable storage medium comprising computer executable instructions that when executed by one or more processors causes the one or more processors to perform operations comprising: accessing a training set of data comprising documents each with a set of highlighted text passages; training a machine learned model using the training set of data, the machine learned model configured to identify a set of text passages of a document to highlight; identifying a first candidate set of text passages of the target document by: converting text of the target document into word vectors, generating a matrix representation of the target document, each row of the matrix representation corresponding to a text passage of the target document, the values of each row corresponding to the word vectors, and identifying a threshold portion of the text passages of the target document that are most relevant to the target document using an unweighted graph generated based on the generated matrix representation; identifying a second candidate set of text passages within the target document based on characteristics of text of the second candidate set of text passages; identifying a third candidate set of text passages within the target document based on feedback from a user; identifying a target set of text passages to highlight by applying the machine learned model to the first candidate set of text passages, the second candidate set of text passages, and the third candidate set of text passages; and modifying the target document by highlighting the target set of text passages. ### Claim 20 A method of automatically highlighting text passages within a target document comprising: identifying a first candidate set of text passages of the target document by: converting text of the target document into word vectors, generating a matrix representation of the target document, each row of the matrix representation corresponding to a text passage of the target document, the values of each row corresponding to the word vectors, and identifying a threshold portion of the text passages of the target document that are most relevant to the target document using an unweighted graph generated based on the generated matrix representation; identifying a second candidate set of text passages within the target document based on characteristics of text of the second candidate set of text passages; identifying a third candidate set of text passages within the target document based on feedback from a user; applying a machine learned model trained on training documents with pre-highlighted passages to the first candidate set of text passages, the second candidate set of text passages, and the third candidate set of text passages to identify a target set of text passages to highlight; and modifying the target document by highlighting the target set of text passages.