# US 12,602,376 B2 · Systems and methods for data curation in a document processing system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,376 B2 - Document ID: US12602376B2 - Status: Granted - Kind: B2 - Application number: 17/358,120 - Filed: 2021-06-25 - Published: 2021-12-30 - Granted: 2026-04-14 - Earliest claimed: 2020-06-25 - Assignees: Pryon Incorporated - Applicants: Pryon Incorporated - Inventors: David Nahamoo; Igor Roditis Jablokov; Vaibhava Goel; Etienne Marcheret; Ellen Eide Kislal; Steven John Rennie; Marie Wenzel Meteer; Joseph Allen Pruitt; John Pruitt; Bryan Dempsey; Gregory Pelton; Lucas Siler - Web page: https://patentgrep.com/patent/US12602376B2 ## CPC - G06F 16/24522: {Translation of natural language queries to structured queries} - G06F 16/3328: {using graphical result space presentation or visualisation} - G06F 16/3329: Natural language query formulation - G06F 16/3349: Reuse of stored results of previous queries - G06F 16/335: Filtering based on additional data, e.g. user or group profiles (filtering in web context G06F16/9535, G06F16/9536) - G06F 16/338: Presentation of query results - G06F 16/9032: Query formulation - G06F 16/93: Document management systems - G06F 16/9574: {of access to content, e.g. by caching} - G06F 40/131: Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces - G06F 40/137: Hierarchical processing, e.g. outlines - G06F 40/151: Transformation - G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30) - G06F 40/247: Thesauruses; Synonyms - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06F 40/30: Semantic analysis - G06N 3/0499: Feedforward networks - G06N 3/09: Supervised learning - G06N 3/096: Transfer learning - G06N 5/022: Knowledge engineering; Knowledge acquisition - G06N 5/04: Inference or reasoning models - G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G06N 3/08: Learning methods - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Disclosed are implementations that include a method including obtaining at a local device, a question-and-answer data item representative of a question-and-answer pair relating to source content of a source document, with the question-and-answer data item being associated with question-and-answer history information representative of historical information and access information for the question-and-answer data item, and determining based on the question-and-answer history information whether to perform data refinement operations on the question-and-answer data item. The method further includes selecting, in response to a determination that the question-and-answer data item is to be refined, one of a plurality of users according to one or more matching criteria relating, at least in part, to subject-matter associated with the question-and-answer data item and expertise information respectively associated with the plurality of users, and communicating at least the question-and-answer data item to a destination device associated with the selected one of the plurality of users. ## Claims ### Claim 1 A method comprising: obtaining at a local device, a question-and-answer data item representative of a question-and-answer pair relating to source content of a source document, wherein the question-and-answer data item is associated with answer responsiveness and feedback information; determining from the answer responsiveness and feedback information a level of answer responsiveness of the question-and-answer item to a submitted query and a level of answer satisfaction; and in response to a determination, based on the level of answer responsiveness of the question-and-answer data item and on the level of answer satisfaction, that the question-and-answer data item requires refinement: selecting one of a plurality of users according to one or more matching criteria relating, at least in part, to subject-matter associated with the question-and-answer data item and expertise information respectively associated with the plurality of users; and communicating at least the question-and-answer data item to a destination device associated with the selected one of the plurality of users for editing the at least one question-and-answer data item; receiving, at the local device, a refined question-and-answer data item derived from the question-and-answer data item communicated to the destination device; and storing the refined question-and-answer data item at a question-answer cache maintained at the local device; wherein the answer responsiveness and feedback information comprise one or more of: frequency at which the associated question-and-answer data item has been accessed, level of satisfaction determined based on feedback by one or more users regarding the question-and-answer data item, a confidence level of responsiveness of an answer portion to a question portion of the question-and-answer data item, or authorship information associated with the question-and-answer data item, and wherein the question-and-answer data item comprises one or more of: a pointer to access from a remote device at least a portion of the source document corresponding to an answer to the question-and-answer pair, at least part of the source content, or a summary of the at least the portion of the source content. ### Claim 2 (depends on 1) The method of claim 1, further comprising: deriving the confidence level of the responsiveness of the answer portion to the question portion according to search results obtained from searching a plurality of document objects of transformed content corresponding to one or more source documents, to identify one or more of the plurality of document objects according to question data representative of the question portion of the question-and-answer data item. ### Claim 3 (depends on 2) The method of claim 2, wherein deriving the confidence level comprises: deriving the confidence level of the responsiveness of the answer portion to the question portion based on a number of the identified one or more plurality of document objects, and on closeness level of the question data to respective question portions associated with the identified one or more of the plurality of document objects. ### Claim 4 (depends on 1) The method of claim 1, wherein determining based on the answer responsiveness and feedback information whether to perform refinement operations on the question-and-answer data item comprises: determining whether to refine the question-and-answer data item based on one or more of: the frequency at which the associated question-and-answer data item has been accessed exceeding a frequency threshold; the level of satisfaction determined based on the feedback by the one or more users regarding the question-and-answer data item being below a satisfaction threshold; the confidence level of the responsiveness of the answer portion to the question portion exceeding a responsiveness threshold; or the authorship information matching one or more pre-identified names. ### Claim 5 (depends on 1) The method of claim 1, further comprising: receiving, at the local device, from a remote device query data representative of a subsequent question relating to the source content of the source document; determining whether the query data matches a question portion of the refined question-and-answer data item, stored in the question-answer cache, according to one or more matching criteria; and in response to a determination that the question portion of the refined question-and-answer data item matches the query data received from the remote device, obtaining from the question-answer cache at least one answer portion of the refined question-and-answer data item, and transmitting the at least one answer portion to the remote device. ### Claim 6 (depends on 5) The method of claim 5, further comprising: in response to a determination that the query data does not match the question portion of the refined question-and-answer data item stored at the question-and-answer cache, searching at the local device a documents storage, comprising a plurality of document objects of transformed content corresponding to one or more source documents, to identify one or more of the plurality of document objects according to the query data; and in response to determination, at the local device, of a match between the query data and at least one portion of the transformed content, generating output data based on the matched at least one portion of the transformed content. ### Claim 7 (depends on 6) The method of claim 6, further comprising: storing the plurality of document objects at the document storage, including: receiving one or more source documents at the local device; segmenting the one or more source documents into a plurality of document segments; transforming each of the plurality of document segments into one or more respective transformed segments according to one or more transformations; and arranging the transformed segments into the plurality of document objects. ### Claim 8 A system comprising: a document processing agent implemented at a local processor-based device, coupled to a memory storage device storing data and processor instructions executable on the local processor-based device, configured to: obtain a question-and-answer data item representative of a question-and-answer pair relating to source content of a source document, wherein the question-and-answer data item is associated with answer responsiveness and feedback information; determine from the answer responsiveness and feedback information a level of answer responsiveness of the question-and-answer item to a submitted query and a level of answer satisfaction; and in response to a determination, based on the level of answer responsiveness of the question-and-answer data item and on the level of answer satisfaction, that the question-and-answer data item requires refinement: select one of a plurality of users according to one or more matching criteria relating, at least in part, to subject-matter associated with the question-and-answer data item and expertise information respectively associated with the plurality of users; and a communication unit configured to communicate at least the question-and-answer data item to a destination device associated with the selected one of the plurality of users for editing the at least one question-and-answer data item receive, at the local device, a refined question-and-answer data item derived from the question-and-answer data item communicated to the destination device; and store the refined question-and-answer data item at a question-answer cache maintained at the local device; wherein the answer responsiveness and feedback information comprise one or more of: frequency at which the associated question-and-answer data item has been accessed, level of satisfaction determined based on feedback by one or more users regarding the question-and-answer data item, a confidence level of responsiveness of an answer portion to a question portion of the question-and-answer data item, or authorship information associated with the question-and-answer data item, and wherein the question-and-answer data item comprises one or more of: a pointer to access from a remote device at least a portion of the source document corresponding to an answer to the question-and-answer pair, at least part of the source content, or a summary of the at least the portion of the source content. ### Claim 9 (depends on 8) The system of claim 8, wherein the document processing agent is further configured to: derive the confidence level of the responsiveness of the answer portion to the question portion according to search results obtained from searching a plurality of document objects of transformed content corresponding to one or more source documents, to identify one or more of the plurality of document objects according to question data representative of the question portion of the question-and-answer data item. ### Claim 10 (depends on 9) The system of claim 9, wherein the document processing agent configured to derive the confidence level is configured to: derive the confidence level of the responsiveness of the answer portion to the question portion based on a number of the identified one or more plurality of document objects, and on closeness level of the question data to respective question portions associated with the identified one or more of the plurality of document objects. ### Claim 11 (depends on 8) The system of claim 8, wherein the document processing agent configured to determine based on the answer responsiveness and feedback information whether to perform refinement operations on the question-and-answer data item is configured to: determine whether to refine the question-and-answer data item based on one or more of: the frequency at which the associated question-and-answer data item has been accessed exceeding a frequency threshold; the level of satisfaction determined based on the feedback by the one or more users regarding the question-and-answer data item being below a satisfaction threshold; the confidence level of the responsiveness of the answer portion to the question portion exceeding a responsiveness threshold; or the authorship information matching one or more pre-identified names. ### Claim 12 (depends on 8) The system of claim 8, wherein the communication unit is further configured to receive, at the local device, from a remote device query data representative of a subsequent question relating to the source content of the source document; and wherein the document processing agent is further configured to: determine whether the query data matches a question portion of the refined question-and-answer data item, stored in the question-answer cache, according to one or more matching criteria; and obtain from the question-answer cache, in response to a determination that the question portion of the refined question-and-answer data item matches the query data received from the remote device, at least one answer portion of the refined question-and-answer data item, and transmit the at least one answer portion to the remote device. ### Claim 13 (depends on 12) The system of claim 12, wherein the document processing agent is further configured to: in response to a determination that the query data does not match the question portion of the refined question-and-answer data item stored at the question-and-answer cache, searching at the local device a documents storage, comprising a plurality of document objects of transformed content corresponding to one or more source documents, to identify one or more of the plurality of document objects according to the query data; and generate, in response to determination, at the local device, of a match between the query data and at least one portion of the transformed content, output data based on the matched at least one portion of the transformed content. ### Claim 14 A non-transitory computer readable media programmed with instructions, executable on one or more processors of a computing system, to obtain at a local device, a question-and-answer data item representative of a question-and-answer pair relating to source content of a source document, wherein the question-and-answer data item is associated with answer responsiveness and feedback information; determine from the answer responsiveness and feedback information a level of answer responsiveness of the question-and-answer item to a submitted query and a level of answer satisfaction; and in response to a determination, based on the level of answer responsiveness of the question-and-answer data item and on the level of answer satisfaction, that the question-and-answer data item requires refinement: select one of a plurality of users according to one or more matching criteria relating, at least in part, to subject-matter associated with the question-and-answer data item and expertise information respectively associated with the plurality of users; and communicate at least the question-and-answer data item to a destination device associated with the selected one of the plurality of users for editing the at least one question-and-answer data item receive, at the local device, a refined question-and-answer data item derived from the question-and-answer data item communicated to the destination device; and store the refined question-and-answer data item at a question-answer cache maintained at the local device; wherein the answer responsiveness and feedback information comprise one or more of: frequency at which the associated question-and-answer data item has been accessed, level of satisfaction determined based on feedback by one or more users regarding the question-and-answer data item, a confidence level of responsiveness of an answer portion to a question portion of the question-and-answer data item, or authorship information associated with the question-and-answer data item, and wherein the question-and-answer data item comprises one or more of: a pointer to access from a remote device at least a portion of the source document corresponding to an answer to the question-and-answer pair, at least part of the source content, or a summary of the at least the portion of the source content.