# US 2017/0286529 A1 · Extracting Structured Data from Handwritten and Audio Notes > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0286529 A1 - Document ID: US20170286529A1 - Status: Published application - Kind: A1 - Application number: 15/475,001 - Filed: 2017-03-30 - Published: 2017-10-05 - Earliest claimed: 2016-03-30 - Assignees: Evernote Corporation - Applicants: Evernote Corporation - Inventors: Chris O'Neill; Andrew Malcolm; Alexander Pashintsev; Eugene Livshitz; Boris Gorbatov; Natalia Galaktionova; Ilya Buryak - Web page: https://patentgrep.com/patent/US20170286529A1 ## CPC - G06F 40/186: Templates - G06F 40/279: Recognition of textual entities - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract This application is directed to recognizing unstructured information based on hints provided by structured information. A computer system obtains unstructured information collected from a handwritten or audio source, and identifies one or more terms from the unstructured information. The one or more terms includes a first term that is ambiguous. The computer system performs a recognition operation on the first term to derive a first plurality of candidate terms for the first term, and obtains first contextual information from an information template associated with the unstructured information. In accordance with the first contextual information, the computer system selects a first answer term from the first plurality of candidate terms, such that the first term is recognized as the first answer term. ## Claims ### Claim 1 A computer-implemented method, comprising: at a computer system having one or more processors and memory storing one or more programs executed by the one or more processors: obtaining unstructured information collected from a handwritten or audio source; identifying one or more terms from the unstructured information, the one or more terms including a first term that is ambiguous; performing a recognition operation on the first term to derive a first plurality of candidate terms for the first term; obtaining first contextual information from an information template associated with the unstructured information; and in accordance with the first contextual information, selecting a first answer term from the first plurality of candidate terms, such that the first term is recognized as the first answer term. ### Claim 2 (depends on 1) The method of claim 1, wherein the first plurality of candidate terms are corresponding to different segmentations or recognitions of the first term. ### Claim 3 (depends on 1) The method of claim 1, the first contextual information including a plurality of contextual options, wherein the first answer term at least partially matches one of the plurality of contextual options, and other candidate terms of the first plurality of candidate terms do not match any of the plurality of contextual options. ### Claim 4 (depends on 1) The method of claim 1, wherein: the information template further comprises a plurality of data units, each data unit having a field name and a form field associated with the field name, the plurality of data units further including a first data unit; the first contextual information relates to the form field of the first data unit; and the first answer term recognized from the first term is used to define the form field of the first data unit. ### Claim 5 (depends on 4) The method of claim 4, wherein the form field corresponds to a plurality of predefined menu entries, and the first answer term at least partially matches one of the plurality of predefined menu entries, while other candidate terms of the first plurality of candidate terms do not match any of the plurality of predefined menu entries. ### Claim 6 (depends on 5) The method of claim 5, wherein the plurality of predefined menu entries are predefined and stored in a database. ### Claim 7 (depends on 5) The method of claim 5, wherein the plurality of predefined menu entries includes two or more company names. ### Claim 8 (depends on 5) The method of claim 5, wherein the one of the plurality of predefined menu entries includes a first name, and the first answer term at least partially matches one of the plurality of predefined menu entries, when the first answer term matches the first name or a variation of the first name, the variation of the first name including one of a group consisting of a synonym, an abbreviation, a homograph and a homophone of the first name. ### Claim 9 (depends on 1) The method of claim 1, wherein the one or more terms further includes a second term located in proximity to the first term in the unstructured information, and selecting the first answer term from the first plurality of candidate terms further comprises: recognizing the second term as a second answer term; determining that the second answer term is related to the first contextual information and the first answer term of the first plurality of candidate terms; and in accordance with the first contextual information, recognizing the first term as the first answer term. ### Claim 10 (depends on 9) The method of claim 9, the first contextual information including a first name, wherein the second term is associated with the second answer term when the second answer term matches the first name or a variation of the first name, the variation of the first name including one of a group consisting of a synonym, an abbreviation, a homograph and a homophone of the first name. ### Claim 11 (depends on 1) The method of claim 1, wherein: the information template further comprises a plurality of data units, each data unit having a field name and a form field associated with the field name, the plurality of data units further including a first data unit; the first contextual information includes the field name of the first data unit; and the first answer term recognized from the first term is used to define the form field of the first data unit. ### Claim 12 (depends on 11) The method of claim 11, wherein the one or more terms further includes a second term located in proximity to the first term in the unstructured information, and selecting the first answer term from the plurality of candidate terms further comprises: recognizing the second term as a second answer term; after determining that the second answer term matches the field name of the first data unit or a variation of the field name of the first data unit, determining that the form field is associated with the first term; and recognizing the first term as the first answer term. ### Claim 13 (depends on 12) The method of claim 12, wherein the variation of the field name of the first data unit includes one of a group consisting of a synonym, an abbreviation, a homograph and a homophone of the field name of the first data unit. ### Claim 14 (depends on 1) The method of claim 1, further comprising: obtaining second contextual information from a parallel source that is independent from the handwritten or audio source and the information template; and verifying that the first term has been properly recognized as the first answer term based on the second contextual information. ### Claim 15 (depends on 1) The method of claim 1, wherein selecting a first answer term from the first plurality of candidate terms further comprises: in accordance with the first contextual information, identifying the first answer term and a second answer term that is recognized as the first term; displaying a user interface to present the first and second candidate terms; and receiving a user selection of the first answer term, such that the first term is recognized as the first answer term based on the user selection. ### Claim 16 A computer system, comprising: one or more processors; and memory storing one or more programs to be executed by the one or more processors, the one or more programs comprising instructions for: obtaining unstructured information collected from a handwritten or audio source; identifying one or more terms from the unstructured information, the one or more terms including a first term that is ambiguous; performing a recognition operation on the first term to derive a first plurality of candidate terms for the first term; obtaining first contextual information from an information template associated with the unstructured information; and in accordance with the first contextual information, selecting a first answer term from the first plurality of candidate terms, such that the first term is recognized as the first answer term. ### Claim 17 (depends on 16) The computer system of claim 16, wherein the one or more programs further comprise instructions for: obtaining second contextual information from a parallel source that is independent from the handwritten or audio source and the information template; and verifying that the first term has been properly recognized as the first answer term based on the second contextual information. ### Claim 18 (depends on 16) The computer system of claim 16, wherein the instructions for selecting a first answer term from the first plurality of candidate terms further comprises instructions for: in accordance with the first contextual information, identifying the first answer term and a second answer term that is recognized as the first term; displaying a user interface to present the first and second candidate terms; and receiving a user selection of the first answer term, such that the first term is recognized as the first answer term based on the user selection. ### Claim 19 A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer system, the one or more programs comprising instructions for: obtaining unstructured information collected from a handwritten or audio source; identifying one or more terms from the unstructured information, the one or more terms including a first term that is ambiguous; performing a recognition operation on the first term to derive a first plurality of candidate terms for the first term; obtaining first contextual information from an information template associated with the unstructured information; and in accordance with the first contextual information, selecting a first answer term from the first plurality of candidate terms, such that the first term is recognized as the first answer term. ### Claim 20 (depends on 19) The non-transitory computer readable storage medium of claim 19, wherein: the information template further comprises a plurality of data units, each data unit having a field name and a form field associated with the field name, the plurality of data units further including a first data unit; the first contextual information includes one of the field name and the form field of the first data unit; and the first answer term recognized from the first term is used to define the form field of the first data unit.