# US 2010/0163316 A1 · Handwriting Recognition System Using Multiple Path Recognition Framework

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0163316 A1
- Document ID: US20100163316A1
- Status: Published application
- Kind: A1
- Application number: 12/345,668
- Filed: 2008-12-30
- Published: 2010-07-01
- Earliest claimed: 2008-12-30
- Assignees: Microsoft Corporation
- Inventors: Ming Chang; Shi Han; Dongmei Zhang; Yu Zou; Xinjian Chen
- Web page: https://patentgrep.com/patent/US20100163316A1

## CPC

- G06V 30/347: {Sampling; Contour coding; Stroke extraction}
- G06V 30/373: {using a special pattern or subpattern alphabet}
- G16C 20/70: Machine learning, data mining or chemometrics
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16C: COMPUTATIONAL CHEMISTRY; CHEMOINFORMATICS; COMPUTATIONAL MATERIALS SCIENCE

## Abstract

Described is a multi-path handwriting recognition framework based upon stroke segmentation, symbol recognition, two-dimensional structure analysis and semantic structure analysis. Electronic pen input corresponding to handwritten input (e.g., a chemical expression) is recognized and output via a data structure, which may include multiple recognition candidates. A recognition framework performs stroke segmentation and symbol recognition on the input, and analyzes the structure of the input to output the data structure corresponding to recognition results. For chemical expressions, the structural analysis may perform a conditional sub-expression analysis for inorganic expressions, or organic bond detection, connection relationship analysis, organic atom determination and/or conditional sub-expression analysis for organic expressions. The structural analysis also performs subscript, superscript analysis and character determination. Further analysis may be performed, e.g., chemical valence analysis and/or semantic structure analysis.

## Claims

### Claim 1

In a computing environment, a method comprising, receiving electronic input corresponding to handwritten input with a two-dimensional structure, performing stroke segmentation and symbol recognition on the input, analyzing the two-dimensional structure of the input, and outputting a data structure corresponding to recognition results of the handwritten input.

### Claim 2 (depends on 1)

The method of claim 1 wherein analyzing the two-dimensional structure of the input comprises performing a conditional sub-expression analysis.

### Claim 3 (depends on 1)

The method of claim 1 wherein analyzing the two-dimensional structure of the input comprises performing a subscript, superscript analysis and a character determination analysis.

### Claim 4 (depends on 1)

The method of claim 1 wherein the handwritten input includes an organic chemical expression, and wherein analyzing the two-dimensional structure of the input comprises performing a bond detection and connection relationship analysis, or performing atom determination, or performing both a bond detection and connection relationship analysis and performing atom determination.

### Claim 5 (depends on 1)

The method of claim 1 further comprising, performing a semantic structure analysis.

### Claim 6 (depends on 5)

The method of claim 5 wherein the handwritten input corresponds to a chemical expression, and wherein performing the semantic structure analysis comprises performing a chemical valence analysis.

### Claim 7 (depends on 5)

The method of claim 5 wherein performing the semantic structure analysis comprises performing a syntax analysis with a syntax tree.

### Claim 8 (depends on 1)

The method of claim 1 wherein outputting the data structure comprises outputting an extended baseline structure tree.

### Claim 9 (depends on 8)

The method of claim 8 wherein outputting the extended baseline structure tree comprises including at least one solution node representing multiple recognition results.

### Claim 10 (depends on 9)

The method of claim 9 wherein outputting the data structure comprises outputting a baseline structure tree having stroke nodes representing strokes, symbol nodes representing symbols, BST symbol nodes representing a compound of a dominant symbol and its sub-baselines and relation nodes representing a baseline.

### Claim 11 (depends on 1)

The method of claim 1 wherein the handwritten input corresponds to a chemical expression, and further comprising, providing a chemical equation template edit tool and a chemical equation labeling tool for receiving sample handwritten chemical expressions.

### Claim 12

In a computing environment, a system comprising, a handwriting recognition framework, including two-dimensional structure analysis logic that receives a data structure comprising stroke and symbol data from a recognizer, processes the data structure based on a structure of the expression, and provides the modified data structure to one or more further analysis components which further modifies the data structure into output.

### Claim 13 (depends on 12)

The system of claim 12 wherein the data structure comprises a baseline structure tree having stroke nodes representing strokes, symbol nodes representing symbols, BST symbol nodes representing a compound of a dominant symbol and its sub-baselines and relation nodes representing a baseline.

### Claim 14 (depends on 13)

The system of claim 13 wherein multiple candidates are recognized, and wherein the framework modifies the baseline structure tree into an extended baseline structure tree by including solution nodes, each solution node corresponding to a recognition candidate.

### Claim 15 (depends on 12)

The system of claim 12 wherein the structure analysis logic performs subscript, superscript analysis and character determination.

### Claim 16 (depends on 12)

The system of claim 12 wherein the structure analysis logic performs conditional sub-expression analysis to find any sub-expression for each conditional symbol recognized from the handwritten input.

### Claim 17 (depends on 12)

The system of claim 12 wherein the one or more further analysis components a semantic structure analysis component, including chemical valence analysis component or a syntax analysis component, or both a chemical valence analysis component and a syntax analysis component.

### Claim 18 (depends on 12)

The system of claim 12 wherein the data to be analyzed comprises a chemical expression including an organic bond, and wherein the structure analysis logic performs organic bond detection, or a connection relationship analysis, or an atom determination, or any combination of a connection relationship analysis, organic atom determination, or conditional sub-expression analysis.

### Claim 19

One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:
processing input ink into stroke data and symbol data in a data structure;
modifying the data structure based upon a structural analysis of the stroke data and symbol data, including:
a) determining whether data to be analyzed corresponds to an organic bond, and if so,
i) performing organic bond detection, or connection relationship analysis, or organic atom determination, or conditional sub-expression analysis, or any combination of organic bond detection, connection relationship analysis, organic atom determination, or conditional sub-expression analysis, and if not,
ii) performing conditional sub-expression analysis;
b) performing subscript, superscript analysis and character determination; and
performing at least one other analysis that further modifies the data structure, including a chemical valence analysis or a syntax analysis, or both a chemical valence analysis and a syntax analysis.

### Claim 20 (depends on 19)

The one or more computer-readable media of claim 19 having further computer-executable instructions comprising extending the data structure by including solution nodes therein, each solution node corresponding to a recognition candidate.
