Handwriting symbol recognition accuracy using speech input

A method comprising:

• receiving handwriting input including handwritten mathematical symbols;
• receiving speech input including a spoken word corresponding to a spoken mathematical symbol;
• recognizing the handwriting input to identify candidate handwritten mathematical symbols;
• recognizing the speech input to identify a candidate spoken word;
• processing the candidate handwritten mathematical symbols into a handwriting graph reflecting probabilities, represented by arcs, that the candidate handwritten mathematical symbols, represented by nodes, correctly identify the handwritten mathematical symbols;
• processing the candidate spoken word by:
• accessing a lookup table with the candidate spoken word recognized from the speech input to identify a candidate spoken mathematical symbol;
• and
• generating a speech graph reflecting a probability, represented by an arc, that the candidate spoken mathematical symbol, represented by a node, correctly identifies the spoken mathematical symbol;
• and
• fusing the handwriting graph and the speech graph to create an enhanced graph by adjusting individual probabilities of the handwriting graph based on individual matching probabilities in the speech graph,
• wherein at least the fusing is performed by a processing device.

A system comprising:

• a handwriting recognizer configured to:
• receive handwriting input including handwritten mathematical symbols;
• recognize the handwriting input to identify candidate handwritten mathematical symbols;
• and
• generate a handwriting graph reflecting probabilities, represented by arcs, that the candidate handwritten mathematical symbols, represented by nodes, correctly identify the handwritten mathematical symbols;
• a speech recognizer configured to:
• receive speech input including a spoken word corresponding to a spoken mathematical symbol;
• recognize the speech input to identify a candidate spoken word;
• access a lookup table with the candidate spoken word recognized from the speech input to identify a candidate spoken mathematical symbol;
• and
• generate a speech graph reflecting a probability, represented by an arc, that the candidate spoken mathematical symbol, represented by a node, correctly identifies the spoken mathematical symbol;
• a fusion mechanism configured to fuse the handwriting graph and the speech graph to create an enhanced graph by adjusting individual probabilities of the handwriting graph based on individual matching probabilities in the speech graph;
• and
• at least one processing device configured to execute the handwriting recognizer, the speech recognizer, or the fusion mechanism.