# US 2026/0220365 A1 · System and method for automatic language detection for handwritten text

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0220365 A1
- Document ID: US20260220365A1
- Status: Published application
- Kind: A1
- Application number: 19/573,552
- Filed: 2026-03-20
- Published: 2026-07-30
- Earliest claimed: 2018-05-29
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Xiao TU; Zhe WANG
- Web page: https://patentgrep.com/patent/US20260220365A1

## CPC

- G06F 40/263: Language identification
- G06V 10/82: using neural networks
- G06F 3/018: {Input/output arrangements for oriental characters}
- G06F 3/03545: {Pens or stylus}
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06N 5/042: {Backward inferencing}
- G06N 5/046: Forward inferencing; Production systems
- G06V 30/1423: {the instrument generating sequences of position coordinates corresponding to handwriting (preprocessing or recognising digital ink G06V30/32)}
- G06V 30/246: using linguistic properties, e.g. specific for English or German language
- G06V 30/32: Digital ink
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods for automatic language detection for handwritten text are performed by systems and devices. Such automatic language detection is performed prior to sending representations of the handwritten text to a language recognition engine. Handwritten inputs including one or more writing strokes are received from an input interface, and are associated with coordinates of the inputs and times that the inputs are made. The handwritten inputs are grouped into words based on the coordinates and times. Writing strokes are normalized, and then the words are individually transformed to generate language vectors, such as through a recurrent neural network. The language vectors are used to determine language probabilities for the handwritten inputs. Based on the language probabilities, the handwritten inputs are provided to a specific language recognition engine to determine the language thereof prior to translation or transcription.

## Claims

### Claim 1

A language detection system comprising:
a processing system comprising one or more processors; and
a memory configured to store program code to be executed by the one or more processors, the program code including instructions for operations comprising:
receiving handwritten inputs from an input interface, the handwritten inputs including one or more writing strokes;
determining at least one language probability based on the one or more writing strokes;
based at least on the at least one language probability, selecting a specific language recognition engine from a plurality of language recognition engines corresponding to different languages; and
providing the handwritten inputs to the selected specific language recognition engine to recognize the handwritten inputs.

### Claim 2 (depends on 1)

The language detection system of claim 1, wherein the operations further comprise associating the handwritten inputs with sets of coordinate-time pairs.

### Claim 3 (depends on 2)

The language detection system of claim 2, wherein the operations further comprise grouping the handwritten inputs into words based at least on the sets of coordinate time pairs.

### Claim 4 (depends on 2)

The language detection system of claim 2, wherein each of the writing strokes includes one or more of the sets of coordinate-time pairs; and
wherein the operations further comprise:
comparing spatial and temporal attributes of each of the one or more writing strokes with others of the one or more writing strokes based at least on the one or more sets of coordinate time pairs; and
grouping the handwritten inputs into words based at least on the spatial and temporal attributes.

### Claim 5 (depends on 4)

The language detection system of claim 4, wherein the spatial and temporal attributes include at least one of:
a direction of a successive word with respect to a previous word;
amounts of time between the one or more writing strokes;
an amount of time taken to complete a writing stroke; or
a proximity of a writing stroke to others of the one or more writing strokes.

### Claim 6 (depends on 1)

The language detection system of claim 1, wherein the handwritten inputs comprise handwriting strokes captured in approximately real-time, and wherein the input interface comprises a user interface that is one or more of a touchscreen, a touchpad, or a camera; or
wherein the handwritten inputs comprise a stored representation of handwriting, and wherein the input interface comprises an access interface to a storage where the representation is stored.

### Claim 7 (depends on 1)

The language detection system of claim 1, wherein the operations further comprise:
associating the handwritten inputs with sets of coordinate-time pairs;
grouping the handwritten inputs into words based at least on the sets of coordinate-time pairs;
normalizing the one or more writing strokes that correspond to the words to generate normalized words;
transforming the normalized words individually to generate language vectors for the normalized words; and
wherein determining the at least one language probability is based at least on the language vectors.

### Claim 8 (depends on 1)

The language detection system of claim 1, wherein the specific language recognition engine is configured to:
generate a language translation or a transcription of the handwritten inputs; and
a user interface configured to:
provide the language translation or the transcription of the handwritten inputs generated by the specific language recognition engine to a display.

### Claim 9

A computer-implemented method for language detection, the method comprising:
receiving handwritten inputs from an input interface, the handwritten inputs including one or more writing strokes;
determining at least one language probability based on the one or more writing strokes;
based at least on the at least one language probability, selecting a specific language recognition engine from a plurality of language recognition engines corresponding to different languages; and
providing the handwritten inputs to the selected specific language recognition engine to recognize the handwritten inputs.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, further comprising:
associating the handwritten inputs with sets of coordinate-time pairs; and
normalizing the one or more writing strokes that correspond to writing strokes of words to generate normalized words.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, further comprising at least one of:
analyzing the normalized words individually to generate language vectors for the normalized words using a recurrent neural network (RNN) where the language vectors are outputs of the RNN; or
analyzing the normalized words individually to generate the language vectors for the normalized words using a bi-directional recurrent neural network (RNN) that generates first vectors from a forward portion of the bi-directional RNN and second vectors from a backward portion of the bi-directional RNN, where the first vectors and the second vectors are concatenated to generate the language vectors that are outputs of the RNN.

### Claim 12 (depends on 9)

The computer-implemented method of claim 9, wherein each of the writing strokes includes one or more sets of coordinate-time pairs, the method further comprising:
comparing spatial and temporal attributes of each of the one or more writing strokes with others of the one or more writing strokes based at least on the one or more sets of coordinate time pairs; and
grouping the handwritten inputs into words based at least on the spatial and temporal attributes.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, wherein the spatial and temporal attributes include at least one of:
a direction of a successive word with respect to a previous word;
amounts of time between the one or more writing strokes;
an amount of time taken to complete a writing stroke; or
a proximity of a writing stroke to others of the one or more writing strokes.

### Claim 14 (depends on 9)

The computer-implemented method of claim 9, wherein the handwritten inputs comprise handwriting strokes captured in approximately real-time, and wherein the input interface comprises a user interface that is one or more of a touchscreen, a touchpad, or a camera; or
wherein the handwritten inputs comprise a stored representation of handwriting, and wherein the input interface comprises an access interface to a storage where the representation is stored.

### Claim 15 (depends on 9)

The computer-implemented method of claim 9, further comprising:
detecting symbols in the handwritten inputs; and
providing the detected symbols to a detection manager to determine the at least one language probability.

### Claim 16 (depends on 9)

The computer-implemented method of claim 9, further comprising:
generating, by the specific language recognition engine, a language translation or a transcription of the handwritten inputs; and
providing the language translation or the transcription of the handwritten inputs generated to a user interface of a display.

### Claim 17 (depends on 9)

The computer-implemented method of claim 9, wherein the determining at least one language probability is also based at least on one or more user profile attributes.

### Claim 18

A computer-implemented method for language detection, the method comprising:
receiving handwritten inputs from an input interface, the handwritten inputs including one or more writing strokes;
grouping the handwritten inputs into words;
transforming the words individually to generate language vectors for the words;
determining at least one language probability based at least on the language vectors;
based at least on the at least one language probability, selecting a specific language recognition engine from a plurality of language recognition engines corresponding to different languages; and
providing the handwritten inputs to the selected specific language recognition engine to recognize the handwritten inputs.

### Claim 19 (depends on 18)

The method of claim 18, wherein grouping the handwritten inputs into words is based on sets of coordinate-time pairs corresponding to the handwritten inputs.

### Claim 20 (depends on 18)

The method of claim 18, wherein the language vectors are generated using a recurrent neural network (RNN) where the language vectors are outputs of the RNN.
