# US 2020/0401796 A1 · System and method for processing text handwriting in a free handwriting mode > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0401796 A1 - Document ID: US20200401796A1 - Status: Published application - Kind: A1 - Application number: 16/715,977 - Filed: 2019-12-16 - Published: 2020-12-24 - Earliest claimed: 2019-06-20 - Assignees: MyScript - Applicants: MyScript - Inventors: Alain Chateigner - Web page: https://patentgrep.com/patent/US20200401796A1 ## CPC - G06F 18/00: Pattern recognition - G06V 30/2268: {using stroke segmentation} - G06F 40/106: Display of layout of documents; Previewing - G06F 40/151: Transformation - G06V 30/262: using context analysis, e.g. lexical, syntactic or semantic context - G06V 30/347: {Sampling; Contour coding; Stroke extraction} - G06V 30/36: {Matching; Classification} - G06F 2218/12: Classification; Matching - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The invention concerns a method comprising: detecting strokes of digital ink input on a computing device in a free handwriting format; detecting a text block from said strokes; performing text recognition on each text line of said text block, including extracting text lines from the text block and generating model data that associate each stroke of the text block with a character, a word and a text line of the text block; normalizing each text line from the free handwriting format into a structured format to comply with a document pattern. The normalization may comprise for each text line: computing a transform function to transform said text line into the structured format; applying the transform function to the text line; and updating the model data of said text line based on the transform function. ## Claims ### Claim 1 A method for processing text handwriting on a computing device, the computing device comprising a processor, a memory and at least one non-transitory computer readable medium for processing text handwriting under control of the processor, the method comprising: detecting a plurality of input strokes of digital ink with an input surface, said input strokes being input in a free handwriting format without any handwriting constraint; displaying said plurality of input strokes on a display device in said free handwriting format; classifying each input stroke as text or non-text, said classifying comprising detecting as text at least one text block from said input strokes; performing text recognition on said at least one text block, said text recognition comprising: extracting text lines from said at least one text block; generating model data that associate each stroke of said at least one text block with a character, a word and a text line of said at least one text block; normalizing each text line from the free handwriting format into a structured format to comply with a document pattern, said normalization comprising for each text line: computing for said text line a respective transform function to transform said text line into the structured format; applying the respective transform function to transform each stroke of said text line into the structured format; and updating the model data of said text line based on the respective transform function. ### Claim 2 (depends on 1) The method of claim 1, comprising storing the model data generated during said text recognition, wherein said updating the model data further comprises storing the updated model data of said at least one text block in replacement of the model data generated during said text recognition. ### Claim 3 (depends on 1) The method of claim 1, comprising, after said normalizing, displaying the text lines of said at least one text block in the structured format. ### Claim 4 (depends on 1) The method of claim 1, wherein the model data of the at least one text block comprise: character information defining a plurality of characters, each character being associated with at least one stroke of digital ink and with a text line of said at least one text block; word information defining a plurality of words, each word being associated with at least one character as defined by the character information; and line information defining each text line of said at least one text block, each text line being associated with at least one word as defined by the word information. ### Claim 5 (depends on 4) The method of claim 4, wherein the line information comprises, for each text line of said at least one text block: origin coordinates representing an origin of the text line; slant information representing a slant of the text line; and height information representing a height of the text line. ### Claim 6 (depends on 1) The method of claim 1, wherein said updating the model data during said normalizing comprises updating the line information of said text line based on the respective transform function. ### Claim 7 (depends on 1) The method of claim 1, wherein said normalizing comprises for each text line: determining input parameters comprising the origin coordinates, the slant information and the height information of said text line; wherein the respective transform function is computed based on the input parameters and on the document pattern. ### Claim 8 (depends on 1) The method of claim 1, wherein the document pattern defines at least one of the following handwriting constraints to be complied with by text: a margin of a display area; a line pattern; and an interline distance. ### Claim 9 (depends on 1) The method of claim 1, wherein each transform function defines at least one of the following transformation components to be applied on a respective text line during said normalizing: a translation component; a scale component; and a rotation component. ### Claim 10 (depends on 9) The method of claim 9, wherein the document pattern comprises a line pattern defining guidelines according to which text is to be arranged in the structured format. ### Claim 11 (depends on 10) The method of claim 10, wherein the scale component of the transform function is determined during said normalizing based on a ratio of the distance between two consecutive guidelines of said line pattern relative to the height of the respective text line. ### Claim 12 (depends on 10) The method of claim 10, wherein the translation component of the transform function is determined during said normalizing to perform a translation of a text line so that the origin of said text line is moved to be aligned with a corresponding guideline of the line pattern which is assigned to said text line during said normalizing. ### Claim 13 (depends on 9) The method of claim 9, wherein said rotating component is determined during said normalizing to rotate a respective text line so at to reduce its slant to zero in accordance with the document pattern. ### Claim 14 (depends on 1) The method of claim 1, wherein during said normalizing, the model data of each text line is updated according to the respective transform function while blocking any text recognition that may result from said applying the respective transform function. ### Claim 15 A computing device for handwriting text, comprising: an input surface for detecting a plurality of strokes of digital ink, said strokes being input in a free handwriting format without any handwriting constraint; a display device for displaying said plurality of input strokes in said free handwriting format; a classifier for classifying each stroke as text or non-text, said classifier being configured to detect as text at least one text block from said input strokes; a line extractor to extract text lines from said at least one text block; a recognition engine for performing a text recognition on each text line of said at least one text block, thereby generating model data that associate each stroke of said at least one text block with a character, a word and a text line of said at least one text block; a text editor for normalizing each text line from the free handwriting format into a structured format to comply with a document pattern, said text editor being configured to perform for each text line: computing for said text line a respective transform function to transform said text line into the structured format; applying the respective transform function to transform each stroke of said text line into the structured format; and updating the model data of said text line based on the respective transform function. ### Claim 16 A non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for processing text handwriting on a computing device, the computing device comprising a processor, a memory and at least one non-transitory computer readable medium for processing text handwriting under control of the processor, the method comprising: detecting a plurality of input strokes of digital ink with an input surface, said input strokes being input in a free handwriting format without any handwriting constraint; displaying said plurality of input strokes on a display device in said free handwriting format; classifying each input stroke as text or non-text, said classifying comprising detecting as text at least one text block from said input strokes; performing text recognition on said at least one text block, said text recognition comprising: extracting text lines from said at least one text block; generating model data that associate each stroke of said at least one text block with a character, a word and a text line of said at least one text block; normalizing each text line from the free handwriting format into a structured format to comply with a document pattern, said normalization comprising for each text line: computing for said text line a respective transform function to transform said text line into the structured format; applying the respective transform function to transform each stroke of said text line into the structured format; and updating the model data of said text line based on the respective transform function.