Apparatus and method for producing labelled cursive handwritten text samples with context style variability suitable for forming training data for training machine learning neural networks for cursive handwriting recognition

A method of training an apparatus for producing labelled cursive handwritten text samples with context style variability, the apparatus comprising at least an input text word embedding unit, a context style word embedding unit, a word embedding transformer, a feature embedding combiner, a generator and a discriminator, the method comprises:

• encoding, by the input text word embedding unit, an input text to obtain an input text word embedding in an input text feature space;
• encoding, by the context style word embedding unit, a handwriting context style description into a context style description embedding in a handwriting context style feature space;
• transforming, by the word embedding transformer, the context style description embedding from the handwriting context style feature space to the input text feature space to obtain a handwriting context style feature embedding;
• combining, by the feature embedding combiner, the handwriting context style feature embedding and the input text word embeddings to form a combined feature embedding;
• generating, by the generator, a synthetic image based on the combined feature embedding, wherein the synthetic image contains characters occurred in the input text with handwriting context style defined in the handwriting context style description;
• discriminating, by the discriminator, the characters in the generated synthetic image as real or fake handwritten characters and generating an update data representative of likelihood between the generated synthetic image and a reference labelled image, wherein the update data is used to optimize the generator.

A method of using an apparatus for producing labelled cursive handwritten text samples with context style variability, the apparatus comprising at least an input text word embedding unit, a context style word embedding unit, a word embedding transformer, a feature embedding combiner, a generator, a recognition unit and an annotation unit;

• the method comprises:
• encoding, by the input text word embedding unit, an input text to obtain an input text word embedding in an input text feature space;
• encoding, by the context style word embedding unit, a handwriting context style description into a context style description embedding in a handwriting context style feature space;
• transforming, by the word embedding transformer, the context style description embedding from the handwriting context style feature space to the input text feature space to obtain a handwriting context style feature embedding;
• combining, by the feature embedding combiner, the handwriting context style feature embedding and the input text word embeddings to form a combined feature embedding;
• generating, by the generator, a synthetic image based on the combined feature embedding;
• recognizing, by the recognition unit, the synthetic image to generate a corresponding machine recognizable text;
• and
• automatically annotating, by the annotation unit, the synthetic image with the corresponding machine recognizable text and the handwriting context style description to create a new labelled image data;
• and
• wherein the synthetic image contains characters occurred in the input text with handwriting context style defined in the handwriting context style description and context style variability introduced by the generator.