Methods and systems for performing on-device image to text conversion

An electronic device comprising:

• memory storing one or more computer programs;
• and
• one or more processors communicatively coupled to the memory,
• wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
• recognize at least one block from at least one image representing content to be converted into text,
• recognize at least two markers relative to the content to be converted into the text, the recognizing of the at least two markers including:
• recognizing at least one lexicon and associated directional properties as at least one first marker, the lexicon including semantic information or grammatical information about individual words or word strings in the at least one image, and
• recognizing at least one of:
• at least one handwritten text, at least one non-text annotation, or at least one user edited text segment, as at least one second marker,
• perform an optical character recognition on the at least one block of the at least one image to create a set of converted text, based on the recognized at least two markers, the performing of the OCR including:
• recognizing at least one printed text in the at least one block of the at least one image, and
• creating the set of converted text by arranging the recognized at least two markers with respect to the recognized at least one printed text, and
• generate a text converted output, the text converted output being generated by a first neural network associating the recognized at least two markers with the set of converted text.

One or more non-transitory computer-readable storage media storing one or more computer programs including computer-executable instructions that, when executed by one or more processors of an electronic device individually or collectively, cause the electronic device to perform operations, the operations comprising:

• recognizing, by an electronic device, at least one block from at least one image representing content to be converted into text;
• recognizing, by the electronic device, at least two markers relative to the content to be converted into the text, the recognizing of the at least two markers including:
• recognizing at least one lexicon and associated directional properties as at least one first marker, the lexicon including semantic information or grammatical information about individual words or word strings in the at least one image;
• and
• recognizing at least one of:
• at least one handwritten text, at least one non-text annotation, or at least one user edited text segment, as at least one second marker;
• performing, by the electronic device, an optical character recognition (OCR) on the at least one block of the at least one image to create a set of converted text, based on the recognized at least two markers, the performing of the OCR including:
• recognizing at least one printed text in the at least one block of the at least one image;
• and
• creating the set of converted text by arranging the recognized at least two markers with respect to the recognized at least one printed text;
• and
• generating, by the electronic device, a text converted output, the text converted output being generated by a first neural network associating the recognized at least two markers with the set of converted text.