Text line detection

An electronic device, comprising:

• at least one processor;
• and
• a memory coupled to the at least one processor and having instructions stored thereon which, when executed by the at least one processor, cause the electronic device to perform operations to:
• determine, using a scene text detection model trained on a plurality of training images comprising text of multiple orientations, fonts, and scales, a first text region and a second text region in an image, the first text region comprising a first portion of at least a first text element and the second text region comprising a second portion of at least a second text element;
• extract, using a convolutional neural network configured to process multiple color spaces, a first feature representation from the first text region and a second feature representation from the second text region, the first and second feature representations comprising at least one of an image feature representation or a semantic feature representation;
• determine, using a semantic information extraction model, a semantic relationship between the first text region and the second text region by performing semantic information extraction on content of the first text element and the second text element using the semantic information extraction model to obtain semantic features representing textual meaning of the first and second text elements;
• extract a third feature representation using the semantic relationship by performing multimodal stitching and fusion on the semantic features of the first and second text elements;
• determine, based at least in part on an evaluation of the first feature representation, the second feature representation, and the third feature representation using a relation prediction machine learning model, a link relationship between the first and second text regions, the link relationship indicating whether the first and second portions of the first and second text elements are located in a same text line, wherein the relation prediction model is trained using training samples comprising pairs of text regions with known linking relationships;
• according to the link relationship indicating that at least a portion of the first text element in the first text region and at least a portion of the second text element in the second text region are located in the same text line, determine a first text line region that at least defines the first text region and the second text region in the image;
• and
• output, via a display device, the first text line region.

At least one non-transitory machine-readable medium comprising machine-executable instructions which, when executed by a device, cause the device to perform operations to:

• determine, using a scene text detection model trained on a plurality of training images comprising text of multiple orientations, fonts, and scales, a first text region and a second text region in an image, the first text region comprising a first portion of at least a first text element and the second text region comprising a second portion of at least a second text element;
• extract, using a convolutional neural network configured to process multiple color spaces, a first feature representation from the first text region and a second feature representation from the second text region, the first and second feature representations comprising at least one of an image feature representation or a semantic feature representation;
• determine, using a semantic information extraction model, a semantic relationship between the first text region and the second text region by performing semantic information extraction on content of the first text element and the second text element using the semantic information extraction model to obtain semantic features representing textual meaning of the first and second text elements;
• extract a third feature representation using the semantic relationship by performing multimodal stitching and fusion on the semantic features of the first and second text elements;
• determine, based at least in part on an evaluation of the first feature representation, the second feature representation, and the third feature representation using a relation prediction machine learning model, a link relationship between the first and second text regions, the link relationship indicating whether the first and second portions of the first and second text elements are located in a same text line, wherein the relation prediction model is trained using training samples comprising pairs of text regions with known linking relationships;
• according to the link relationship indicating that at least a portion of the first text element in the first text region and at least a portion of the second text element in the second text region are located in the same text line, determine a first text line region that at least defines the first text region and the second text region in the image;
• and
• output, via a display device, the first text line region.