Method and apparatus for processing document image based on text recognition and image division, and electronic device

An electronic device, comprising:

• at least one processor;
• and
• a memory communicatively connected to the at least one processor;
• wherein,
• the memory stores instructions executable by the at least one processor, wherein when the instructions are executed by the at least one processor, the at least one processor is configured to:
• acquire a document image;
• perform text recognition on the document image with an Optical Character Recognition (OCR) technology to acquire text nodes of multiple granularities, visual nodes of multiple granularities, respective node information of the text nodes, and respective node information of the visual nodes, wherein the node information of the text nodes comprises position information of the text nodes, and the node information of the visual nodes comprises position information of the visual nodes;
• construct a multi-granularity and multi-modality document graph based on the text nodes, the visual nodes, the respective node information of the text nodes, and the respective node information of the visual nodes;
• and
• determine, by a semantic feature extraction model, multi-granularity semantic feature information of the document image based on the multi-granularity and multi-modality document graph, the respective node information of the text nodes, and the respective node information of the visual nodes, wherein the semantic feature extraction model comprises an encoder, an aggregation layer, and a fusion layer.

A non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are configured to cause a computer to perform a method for processing a document image, the method comprising:

• acquiring a document image;
• performing text recognition on the document image with an Optical Character Recognition (OCR) technology to acquire text nodes of multiple granularities, visual nodes of multiple granularities, respective node information of the text nodes, and respective node information of the visual nodes, wherein the node information of the text nodes comprises position information of the text nodes, and the node information of the visual nodes comprises position information of the visual nodes;
• constructing a multi-granularity and multi-modality document graph based on the text nodes, the visual nodes, the respective node information of the text nodes, and the respective node information of the visual nodes;
• and
• determining, by a semantic feature extraction model, multi-granularity semantic feature information of the document image based on the multi-granularity and multi-modality document graph, the respective node information of the text nodes, and the respective node information of the visual nodes, wherein the semantic feature extraction model comprises an encoder, an aggregation layer, and a fusion layer.