Reading order with pointer transformer networks
AssigneeGoogle Inc.
InventorsHenri Rebecq, Federico Tombari, Diego Martin Arroyo
Life of this patent
Estimating the term…
Abstract
A method including receiving an image representing a document including a plurality of layout components, identifying textual information associated with the plurality of layout components, identifying visual information associated with the plurality of layout components, combining the textual information with the visual information, and predicting a reading order of the plurality of layout components based on the combined textual information and visual information using a self-attention encoder/decoder.
Claims21
A method comprising:
- receiving an image representing a document including a plurality of layout components;
- identifying textual information associated with the plurality of layout components;
- identifying visual information associated with the plurality of layout components;
- combining the textual information with the visual information;
- and
- predicting a reading order of the plurality of layout components based on the combined textual information and visual information using a self-attention encoder/decoder.
A non-transitory computer-readable storage medium comprising instructions stored thereon that, when executed by at least one processor, are configured to cause a computing system to:
- receive an image representing a document including a plurality of layout components;
- identify textual information associated with the plurality of layout components;
- identify visual information associated with the plurality of layout components;
- combine the textual information with the visual information;
- and
- predict a reading order of the plurality of layout components based on the combined textual information and visual information using a self-attention encoder/decoder.
(canceled)
An apparatus comprising:
- at least one processor;
- and
- at least one memory including computer program code;
- the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to:
- receive an image representing a document including a plurality of layout components;
- identify textual information associated with the plurality of layout components;
- identify visual information associated with the plurality of layout components;
- combine the textual information with the visual information;
- and
- predict a reading order of the plurality of layout components based on the combined textual information and visual information using a self-attention encoder/decoder.
Dependencies are inferred from the claim text.