Iterative training for text-image-layout transformer
AssigneeApplica Sp. Z O.O.
InventorsAdam Dancewicz, Filip Gralinski, Lukasz Konrad Borchmann
Life of this patent
Estimating the term…
Abstract
Disclosed herein is a system and method for Natural Language Processing (NLP) of real world documents. the system and method combines various models not previously combined and overcomes the challenges of this combination. Models include an encoder-decoder model, a spatial model, and a multi-modal model. An iterative training process receives documents and generates outputs, wherein the iterative training process comprises enabling information retrieval from documents without training data.
Claims30
A system for Natural Language Processing (NLP) of real world documents, the system comprising:
- a text-image-layout transformer (TILT) natural language processing (NLP) system that is executed on one or more processors, the TILT system comprises executable instructions that when executed by the processor, perform a method, the method comprising,
- executing one or more models selected from a group comprising,
- an encoder-decoder model;
- a spatial model, and;
- a multi-modal model;
- receiving data comprising at least text data, layout data, and image data;
- and
- operating on the received data to generate a useful output that relates to analysis of the received data;
- and
- the system further executing an iterative training process that receives documents and generates outputs, wherein the iterative training process comprises enabling information retrieval from documents without training data.
A method for natural language processing (NLP), the method comprising:
- one or more processors executing instructions to perform the method, the instructions comprising,
- receiving a real-world document that comprises at least text data, layout data, and image data;
- executing one or more models selected from a group comprising,
- an encoder-decoder model;
- a spatial model, and;
- a multi-modal model;
- operating on the real-world document to generate a useful output;
- and
- further executing an iterative training process that receives documents and generates outputs, wherein the iterative training process comprises enabling information retrieval from documents without training data.
A non-transient computer medium having stored therein instructions, which when executed by a processor perform a method, the method comprising:
- receiving a real-world document that comprises at least text data, layout data, and image data;
- executing one or more models selected from a group comprising,
- an encoder-decoder model;
- a spatial model, and;
- a multi-modal model;
- and
- operating on the real-world document to generate a useful output;
- and
- further executing an iterative training process that receives documents and generates outputs, wherein the iterative training process comprises enabling information retrieval from documents without training data.
Dependencies are inferred from the claim text.