Information extraction system for unstructured documents using independent tabular and textual retrieval augmentation

A method for prompting a language model with content, the method comprising:

• receiving, by one or more processors, the content including text and one or more tables;
• forming, by the one or more processors, one or more table chunks and one or more text chunks from the content by separating the one or more tables from the text, wherein each table chunk of the one or more table chunks comprising at least a portion of a table of the one or more tables and each text chunk comprising at least a portion of the text;
• generating, by the one or more processors, one or more first index entries for the one or more table chunks and one or more second index entries for the one or more text chunks by converting the one or more table chunks and the one or more text chunks into vector text embeddings using a text embedding model;
• identifying, by the one or more processors, a relevant table chunk of the one or more table chunks or a relevant text chunk of the one or more text chunks based on a search of the one or more first index entries or the one or more second index entries related to a prompt for the language model;
• and
• storing a response from the language model to the prompt with the relevant table chunk or the relevant text chunk.