Recognizing tables in unstructured text

A method of generating a table from unstructured text, the method comprising:

• identifying headers for the table within the unstructured text using a classifier model including an embedding-based classifier model trained to recognize individual headers within the unstructured text;
• generating a table sketch from which a final table is constructed, the table sketch generated by prompting a Large Language Model (LLM) with the headers identified by the classifier model and a portion of the unstructured text, the table sketch including identified headers and a number of table rows;
• and
• adding additional table rows to the table sketch using a consistency ranker that is operating on the unstructured text to generate the final table that captures data from the unstructured text in table form.

A method of generating a table from unstructured text, the method comprising:

• detecting header rows from the unstructured text using a custom model;
• generating, with a Large Language Model (LLM), (1) example rows and (2) auxiliary information comprising at least one of a row count, a column count, a header count, and a data pattern for a final table;
• constructing the final table using a symbolic system operating on the example rows and auxiliary information to guide construction, wherein the symbolic system comprises a model that relies on row and column level consistency measures based on (1) semantic features including embedding similarity and (2) symbolic features including regexes and types;
• and
• iteratively refining the constructed final table by identifying a worst scoring row as to the consistency measures and adjusting that worst scoring row until either the consistency scores are above a threshold or a maximum number of iterations is reached.