Text-based machine learning extraction of table data from a read-only document
AssigneeCoupa Software Incorporated
InventorsHongyang Yu, Hanieh Borhanazad, Sandip Mandlecha
Life of this patent
Estimating the term…
Abstract
Embodiments of the disclosed technologies provide solutions for automatically reading digital electronic documents that contain tables and correctly extracting table data, rows and columns from the documents with high accuracy and high throughput. Embodiments are capable of converting a table portion of a read-only document to a searchable, editable data record using text rectangle (TR)-level numerical data that indicates probabilities of TRs belonging to canonicals and at least one convolutional neural network (CNN) that processes the TR-level numerical data to produce table-level numerical data.
Claims20
A method for converting a table portion of a read-only digital electronic document to a searchable, editable digital electronic data record, the method comprising:
- extracting text rectangle (TR) data from the read-only document;
- converting the TR data to a feature map that indicates both TR-level numerical data and spatial locations of TRs in the read-only document, the TR-level numerical data indicating probabilities of TRs belonging to text canonicals;
- by at least one convolutional neural network (CNN), processing the TR-level numerical data to produce table-level numerical data including anchor data that indicates a spatial location of the table portion within the read-only document, probabilities of TRs belonging to row canonicals, and probabilities of TRs belonging to column canonicals;
- formatting the table-level numerical data using a structured data format to produce formatted table data;
- storing the formatted table data in the searchable, editable data record;
- wherein the method is performed by one or more computing devices.
A system comprising:
- at least one hardware processor;
- memory coupled to the at least one hardware processor;
- instructions stored in the memory that when executed by the at least one hardware processor cause the system to be capable of:
- receiving, from a network, a message containing a read-only document;
- extracting text rectangle (TR) data from the read-only document;
- converting the TR data to a feature map that indicates both TR-level numerical data and spatial locations of TRs in the read-only document, the TR-level numerical data indicating probabilities of TRs belonging to text canonicals;
- by at least one convolutional neural network (CNN), processing the TR-level numerical data to produce table-level numerical data including anchor data that indicates a spatial location of a table portion within the read-only document, probabilities of TRs belonging to row canonicals, and probabilities of TRs belonging to column canonicals;
- formatting the table-level numerical data using a structured data format to produce formatted table data;
- storing the formatted table data in a searchable, editable data record;
- displaying the searchable, editable data record when data stored in the searchable, editable data record matches a query;
- wherein the method is performed by one or more computing devices.
A computer-implemented method for creating a three-dimensional tensor for input to a convolutional neural network useful in extracting digital data representing tables from electronic documents, the method comprising:
- extracting text rectangle (TR) data from a read-only document;
- converting the TR data to TR-level numerical data that indicates probabilities of TRs belonging to label canonicals or value canonicals;
- stretching and partitioning the read-only document to create a grid;
- projecting the TR-level numerical data onto the grid;
- a particular value of the TR-level numerical data mapping to at least one particular cell of the grid and corresponding to a level of pixel brightness associated with the particular cell;
- storing the grid including the TR-level numerical data as a three-dimensional tensor;
- wherein the method is performed by one or more computing devices.
Dependencies are inferred from the claim text.