US 2024/0177514 A1Published application

Learning a form structure

AssigneeMicrosoft Technology Licensing, LLC

InventorsMattan SERRY, Zvi FIGOV

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system learns the structure of a form. The structure of the form can be learned from a single image (e.g., a photograph that includes the form) without user annotation. The form includes typewritten and handwritten text entries. The system groups text entries in the form based on lines detected in the form. The system then measures a distance and an angle between two text entry locations in the group of text entries. The group of text entries, the distances, and the angles can be captured in a bipartite graph. The bipartite graph represents possible pairing solutions where a typewritten text entry is paired with a handwritten text entry. The system identifies an optimal pairing solution, from the possible pairing solutions, using the distances and angles. The optimal pairing solution is identified by minimizing the standard deviation of the distances and/or by minimizing the circular standard deviation of the angles.

Claims20

Dependent claims

A method comprising:

receiving a form with a plurality of typewritten text entries and a plurality of handwritten text entries;
identifying a plurality of first locations on the form that respectively correspond to the plurality of typewritten text entries;
identifying a plurality of second locations on the form that respectively correspond to the plurality of handwritten text entries;
detecting a line on the form;
identifying a group of text entries that have corresponding locations that are not separated by the line, wherein the group of text entries includes:
a set of typewritten text entries of the plurality of typewritten text entries;
and
a set of handwritten text entries of the plurality of handwritten text entries;
for each typewritten text entry in the set of typewritten text entries:
determining a distance from a first location corresponding to the typewritten text entry to a second location corresponding to a handwritten text entry in the set of handwritten text entries;
and
determining an angle based on the first location corresponding to the typewritten text entry and the second location corresponding to the handwritten text entry;
identifying a pairing solution for the set of typewritten text entries and the set of handwritten text entries using the distances and the angles determined for each typewritten text entry in the set of typewritten text entries;
and
outputting the pairing solution.

12 more elements

A system comprising:

a processing system;
and
computer-readable storage media storing instructions that, when executed by the processing system, cause the system to:
identify a plurality of first locations on a form that respectively correspond to a plurality of typewritten text entries;
identify a plurality of second locations on the form that respectively correspond to a plurality of handwritten text entries;
detect a line on the form;
identify a group of text entries that have corresponding locations that are not separated by the line, wherein the group of text entries includes:
a set of typewritten text entries of the plurality of typewritten text entries;
and
a set of handwritten text entries of the plurality of handwritten text entries;
for each typewritten text entry in the set of typewritten text entries:
determine a distance from a first location corresponding to the typewritten text entry to a second location corresponding to a handwritten text entry in the set of handwritten text entries;
and
determine an angle based on the first location corresponding to the typewritten text entry and the second location corresponding to the handwritten text entry;
identify a pairing solution for the set of typewritten text entries and the set of handwritten text entries using the distances and the angles determined for each typewritten text entry in the set of typewritten text entries;
and
output the pairing solution.

14 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/071,465Filed 29 Nov 2022
Published: 30 May 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification7 codes

G06V 30/412
Analysis of document content: layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
G06V 10/32
Image preprocessing: normalisation of the pattern dimensions
G06V 10/768
Using pattern recognition or machine learning: using context analysis, e.g. recognition aided by known co-occurring patterns
G06V 30/2455
Using graphical properties, e.g. alphabet type or font: discrimination between machine-print, hand-print and cursive writing

Family

No related US applications are recorded in the indexed filing history.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key