# US 2025/0148821 A1 · Handwriting recognition pipelines for genealogical records

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148821 A1
- Document ID: US20250148821A1
- Status: Published application
- Kind: A1
- Application number: 18/952,858
- Filed: 2024-11-19
- Published: 2025-05-08
- Earliest claimed: 2021-07-09
- Assignees: Ancestry.com Operations Inc.
- Applicants: Ancestry.com Operations Inc.
- Inventors: Masaki Stanley Fujimoto; Kalyan Chakravarthi Murahari; Siteng Chen
- Web page: https://patentgrep.com/patent/US20250148821A1

## CPC

- G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06Q 10/10: Office automation; Time management
- G06Q 50/18: Legal services
- G06Q 50/26: Government or public services (business processes related to the transportation industry G06Q50/40)
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/809: {of classification results, e.g. where the classifiers operate on the same input data}
- G06V 10/82: using neural networks
- G06V 30/1801: {Detecting partial patterns, e.g. edges or contours, or configurations, e.g. loops, corners, strokes or intersections (extracting features by contour coding G06V30/182)}
- G06V 30/18057: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 30/226: of cursive writing
- G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
- G06V 30/42: based on the type of document
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Disclosed herein relates to example embodiments for recognizing handwritten information in a genealogical record. A computing server may receive a genealogical record. The genealogical record may take the form of an image of a physical form having a structured layout, fields, and handwritten information. The computing server may divide the genealogical record into a plurality of areas based on the structured layout. The computing server may identify, for a particular area, a type of field that is included within the particular area. The computing server may select a handwriting recognition model for identifying the handwritten information in the particular area. The handwriting recognition model may be selected based on the type of the field. The computing server may input an image of the particular area to the handwriting recognition model to generate text of the handwritten information. The computing server may store the text of the handwritten information.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A computer-implemented method, comprising:
determining, for a template image defining a layout of handwriting content for an image set, template area coordinates defining bounding boxes dividing the template image into template areas;
transforming the template area coordinates to align with dimensions of an input digital image within the image set; and
determining a location of a handwriting content field within the input digital image by extracting, from the input digital image, landmark coordinates indicating locations of handwriting content based on transforming the template area coordinates.

### Claim 22 (depends on 21)

The computer-implemented method of claim 21, wherein determining the location of the handwriting content field comprises determining a landmark score for the handwriting content field using a landmark comparer neural network to compare landmarks in the template image and the input digital image.

### Claim 23 (depends on 21)

The computer-implemented method of claim 21, wherein transforming the template area coordinates to align with the dimensions of the input digital image comprises:
extracting a first image region from the template image and a second image region from the input digital image;
determining, utilizing a keypoint detection model, a first set of keypoints for the first image region and a second set of keypoints for the second image region; and
comparing the first set of keypoints and the second set of keypoints.

### Claim 24 (depends on 23)

The computer-implemented method of claim 23, wherein comparing the first set of keypoints and the second set of keypoints comprises:
filtering erroneous keypoints shared between the first set of keypoints and the second set of keypoints;
generating, utilizing an estimator model robust to the erroneous keypoints, a homography matrix defining a transform for aligning the first set of keypoints and the second set of keypoints; and
determining an alignment between the template image and the input digital image based on the homography matrix.

### Claim 25 (depends on 24)

The computer-implemented method of claim 24, wherein determining the alignment between the template image and the input digital image comprises determining a symmetric mean average percent error from the transform aligning the first set of keypoints and the second set of keypoints.

### Claim 26 (depends on 23)

The computer-implemented method of claim 23, wherein the keypoint detection model comprises a deep learning model.

### Claim 27 (depends on 21)

The computer-implemented method of claim 21, further comprising generating, from the handwriting content field within the input digital image, digital text characters utilizing a handwriting recognition model that focuses on discrete image regions of the input digital image.

### Claim 28

A system comprising:
at least one processor; and
one or more memory devices coupled to the at least one processor, the one or more memory devices storing instructions that, when executed by the at least one processor, cause the at least one processor to:
determine, for a template image defining a layout of handwriting content for an image set, template area coordinates defining bounding boxes dividing the template image into template areas;
transform the template area coordinates to align with dimensions of an input digital image within the image set; and
determine a location of a handwriting content field within the input digital image by extracting, from the input digital image, landmark coordinates indicating locations of handwriting content based on transforming the template area coordinates.

### Claim 29 (depends on 28)

The system of claim 28, wherein the one or more memory devices store further instructions that, when executed by the at least one processor, cause the at least one processor to determine the location of the handwriting content field by determining a landmark score for the handwriting content field using a landmark comparer neural network to compare landmarks in the template image and the input digital image.

### Claim 30 (depends on 28)

The system of claim 28, wherein the one or more memory devices store further instructions that, when executed by the at least one processor, cause the at least one processor to transform the template area coordinates to align with the dimensions of the input digital image by:
extracting a first image region from the template image and a second image region from the input digital image;
determining, utilizing a keypoint detection model, a first set of keypoints for the first image region and a second set of keypoints for the second image region; and
comparing the first set of keypoints and the second set of keypoints.

### Claim 31 (depends on 30)

The system of claim 30, wherein the one or more memory devices store further instructions that, when executed by the at least one processor, cause the at least one processor to compare the first set of keypoints and the second set of keypoints by:
filtering erroneous keypoints shared between the first set of keypoints and the second set of keypoints;
generating, utilizing an estimator model robust to the erroneous keypoints, a homography matrix defining a transform for aligning the first set of keypoints and the second set of keypoints; and
determining an alignment between the template image and the input digital image based on the homography matrix.

### Claim 32 (depends on 31)

The system of claim 31, wherein the one or more memory devices store further instructions that, when executed by the at least one processor, cause the at least one processor to determine the alignment between the template image and the input digital image by determining a symmetric mean average percent error from the transform aligning the first set of keypoints and the second set of keypoints.

### Claim 33 (depends on 30)

The system of claim 30, wherein the keypoint detection model comprises a deep learning model.

### Claim 34 (depends on 28)

The system of claim 28, wherein the one or more memory devices store further instructions that, when executed by the at least one processor, cause the at least one processor to generate, from the handwriting content field within the input digital image, digital text characters utilizing a handwriting recognition model that focuses on discrete image regions of the input digital image.

### Claim 35

A non-transitory computer readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
determining, for a template image defining a layout of handwriting content for an image set, template area coordinates defining bounding boxes dividing the template image into template areas;
transforming the template area coordinates to align with dimensions of an input digital image within the image set; and
determining a location of a handwriting content field within the input digital image by extracting, from the input digital image, landmark coordinates indicating locations of handwriting content based on transforming the template area coordinates.

### Claim 36 (depends on 35)

The non-transitory computer readable medium of claim 35, wherein
determining the location of the handwriting content field comprises determining a landmark score for the handwriting content field using a landmark comparer neural network to compare landmarks in the template image and the input digital image.

### Claim 37 (depends on 35)

The non-transitory computer readable medium of claim 35, wherein transforming the template area coordinates to align with the dimensions of the input digital image comprises:
extracting a first image region from the template image and a second image region from the input digital image;
determining, utilizing a keypoint detection model, a first set of keypoints for the first image region and a second set of keypoints for the second image region; and
comparing the first set of keypoints and the second set of keypoints.

### Claim 38 (depends on 37)

The non-transitory computer readable medium of claim 37, wherein comparing the first set of keypoints and the second set of keypoints comprises:
filtering erroneous keypoints shared between the first set of keypoints and the second set of keypoints;
generating, utilizing an estimator model robust to the erroneous keypoints, a homography matrix defining a transform for aligning the first set of keypoints and the second set of keypoints; and
determining an alignment between the template image and the input digital image based on the homography matrix.

### Claim 39 (depends on 38)

The non-transitory computer readable medium of claim 38, wherein determining the alignment between the template image and the input digital image comprises determining a symmetric mean average percent error from the transform aligning the first set of keypoints and the second set of keypoints.

### Claim 40 (depends on 35)

The non-transitory computer readable medium of claim 35, wherein the operations further comprise generating, from the handwriting content field within the input digital image, digital text characters utilizing a handwriting recognition model that focuses on corner regions of the input digital image.
