# US 2021/0342571 A1 · Automated signature extraction and verification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342571 A1
- Document ID: US20210342571A1
- Status: Published application
- Kind: A1
- Application number: 17/377,243
- Filed: 2021-07-15
- Published: 2021-11-04
- Earliest claimed: 2019-09-09
- Assignees: Morgan Stanley Services Group Inc.
- Applicants: Morgan Stanley Services Group Inc.
- Inventors: Sruthi Sangala; Titash Mandal; Sonil Trivedi; Sujit Eapen
- Web page: https://patentgrep.com/patent/US20210342571A1

## CPC

- G06V 10/82: using neural networks
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06F 18/24: Classification techniques
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06V 30/148: Segmentation of character regions
- G06V 30/18076: {by analysing connectivity, e.g. edge linking, connected component analysis or slices}
- G06V 30/19173: {Classification techniques}
- G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
- G06V 40/33: {based only on signature image, e.g. static signature recognition}
- G06V 40/376: {Acquisition}
- G06V 40/394: {Matching; Classification}
- G06V 30/10: Character recognition
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system for extraction and verification of handwritten signatures from arbitrary documents. The system comprises one or more computing devices configured to: receive a digital image of a document; perform a dilating transformation via convolution matrix on the digital image to obtain a modified image; determine a plurality of regions of connected markings in the digital image; based at least in part on a pixel density or proximity to an anchor substring of each region, determine whether any region contains any handwritten signature; extract first image data of the region containing a handwritten signature from the digital image; retrieve second image data of a confirmed example signature for a purported signer of the handwritten signature; and based on a comparison of the first image data with the second image data, forward a determination of whether the first image data and second image data are similar.

## Claims

### Claim 1

A system for extraction and verification of handwritten signatures, comprising:
one or more processors;
non-transitory memory comprising instructions that, when executed by the one or more processors, cause the one or more processors to:
receive a transmission comprising a digital image of a document;
perform a dilating transformation via convolution matrix on the digital image to obtain a modified image;
determine a plurality of regions of connected markings in the modified image;
based at least in part on a pixel density of or a proximity to an anchor substring of each of the plurality of regions, determine whether any regions of the plurality of regions contain any handwritten signature; and
if any regions do contain any handwritten signature:
extract first image data of each of those regions containing any handwritten signature from the digital image;
retrieve second image data of a confirmed example signature for purported signer of each handwritten signature from those regions; and
based on a comparison of the first image data with the second image data, forward a determination of whether the first image data and second image data represent signatures that are similar.

### Claim 2 (depends on 1)

The system of claim 1, wherein the comparison of the first image data with the second image data comprises:
subsequent to extracting the first image data and retrieving the second image data, creating at least one feature vector that incorporates a plurality of similarity metrics between the first and second image data; and
passing the at least one feature vector to a model to estimate a likelihood that that the first image data and second image data are similar to each other, wherein the determination of whether the first image data and second image data are similar is made based on the likelihood exceeding a predetermined threshold.

### Claim 3 (depends on 1)

The system of claim 1, wherein the determination whether any regions of the plurality of regions each contain any handwritten signature is additionally based on a ratio of height to width of each region.

### Claim 4 (depends on 1)

The system of claim 1, wherein the determination whether any regions of the plurality of regions each contain any handwritten signature is additionally based on automated analysis that determines a relative confidence that the region contains handwritten text compared to typewritten text based of the contents of each region.

### Claim 5 (depends on 1)

The system of claim 1, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:
automatically determine that no handwritten signature is present in the modified image; and
generate a message to a source of the digital image indicating that the document does not contain any signature.

### Claim 6 (depends on 2)

The system of claim 2, wherein the plurality of similarity metrics comprises a histogram of oriented gradients.

### Claim 7 (depends on 2)

The system of claim 2, wherein the plurality of similarity metrics comprises a cosine similarity metric.

### Claim 8 (depends on 2)

The system of claim 2, wherein the plurality of similarity metrics comprises an energy-entropy comparison.

### Claim 9 (depends on 2)

The system of claim 2, wherein the plurality of similarity metrics comprises a Pearson coefficient between sets of extracted waveforms from tiles.

### Claim 10 (depends on 1)

The system of claim 1, wherein the document is in an unknown format defined by an organization other than an organization operating the system, such that the system lacks access to any information regarding where a signature may be expected in the document before analysis of the document begins.

### Claim 11 (depends on 1)

The system of claim 1, wherein, when the determination whether any regions of the plurality of regions contain any handwritten signature is based at least in part on a pixel density of each of the plurality of regions, the determination comprises comparing pixel densities of each of the regions to each other in order to prioritize regions with the least density compared to the other regions.

### Claim 12

A method for extracting a handwritten signature from a digital image, comprising:
receiving a transmission comprising a digital image of a document;
performing a dilating transformation via convolution matrix on the digital image to obtain a modified image;
determining a plurality of regions of connected markings in the modified image;
based at least in part on a pixel density of or a proximity to an anchor substring of each of the plurality of regions, determining whether any regions of the plurality of regions contain any handwritten signature; and
if any regions do contain any handwritten signature:
extracting image data of each of those regions containing any handwritten signature from the digital image.

### Claim 13 (depends on 12)

The method of claim 12, wherein the determination whether any regions of the plurality of regions contain any handwritten signature is additionally based on a ratio of height to width of each region.

### Claim 14 (depends on 12)

The method of claim 12, wherein the determination whether any regions of the plurality of regions contain any handwritten signature is additionally based on automated analysis that determines a relative confidence that the region contains handwritten text compared to typewritten text based of the contents of each region.

### Claim 15 (depends on 12)

The method of claim 12, further comprising:
automatically determining that no handwritten signature is present in the modified image; and
generating a message to a source of the digital image indicating that the document does not contain any signature.

### Claim 16

A method for verifying that a handwritten signature was made by a given human user, comprising:
receiving first image data comprising a purported signature of an individual;
retrieving second image data comprising a known signature of an individual;
extracting at least one global feature from the first and second image data to obtain at least one global similarity metric between the first and second image data;
extracting a plurality of local features from the first and second image data to obtain a plurality of local similarity metrics between the first and second image data;
creating a feature vector that incorporates the global and local similarity metrics between the first and second image data; and
passing the feature vector to a model to estimate a likelihood that that the first image data and second image data are similar, wherein determining Whether the first image data and second image are similar is made based on the likelihood exceeding a predetermined threshold.

### Claim 17 (depends on 16)

The method of claim 16, wherein the plurality of local similarity metrics comprises a histogram of oriented gradients.

### Claim 18 (depends on 16)

The method of claim 16, wherein the plurality of local similarity metrics comprises a cosine similarity metric.

### Claim 19 (depends on 16)

The method of claim 16, wherein the plurality of local similarity metrics comprises an energy-entropy comparison.

### Claim 20 (depends on 16)

The method of claim 16, wherein the at least one global similarity metric comprises a Pearson coefficient between sets of extracted waveforms from tiles.
