# US 2024/0005148 A1 · System and method for finding and classifying patterns in an image with a vision system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005148 A1
- Document ID: US20240005148A1
- Status: Published application
- Kind: A1
- Application number: 18/103,432
- Filed: 2023-01-30
- Published: 2024-01-04
- Earliest claimed: 2018-06-06
- Assignees: Cognex Corporation
- Applicants: Cognex Corporation
- Inventors: Lei Wang; Vivek Anand; Lowell D. Jacobson; David Y. Li
- Web page: https://patentgrep.com/patent/US20240005148A1

## CPC

- G06N 3/08: Learning methods
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 10/82: using neural networks
- G06V 10/7553: {based on shape, e.g. active shape models [ASM]}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

This invention provides a system and method for finding patterns in images that incorporates neural net classifiers. A pattern finding tool is coupled with a classifier that can be run before or after the tool to have labeled pattern results with sub-pixel accuracy. In the case of a pattern finding tool that can detect multiple templates, its performance is improved when a neural net classifier informs the pattern finding tool to work only on a subset of the originally trained templates. Similarly, in the case of a pattern finding tool that initially detects a pattern, a neural network classifier can then determine whether it has found the correct pattern. The neural network can also reconstruct/clean-up an imaged shape, and/or to eliminate pixels less relevant to the shape of interest, therefore reducing the search time, as well significantly increasing the chance of lock on the correct shapes.

## Claims

### Claim 1

A system for pattern-finding in an image comprising:
a pattern-finding tool that is trained based upon one or more template(s) related to one or more training image(s) containing a pattern of interest;
a neural net classifier trained on the one or more training image(s);
a template matching process in which, during runtime, either:
(a) the trained neural net classifier provides one or more template(s) based on a runtime image to the pattern-finding tool, and the trained pattern finding tool performs pattern matching based upon the one or more template images combined with the runtime image, or
(b) the trained pattern-finding tool provides found patterns from the runtime image to the trained neural net classifier, and the trained neural net classifier performs pattern-matching based on the found patterns and the runtime image.

### Claim 2 (depends on 1)

The system as set forth in claim 1 wherein the pattern-finding tool is adapted to be trained using a plurality of the templates.

### Claim 3 (depends on 1)

The system as set forth in claim 1 wherein the pattern-finding tool is trained on a single template.

### Claim 4 (depends on 1)

The system as set forth in claim 1 wherein the neural net classifier is associated with a neural network.

### Claim 5 (depends on 4)

The system as set forth in claim 4 wherein the neural network is trained to locate one or more candidate shapes in the one or more training images, and arranged to identify probability of the presence of the one or more shapes in the image during runtime operation.

### Claim 6 (depends on 5)

The system as set forth in claim 5 wherein the neural network generates (a) a weighted mask having features of the one or more candidate shapes that exceed a probability threshold or (b) a reconstructed image in which the features of a model of the one or more candidate shapes are substituted in the image where the neural network identifies presence of the features of the one or more candidate shapes that exceed the probability threshold.

### Claim 7 (depends on 6)

The system as set forth in claim 6 wherein the neural network defines the weighted mask in which each pixel therein has a score related to identification of the one or more shapes.

### Claim 8 (depends on 6)

The system as set forth in claim 6 wherein the neural network provides data as to presence of a type of the one or more candidate shapes to the pattern-finding tool and the pattern finding tool limits processes to those related to locate the type.

### Claim 9 (depends on 5)

The system as set forth in claim 5 wherein the reconstructed image is defined as a binary image.

### Claim 10 (depends on 5)

The system as set forth in claim 5 wherein the neural network comprises a convolutional neural network (CNN).

### Claim 11

A system for pattern-finding in an image comprising:
a neural network trained to locate one or more candidate shapes in an image and arranged to identify probability of the presence of the one or more shapes in the image during runtime operation and thereby generates (a) a weighted mask having features of the one or more candidate shapes that exceed a probability threshold and (b) a reconstructed image in which the features of a model of the one or more candidate shapes are substituted in the image where the neural network identifies presence of the features of the one or more candidate shapes that exceed the probability threshold.

### Claim 12 (depends on 11)

The system as set forth in claim 11 further comprising a pattern-finding tool that is trained using one or more models relative to the one or more candidate shapes to find the one or more candidate shapes in (a) the weighted mask and (b) the reconstructed image.

### Claim 13 (depends on 12)

The system as set forth in claim 12 wherein the neural network provides data as to presence of a type of the one or more candidate shapes to the pattern-finding tool and the pattern finding tool limits processes to those related to locate the type.

### Claim 14 (depends on 12)

The system as set forth in claim 12 wherein the pattern-finding tool is trained on a single one of the candidate shapes.

### Claim 15 (depends on 11)

The system as set forth in claim 11 wherein, in the weighted mask includes a plurality of pixels and wherein each of the pixels therein has a score related to identification of the one or more shapes.

### Claim 16 (depends on 11)

The system as set forth in claim 11 wherein the reconstructed image is defined as a binary image.

### Claim 17 (depends on 11)

The system as set forth in claim 11 wherein the neural network comprises a convolutional neural network (CNN).

### Claim 18

A system for pattern-finding in an image comprising:
a neural network trained to locate one or more candidate shapes in an image and arranged to identify probability of the presence of the one or more shapes in the image during runtime operation and thereby generates a weighted mask having features of the one or more candidate shapes that exceed a probability threshold.

### Claim 19 (depends on 18)

The system as set forth in claim 18 further comprising a pattern-finding tool that is trained using one or more models relative to the one or more candidate shapes to find the one or more candidate shapes in the weighted mask.

### Claim 20 (depends on 19)

The system as set forth in claim 19 wherein the neural network generates a reconstructed image that defines a binary image.
