Transformer-based line segment matching across images
AssigneeCognex Corporation
InventorsMo Chen, Lowell D. Jacobson, Lei Wang
Life of this patent
Estimating the term…
Abstract
A system for matching line segments between images includes a transformer-based neural network with an encoder processing input images and a decoder processing a query line segment from a first image. A parallel decoding module estimates corresponding line segments in a second image based on processed feature maps and query line segments from the first image. The system may include a recursive zoom-in module that iteratively refines the estimated corresponding line segments by zooming into regions around initial estimates to achieve sub-pixel accuracy. The transformer-based neural network can be trained using a data generation process that adapts based on training loss curves.
Claims20
A system for matching geometric shapes between images, comprising:
- a convolutional neural network that extracts feature maps from a reference image and a test image;
- a transformer encoder that processes the feature maps from the reference image and the test image to generate encoded features;
- a transformer decoder comprising a masked multi-head self-attention mechanism that processes a query shape represented as a sequence of ordered points from the reference image, wherein the masked multi-head self-attention mechanism models interrelationships between the ordered points on the query shape;
- and
- a multilayer perceptron that generates corresponding shape points in the test image based on output from the transformer decoder.
A method for matching geometric shapes between images, comprising:
- extracting feature maps from a reference image and a test image using a convolutional neural network;
- processing the feature maps using a transformer encoder to generate encoded features;
- processing a query shape represented as a sequence of ordered points from the reference image using a transformer decoder comprising a masked multi-head self-attention mechanism, wherein the masked multi-head self-attention mechanism captures causal dependencies between sequential points along a shape contour;
- and
- generating corresponding shape points in the test image using a multilayer perceptron based on output from the transformer decoder.
A computer-implemented method for achieving sub-pixel accuracy in shape correspondence between images, comprising:
- obtaining an initial coarse estimate of a corresponding shape in a test image for a query shape in a reference image using a transformer-based neural network comprising a transformer decoder with masked multi-head self-attention;
- estimating an initial homography using the coarse estimate;
- warping corresponding local patches from the reference image and test image based on the initial homography;
- and
- applying the transformer-based neural network to the warped patches to obtain a refined shape correspondence with sub-pixel accuracy.
Dependencies are inferred from the claim text.