Back to results
US 12,639,579 B2Granted
Systems and methods for virtual and augmented reality
AssigneeMagic Leap, Inc.
InventorsPaul-Edouard Sarlin, Daniel DeTone, Tomasz Jan Malisiewicz, Andrew Rabinovich
Life of this patent
Estimating the term…
Abstract
The description relates the feature matching. Our approach establishes pointwise correspondences between challenging image pairs. It takes off-the-shelf local features as input and uses an attentional graph neural network to solve an assignment optimization problem. The deep middle-end matcher acts as a middle-end and handles partial point visibility and occlusion elegantly, producing a partial assignment matrix.
Claims16
A computer system comprising:
- a computer-readable medium;
- a processor connected to the computer-readable medium;
- and
- a set of instructions on the computer-readable medium, including:
- a deep middle-end matcher architecture that is executable by the processor and includes:
- an attentional graph neural network that has:
- a keypoint encoder to map keypoint positions p and their visual descriptors d into a single vector;
- and
- at least one layer that, based on the vector, creates representations f, wherein the at least one layer includes alternating self-attention and cross-attention layers that, based on the vector, repeated L times to create representations f;
- and
- an optimal matching layer that creates an M by N score matrix from the representations f and finds an optimal partial assignment based on the M by N score matrix, where:
- p are keypoint positions,
- f are representations,
- d are descriptors,
- L are a plurality of times,
- M by N is a score matrix with a length M and a width N.
13 more elements
A computer-implemented method that includes:
- creating, with an attentional graph neural network, a representation by mapping, with a keypoint encoder of the attentional graph neural network keypoint positions p and their visual descriptors d into a single vector;
- creating, with an optimal matching layer, a score from the representation by executing, with at least one layer of the attentional graph neural network, based on the vector, representations f, including executing with alternating self- and cross-attention layers of the attentional graph neural network, based on the vector, for L repeated times, to create representations f;
- and
- finding, with the optimal matching layer, an M by N score matrix from the representations f and finds an optimal partial assignment based on the M by N score matrix,
- where:
- p are keypoint positions,
- f are representations,
- L are a plurality of times,
- M by N is a score matrix with a length M and a width N.
6 more elements
Dependencies are inferred from the claim text.