Evaluating spatial relationships using vision transformers
AssigneeAmazon Technologies, Inc.
InventorsKun Li, Pragyana K. Mishra
Life of this patent
Estimating the term…
Abstract
Techniques for detection of spatial relationships are provided. An input image is divided into a set of patches, and a first feature tensor is generated for a first patch of the set of patches. The first feature tensor is processed using an attention mechanism to generate a first transformed feature tensor, and a classification indicating distancing between physical entities in the input image is generated based at least in part on the first transformed feature tensor.
Claims20
A computer-implemented method of classifying input images based on physical distancing between people, comprising:
- dividing an input image into a set of patches;
- generating a first feature tensor for a first patch of the set of patches;
- processing the first feature tensor using an attention mechanism to generate a first transformed feature tensor, comprising:
- determining a first query matrix for the first patch based on the first feature tensor and a first weight matrix;
- determining a first key matrix for the first patch based on the first feature tensor and a second weight matrix;
- determining a first value matrix for the first patch based on the first feature tensor and a third weight matrix;
- and
- computing a first attention for the first patch based at least in part on the first query matrix, a second key matrix of a second patch from the set of patches, and a second value matrix of the second patch;
- and
- generating a first classification for the first patch by processing the first transformed feature tensor using a classifier, wherein the classification relate to physical distancing between people in the input image.
A computer-implemented method, comprising:
- dividing an input image into a set of patches;
- generating a first feature tensor for a first patch of the set of patches;
- processing the first feature tensor using an attention mechanism to generate a first transformed feature tensor;
- and
- generating a classification indicating distancing between physical entities in the input image based at least in part on the first transformed feature tensor.
A non-transitory computer readable medium comprising computer readable code executable by one or more processors to perform an operation comprising:
- dividing an input image into a set of patches;
- generating a first feature tensor for a first patch of the set of patches;
- processing the first feature tensor using an attention mechanism to generate a first transformed feature tensor;
- and
- generating a classification indicating distancing between physical entities in the input image based at least in part on the first transformed feature tensor.
Dependencies are inferred from the claim text.