Back to results
US 12,614,284 B2Granted
Class agnostic object mask generation
AssigneeNVIDIA Corporation
InventorsShiyi Lan, Zhiding Yu, Subhashree Radhakrishnan, Jose Manuel Alvarez Lopez, Animashree Anandkumar
Life of this patent
Estimating the term…
Abstract
Class agnostic object mask generation uses a vision transformer-based auto-labeling framework requiring only images and object bounding boxes to generate object (segmentation) masks. The generated object masks, images, and object labels may then be used to train instance segmentation models or other neural networks to localize and segment objects with pixel-level accuracy.
Claims20
A method of generating a binary mask, comprising:
- obtaining a cropped image region including an object that is associated with a label;
- expanding the cropped image region to provide an expanded image region including at least one row or column of pixels that exclude the object;
- and
- applying parameters to the expanded image region by a neural network to predict the binary mask corresponding to the object, wherein the neural network comprises:
- an image encoder configured to process the expanded image region according to a first portion of the parameters to compute an encoded image region;
- and
- an attention-based decoder that processes the encoded image region according to a second portion of the parameters to produce the binary mask, wherein the attention- based decoder comprises:
- a first processing path comprising a max-pooling layer followed by a fully connected layer that generates a first output;
- a second processing path comprising multiple convolutional layers that generate a second output;
- and
- an operator that combines the first output and the second output to compute the binary mask.
8 more elements
A system, comprising:
- a memory that stores an image including an object that is associated with a label;
- and
- a processor that is connected to the memory, wherein the processor is configured to generate a binary mask by:
- obtaining, from the image, a cropped image region that includes the object;
- expanding the cropped image region to provide an expanded image region including at least one row or column of pixels that exclude the object;
- and
- applying parameters to the expanded image region by a neural network to predict the binary mask corresponding to the object, wherein the neural network comprises:
- an image encoder configured to process the expanded image region according to a first portion of the parameters to compute an encoded image region;
- and
- an attention-based decoder that processes the encoded image region according to a second portion of the parameters to produce the binary mask, wherein the attention- based decoder comprises:
- a first processing path comprising a max-pooling layer followed by a fully connected layer that generates a first output;
- a second processing path comprising multiple convolutional layers that generate a second output;
- and
- an operator that combines the first output and the second output to compute the binary mask.
11 more elements
A non-transitory computer-readable medium storing computer instructions for generating a binary mask that, when executed by one or more processors, cause the one or more processors to perform the steps of:
- obtaining a cropped image region including an object that is associated with a label;
- expanding the cropped image region to provide an expanded image region including at least one row or column of pixels that exclude the object;
- and applying parameters to the expanded image region by a neural network to predict the binary mask corresponding to the object, wherein the neural network comprises:
- an image encoder configured to process the expanded image region according to a first portion of the parameters to compute an encoded image region;
- and
- an attention-based decoder that processes the encoded image region according to a second portion of the parameters to produce the binary mask, wherein the attention-based decoder comprises:
- a first processing path comprising a max-pooling layer followed by a fully connected layer that generates a first output;
- a second processing path comprising multiple convolutional layers that generate a second output;
- and
- an operator that combines the first output and the second output to compute the binary mask.
7 more elements
Dependencies are inferred from the claim text.