Ventral-dorsal neural networks: object detection via selective attention
AssigneeAncestry.com Operations Inc.
InventorsMohammad K. Ebrahimpour, Yen-Yun Yu, Jiayun Li, Jack Reese, Azadeh Moghtaderi
Life of this patent
Estimating the term…
Abstract
Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.
Claims20
A computer-implemented method for object detection within a visual medium, comprising:
- receiving a visual medium comprising a plurality of objects;
- identifying, via a first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium, comprising:
- identifying, via a sensitivity analysis, pixels within the visual medium that are above a predetermined threshold, wherein the pixels above the predetermined threshold define the one or more relevant visual regions;
- computing a measure of relevance for a plurality of pixel locations;
- generating, based at least on the measure of relevance for the plurality of pixel locations, a visual mask comprising a data structure containing pixel values;
- applying the visual mask to modify pixel intensity values of the one or more irrelevant visual regions to generate a masked visual medium;
- identifying, via a second neural network, one or more objects of interest within the masked visual medium;
- and
- outputting an identification of the one or more objects of interest.
A non-transitory computer-readable storage medium having stored thereon instructions for causing at least one computer system to detect objects within a visual medium, the instructions comprising:
- receiving a visual medium comprising a plurality of objects;
- identifying, via a first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium, comprising:
- identifying, via a sensitivity analysis, pixels within the visual medium that are above a predetermined threshold, wherein the pixels above the predetermined threshold define the one or more relevant visual regions;
- computing a measure of relevance for a plurality of pixel locations;
- generating, based at least on the measure of relevance for the plurality of pixel locations, a visual mask comprising a data structure containing pixel values;
- applying the visual mask to modify pixel intensity values of the one or more irrelevant visual regions to generate a masked visual medium;
- identifying, via a second neural network, one or more objects of interest within the masked visual medium;
- and
- outputting an identification of the one or more objects of interest.
A system for detecting objects within a visual medium, comprising:
- one or more processors;
- and
- a memory coupled with the one or more processors, the memory configured to store instructions that when executed by the one or more processors cause the one or more processors to:
- receive a visual medium comprising a plurality of objects;
- identify, via a first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium, comprising:
- identifying, via a sensitivity analysis, pixels within the visual medium that are above a predetermined threshold, wherein the pixels above the predetermined threshold define the one or more relevant visual regions;
- computing a measure of relevance for a plurality of pixel locations;
- generating, based at least on the measure of relevance for the plurality of pixel locations, a visual mask comprising a data structure containing pixel values;
- apply the visual mask to modify pixel intensity values of the one or more irrelevant visual regions to generate a masked visual medium;
- identify, via a second neural network, one or more objects of interest within the masked visual medium;
- and
- output an identification of the one or more objects of interest.
Dependencies are inferred from the claim text.