# US 2021/0174083 A1 · Ventral-dorsal neural networks: object detection via selective attention

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174083 A1
- Document ID: US20210174083A1
- Status: Published application
- Kind: A1
- Application number: 17/178,822
- Filed: 2021-02-18
- Published: 2021-06-10
- Earliest claimed: 2018-09-21
- Assignees: Ancestry.com Operations Inc.
- Applicants: Ancestry.com Operations Inc.
- Inventors: Mohammad K. Ebrahimpour; Yen-Yun Yu; Jiayun Li; Jack Reese; Azadeh Moghtaderi
- Web page: https://patentgrep.com/patent/US20210174083A1

## CPC

- G06N 3/08: Learning methods
- G06F 17/15: Correlation function computation {including computation of convolution operations (arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00)}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.

## Claims

### Claim 1

A computer-implemented method for object detection within a visual medium, comprising:
receiving the visual medium comprising one or more objects;
identifying, using a first neural network, one or more irrelevant visual regions within the visual medium;
generating, based on the one or more irrelevant visual regions, a visual mask to be applied to the visual medium;
applying the visual mask to the visual medium to generate a masked visual medium, wherein applying the visual mask to the visual medium causes the one or more irrelevant visual regions to be filtered out from the visual medium;
identifying, using a second neural network, one or more objects of interest within the masked visual medium; and
outputting an identification of the one or more objects of interest.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein applying the visual mask to the visual medium includes modifying pixel intensity values associated with the one or more irrelevant visual regions.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein pixel intensity values associated with one or more relevant visual regions are non-zero after applying the visual mask to the visual medium.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, wherein the pixel intensity values associated with the one or more irrelevant visual regions are zero after applying the visual mask to the visual medium.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the visual mask comprises a data structure containing pixel values.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the first neural network is a deep convolutional attention based object detection neural network.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the second neural network is a supervised object detection neural network.

### Claim 8

A non-transitory computer-readable storage medium having stored thereon instructions for causing at least one computer system to detect objects within a visual medium, the instructions comprising:
receiving the visual medium comprising one or more objects;
identifying, using a first neural network, one or more irrelevant visual regions within the visual medium;
generating, based on the one or more irrelevant visual regions, a visual mask to be applied to the visual medium;
applying the visual mask to the visual medium to generate a masked visual medium, wherein applying the visual mask to the visual medium causes the one or more irrelevant visual regions to be filtered out from the visual medium;
identifying, using a second neural network, one or more objects of interest within the masked visual medium; and
outputting an identification of the one or more objects of interest.

### Claim 9 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein applying the visual mask to the visual medium includes modifying pixel intensity values associated with the one or more irrelevant visual regions.

### Claim 10 (depends on 9)

The non-transitory computer-readable storage medium of claim 9, wherein pixel intensity values associated with one or more relevant visual regions are non-zero after applying the visual mask to the visual medium.

### Claim 11 (depends on 9)

The non-transitory computer-readable storage medium of claim 9, wherein the pixel intensity values associated with the one or more irrelevant visual regions are zero after applying the visual mask to the visual medium.

### Claim 12 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein the visual mask comprises a data structure containing pixel values.

### Claim 13 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein the first neural network is a deep convolutional attention based object detection neural network.

### Claim 14 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein the second neural network is a supervised object detection neural network.

### Claim 15

A system for detecting objects within a visual medium, comprising:
one or more processors; and
a memory coupled with the one or more processors, the memory configured to store instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:
receiving the visual medium comprising one or more objects;
identifying, using a first neural network, one or more irrelevant visual regions within the visual medium;
generating, based on the one or more irrelevant visual regions, a visual mask to be applied to the visual medium;
applying the visual mask to the visual medium to generate a masked visual medium, wherein applying the visual mask to the visual medium causes the one or more irrelevant visual regions to be filtered out from the visual medium;
identifying, using a second neural network, one or more objects of interest within the masked visual medium; and
outputting an identification of the one or more objects of interest.

### Claim 16 (depends on 15)

The system of claim 15, wherein applying the visual mask to the visual medium includes modifying pixel intensity values associated with the one or more irrelevant visual regions.

### Claim 17 (depends on 16)

The system of claim 16, wherein pixel intensity values associated with one or more relevant visual regions are non-zero after applying the visual mask to the visual medium.

### Claim 18 (depends on 16)

The system of claim 16, wherein the pixel intensity values associated with the one or more irrelevant visual regions are zero after applying the visual mask to the visual medium.

### Claim 19 (depends on 15)

The system of claim 15, wherein the first neural network is a deep convolutional attention based object detection neural network.

### Claim 20 (depends on 15)

The system of claim 15, wherein the second neural network is a supervised object detection neural network.
