US 10,706,335 B2Granted

Multi-perspective detection of objects

AssigneeRapiscan Laboratories, Inc.

InventorsSimanta Gautam, Brian Shimanuki, Bruno Brasil Ferrari Faviero, Brian Xie

Life of this patent

Estimating the term…

Loading patent…

Abstract

Technology disclosed herein may involve a computing system that (i) generates (a) a first feature map based on a first visual input from a first perspective of a scene utilizing at least one first neural network and (b) a second feature map based on a second visual input from a second, different perspective of the scene utilizing at least one second neural network, where the first perspective and the second perspective share a common dimension, (ii) based on the first feature map and a portion of the second feature map corresponding to the common dimension, generates cross-referenced data for the first visual input, (iii) based on the second feature map and a portion of the first feature map corresponding to the common dimension, generates cross-referenced data for the second visual input, and (iv) based on the cross-referenced data, performs object detection on the scene.

Claims20

Dependent claims

A computing system comprising:

at least one first neural network configured to accept first visual input from a first perspective of a scene;
at least one second neural network configured to accept second visual input from a second perspective of the scene that differs from the first perspective of the scene, wherein the first perspective and the second perspective share a common dimension;
at least one processor;
a non-transitory computer-readable medium;
program instructions that are stored on the non-transitory computer-readable medium that are executable by the at least one processor such that the computing system is configured to:
generate (i) a first feature map based on the first visual input utilizing the at least one first neural network and (ii) a second feature map based on the second visual input utilizing the at least one second neural network;
based on the first feature map and a portion of the second feature map corresponding to the common dimension, generate cross-referenced data for the first visual input;
based on the second feature map and a portion of the first feature map corresponding to the common dimension, generate cross-referenced data for the second visual input;
and
based on the cross-referenced data for the first visual input and for the second visual input, perform object detection on the scene.

7 more elements

A non-transitory computer-readable medium comprising program instructions stored thereon that are executable by at least one processor such that a computing system is configured to:

generate (i) a first feature map based on a first visual input from a first perspective of a scene utilizing at least one first neural network configured to accept the first visual input and (ii) a second feature map based on a second visual input from a second perspective of the scene that differs from the first perspective of the scene utilizing at least one second neural network configured to accept the second visual input, wherein the first perspective and the second perspective share a common dimension;
based on the first feature map and a portion of the second feature map corresponding to the common dimension, generate cross-referenced data for the first visual input;
based on the second feature map and a portion of the first feature map corresponding to the common dimension, generate cross-referenced data for the second visual input;
and
based on the cross-referenced data for the first visual input and for the second visual input, perform object detection on the scene.

2 more elements

A computer-implemented method comprising:

generating (i) a first feature map based on a first visual input from a first perspective of a scene utilizing at least one first neural network configured to accept the first visual input and (ii) a second feature map based on a second visual input from a second perspective of the scene that differs from the first perspective of the scene utilizing at least one second neural network configured to accept the second visual input, wherein the first perspective and the second perspective share a common dimension;
based on the first feature map and a portion of the second feature map corresponding to the common dimension, generating cross-referenced data for the first visual input;
based on the second feature map and a portion of the first feature map corresponding to the common dimension, generating cross-referenced data for the second visual input;
and
based on the cross-referenced data for the first visual input and for the second visual input, performing object detection on the scene.

2 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/658,934Filed 21 Oct 2019 · continuation
Priority: 20 Jul 2018
Published: US 2020/0050887 A113 Feb 2020
Granted: 7 Jul 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification15 codes

G06N 3/08
Neural networks: learning methods
G06F 18/253
Fusion techniques: of extracted features
G06N 3/02
Computing arrangements based on biological models: neural networks
G06N 3/045
Architecture, e.g. interconnection topology: combinations of networks

Family

Application62/701,365Us provisional application · filed 20 Jul 2018
Application16/189,300Parent of this continuation · filed 13 Nov 2018
US 2020/0050887 A1Pre-grant publication · published 13 Feb 2020
US 10,452,959 B1Parent of this continuation
US 10,706,335 B2This patent

From the US filing history. Foreign family members aren’t listed.