US 2022/0012494 A1Published application

Intelligent multi-visual camera system and method

AssigneeTerraClear Inc.

InventorsIsabelle Butterfield, Paul Barsic, Alex Terekhov

Life of this patent

Estimating the term…

Loading patent…

Abstract

An intelligent multi-visual camera system is disclosed that includes a multiple visual sensor array and a control system. The multiple visual sensor array includes multiple visual cameras spaced apart from each other. The control system also receives input from the multiple visual cameras, stores instructions that cause the processor to: initiate a registration system that projects images from the multiple visual cameras into a single three-dimensional volume and coordinate overlapping pixels of adjacent images amongst each other; detect one or more features of interest in the images from the multiple visual cameras; track one or more features of interest in one frame in one image from the multiple visual cameras into a subsequent frame in a subsequent image from the multiple visual cameras; deduplicate the projected images from the multiple visual cameras onto the ground plane; and communicate information regarding the features of interest that have been detected.

Claims48

Dependent claims

A method of using an intelligent multi-visual camera system, the method comprising:

providing a multiple visual sensor array including multiple visual cameras spaced apart from each other, wherein each of the multiple visual cameras is mounted on a support frame, wherein each camera acquires its own independent image;
initiating a registration system that projects images from the multiple visual cameras into a single three-dimensional volume;
detecting features of interest in the images from the multiple visual cameras using a neural network, wherein detection of features of interest includes classification, pixel location in the image, and boundary representation in the image;
identifying a feature of interest in one frame in an image from the multiple visual cameras, and identifying the same feature of interest in a subsequent frame in an image from the multiple visual cameras, which provides improved computational speed and persistent feature association over sequential images;
projecting the features of interest from the multiple visual cameras into the three-dimensional volume to identify instances where the same objects have been identified by multiple cameras and ensure unique features of interest;
predicting a presence of an object in the absence of object detection by logging positions of objects over time;
employing a logging system that uses a feedback loop to assess quality of collected data;
and
displaying collected imagery as a single synthesized view of camera feed overlap from the multiple visual cameras with overlaying boxes on the displayed collected imagery identifying objects of interest.

6 more elements

A method of using an intelligent multi-visual camera system for assisting robotics by sensing, detecting, and communicating information about objects in an environment, the method comprising:

providing a multiple visual sensor array including multiple visual cameras spaced apart from each other, wherein each of the multiple visual cameras is mounted on a support frame, and wherein each camera acquires its own independent image;
initiating a registration system that projects images from the multiple visual cameras into a single three-dimensional volume;
detecting one or more features of interest in the images from the multiple visual cameras, wherein detection of features of interest includes classification, pixel location in the image, and boundary representation in the image;
tracking one or more features of interest in one frame in an image from the multiple visual cameras, and identifying the same feature of interest in a subsequent frame in an image from the multiple visual cameras;
projecting the features of interest from the multiple visual cameras into the three-dimensional volume to identify instances where the same objects have been identified by multiple cameras and ensure unique features of interest;
and
communicating information regarding the one or more features of interest that have been detected.

4 more elements

An intelligent multi-camera system, the system comprising:

a multiple sensor array including multiple cameras spaced apart from each other, wherein each of the multiple cameras is mounted on a support frame, and wherein each camera acquires its own independent image;
and
a control system that receives input from the multiple cameras, the control system including a processor and a memory storing computer instructions that, when executed by the processor, cause the processor to:
initiate a registration system that projects images from the multiple cameras into a single three-dimensional volume;
detect one or more features of interest in the images from the multiple cameras, wherein detection of features of interest includes classification, pixel location in the image, and boundary representation in the image;
track one or more features of interest in one frame in an image from the multiple cameras, and identify the same one or more features of interest in a subsequent frame in an image from the multiple cameras;
project the features of interest from the multiple cameras into the three-dimensional volume to identify instances where the same objects have been identified by multiple cameras and ensure unique features of interest;
and
communicate information regarding the features of interest that have been detected.

6 more elements

Dependencies are inferred from the claim text.