# US 2015/0104064 A1 · Method and system for detection of foreign objects in maritime environments

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0104064 A1
- Document ID: US20150104064A1
- Status: Published application
- Kind: A1
- Application number: 14/401,180
- Filed: 2013-05-13
- Published: 2015-04-16
- Earliest claimed: 2012-05-15
- Assignees: Dvp Technologies Ltd.
- Applicants: Dvp Technologies Ltd.
- Inventors: Rami Guissin; Eitan Lavi; Akiva Stolovich; Sharon Bason
- Web page: https://patentgrep.com/patent/US20150104064A1

## CPC

- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/32: using correlation-based methods
- G08G 3/02: Anti-collision systems
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10048: Infrared image
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/30232: Surveillance
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

The present invention provides techniques for detecting foreign objects in a region of interest in maritime environments. Image data indicative of a sequence of successively acquired images of the region of interest, is analyzed to determine candidate points of interest, and data indicative of said points is processed to identify candidate points that are adjacently accumulated in different locations in images data. Grouping data may be then generated based on the identified accumulations of candidate points indicative of a group of said candidate points. The grouping data is processed to identify spatio-temporal correlation between the points in the group and determine a corresponding track function, thereby enabling detection of a presence of a foreign object in the image data.

## Claims

### Claim at position 1

1-26. (canceled)

### Claim 27

A method for detecting foreign objects in a region of interest in maritime environment, the method comprising:
receiving image data indicative of a sequence of successively acquired images of the region of interest, said images having a cluttered background, and analyzing each image in said sequence of images to determine candidate points of interest therein, and generating data indicative thereof;
processing the data indicative of the candidate points in the image data, and identifying the candidate points adjacently accumulated in different locations in two or more of the consecutive images, and generating grouping data indicative of a group of said candidate points; and
processing the grouping data, said processing comprising identifying spatio-temporal correlation between the points in the group and determining paths formed by the locations of the grouped points in the images, said grouping and determination of the path determining a track function corresponding to the path of the grouped points' locations and being configured to overcome intermittent obstruction of the foreign objects due to the cluttered background of the acquired sequence of images, thereby enabling detection of a presence of a foreign object.

### Claim 28 (depends on 27)

The method of claim 27, wherein the analyzing of the image data for detection of candidate points comprises filtering and thresholding the image data.

### Claim 29 (depends on 28)

The method of claim 28, wherein the filtering comprises applying to the image data one or more of the following operations: noise removal; removal of artifacts; and matched filtering.

### Claim 30 (depends on 27)

The method of claim 27, wherein the identifying of one or more groups of the candidate points comprises determining a region in the images encompassing a majority of the candidate points of said group.

### Claim 31 (depends on 30)

The method of claim 30, wherein said determination of the spatio-temporal correlation comprises identifying two or more of the groups of points for which locations of the points in the images is indicative of a substantially continuous path.

### Claim 32 (depends on 27)

The method of claim 27, wherein the determining of track function comprises identifying micro tracks indicative of formation of short term tracks in the received image data and generating data indicative thereof, and based on said data identifying macro tracks indicative of formation of long term tracks by said micro tracks, and generating data indicative thereof.

### Claim 33 (depends on 27)

The method of claim 27, wherein said analyzing of the sequence of images comprises:
applying to each analyzed image one or more decimation function and generating respective decimated image data;
computing signal to clutter ratio for each pixel in said analyzed image and for each pixel in said decimated image data;
processing the generated signal to clutter ratios and generating therefrom one or more respective binary image data for said analyzed image and for said respective decimated image data according to a thresholding criterion; and
interpolating binary image data associated with said decimated image data, and generating data indicative of interpolated image data.

### Claim 34 (depends on 33)

The method of claim 33, wherein said generation of the data indicative of the interpolated image data further comprises applying an OR logic function to the binary image data obtained for the analyzed image and for the interpolated binary images associated with decimated image data.

### Claim 35 (depends on 33)

The method of claim 33, wherein said generation of the decimated image data comprises applying to each image two or more of the decimation functions having different decimation factors, said decimated image data thereby comprising two or more respective decimated image data pieces.

### Claim 36 (depends on 33)

The method of claim 33, comprising carrying out the following operations:
applying spatial blobbing function to the data indicative of the interpolated image data thereby obtaining segmented blob maps comprising blob data associated with points appearing in the image;
filtering the segmented blob maps and generating respective spatiotemporal geometric centers for each blob in said maps, said geometric centers corresponding to the candidate points; and
generating binary images indicative of said spatiotemporal geometric centers.

### Claim 37 (depends on 36)

The method of claim 36, wherein said operations are applied to the binary image data associated with the analyzed image and with the interpolated image data resulting from said interpolation of the binary images associated with said decimated image data.

### Claim 38 (depends on 36)

The method of claim 36, wherein said operations are applied to the data resulting from the application of the OR logic function.

### Claim 39 (depends on 36)

The method of claim 36, comprising generating micro tracks by carrying out the following operations:
identifying the groups of adjacently located spatiotemporal geometric centers; applying a smoothing function to said groups of adjacently located spatiotemporal geometric centers and morphological operators to generate respective blobs; and
mapping each blob to an oriented ellipse and determining slope and length of an elongated axis thereof and generating micro track data accordingly.

### Claim 40 (depends on 39)

The method of claim 39, comprising correlating micro track data obtained for the different blobs to determine formation of continuous paths and generating macro track data indicative thereof.

### Claim 41 (depends on 27)

The method of claim 27, comprising analyzing at least one of the grouping data and the track data and selectively generating an alert signal upon identifying that said data is indicative of presence of a possible threat.

### Claim 42 (depends on 41)

The method of claim 41, comprising:
responsive to the alert signal,
determining a specific area in the region of interest according to at least one of the grouping data and the track data; and
receiving one or more images acquired form said specific area.

### Claim 43 (depends on 27)

The method of claim 27 comprising using the track function to identify and discard false tracks of objects sporadically appearing in the images for short time durations due to the cluttered background.

### Claim 44 (depends on 27)

A computerized system for carrying out the method of claim 27 for detecting foreign objects in a region of interest in maritime environments.

### Claim 45

A computerized system for use in detecting foreign objects in a region of interest in maritime environments, the system comprising:
data input utility for receiving image data of two or more images successively acquired from said region of interest, said images having a cluttered background;
one or more processors and memory configured and operable to receive and analyze said image data using one or more modules stored in said memory, comprising:
a plot detection module for identifying one or more groups of candidate points adjacently accumulated in regions in said two or more images and generating data indicative thereof; and
a track detection module for determining correlations between the one or more groups of candidate points for determining formation of continuous paths formed by the locations of said one or more groups of candidate points in the images, and determining a track function corresponding to the paths of the one or more groups of candidate points to overcome intermittent obstruction of the foreign objects due to the cluttered background of the acquired sequence of images, and generating track data accordingly.

### Claim 46 (depends on 45)

The system of claim 45, comprising an image filtering module for applying to the image data one or more of the following operations: noise removal; removal of artifacts; and matched filtering.

### Claim 47 (depends on 45)

The system of claim 45, comprising a decision making module for determining presence of a foreign object based on the generated track data.

### Claim 48 (depends on 45)

The system of claim 45, wherein the plot detection module comprises:
a decimation utility for decimating the received image data and generating respective decimated image data;
a spatial filter utility for determining a signal to clutter ratio for each pixel in the analyzed image data and for each pixel in the decimated image data, and generating pixel data indicative thereof;
a threshold utility for generating binary images for said analyzed image data and said decimated image data based on said pixel data and a thresholding criterion; and
an interpolation utility for interpolating binary images associated with said decimated image data.

### Claim 49 (depends on 48)

The system of claim 48, wherein the plot detection module comprises OR function utility for applying an OR logic function to the binary image data associated with the analyzed image data and the interpolated binary images, and generating data indicative thereof.

### Claim 50 (depends on 48)

The system of claim 48, wherein the plot detection module comprises:
a blob analysis utility for processing the binary image data, or the data obtained from the OR function utility, and generating blob maps comprising blob data associated with points appearing in the received images;
a temporal filtering utility for filtering the segmented blob maps and generating respective spatiotemporal geometric centers for each blob in said maps;
a summation utility for accumulating geometric centers generated for the received images and generating summation data indicative thereof; and
a plot identification utility for identifying the candidate points based on the geometric centers accumulated in said summation data.

### Claim 51 (depends on 45)

The system of claim 45, wherein the track detection module comprises a micro track detection unit, comprising:
a blobbing utility for identifying groups of adjacently located candidate points and generating respective blobs to each one of said groups; and
a mapping utility for fitting to each blob an oriented ellipse and generating micro track data accordingly.

### Claim 52 (depends on 51)

The system of claim 51, wherein the track detection module comprises a macro track detection unit for correlating micro track data obtained for the different blobs to determine formation of continuous paths and generating macro track data indicative thereof.

### Claim 53

A computerized system for use in detecting foreign objects in a region of interest in maritime environments, the system comprising:
data input for receiving image data of two or more images successively acquired from said region of interest, said images having a cluttered background;
a filtering arrangement for filtering out disturbances from said images;
image processing unit for identifying candidate points of interest in the filtered images;
a spatio-temporal correlation unit for indentifying groups of such candidate points which are adjacently located in regions of two or more consecutive images of said sequence of images;
a track detection module for determining correlations between the groups of candidate points for determining formation of continuous paths formed by the locations of said one or more groups of candidate points in the images, determining a track function corresponding to the paths of the groups of candidate points to overcome intermittent obstructions of the foreign objects due to the cluttered background of the acquired sequence of images, and generating track data accordingly; and
a decision making unit for determining presence of a foreign object based on the generated track data.
