# US 2025/0095325 A1 · Method and apparatus for obtaining information about object using plurality of sensors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095325 A1
- Document ID: US20250095325A1
- Status: Published application
- Kind: A1
- Application number: 18/963,674
- Filed: 2024-11-28
- Published: 2025-03-20
- Earliest claimed: 2022-12-30
- Assignees: Avikus Co., Ltd.
- Applicants: Avikus Co., Ltd.
- Inventors: Dae Yong HAN; Joontae Hwang
- Web page: https://patentgrep.com/patent/US20250095325A1

## CPC

- B63B 43/18: preventing collision {or grounding}; reducing collision damage {(reducing pollution by collision B63B25/082)}
- G06V 10/40: Extraction of image or video features
- G01C 21/20: Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence)
- G01C 21/203: {specially adapted for water-borne vessels}
- G01S 13/867: {Combination of radar systems with cameras}
- G01S 13/937: of marine craft
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/93: for anti-collision purposes
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/80: Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10)
- G08G 3/02: Anti-collision systems
- B63B 49/00: Arrangements of nautical instruments or navigational aids (nautical measuring instruments G01C; radio navigation, analogous arrangements using other waves G01S)
- B63B 79/40: for controlling the operation of vessels, e.g. monitoring their speed, routing or maintenance schedules
- G06V 10/82: using neural networks
- B63B: SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING (arrangements of vessel ventilation, heating, cooling, or air-conditioning B63J2/00; floating substructures as supports of dredgers or soil-shifting machines E02F9/06)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

A navigation assistance system using a plurality of sensors according to an aspect of the present disclosure includes a display configured to provide a monitoring image, a sensor unit including at least one sensor, a memory in which at least one program is stored, and at least one processor configured to execute the at least one program, wherein the at least one processor obtains image frame information of an object using a first sensor, calculates information about the object using a second sensor, calculates a risk of collision between a host ship and the object by fusing the image frame information of the object and the information about the object, and determines an avoidance path based on the calculated collision risk.

## Claims

### Claim 1

A method of obtaining information about an object using a plurality of sensors, the method comprising:
obtaining an image frame in real time using a first sensor;
forming a bounding box comprising at least one object within the image frame;
generating a multi-dimensional object area corresponding to the at least one object using a projection surface to which the bounding box is projected;
obtaining data included in the at least one object area using a second sensor; and
calculating information about the at least one object based on the obtained data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the forming the bounding box comprises:
inputting the obtained image frame into an object specifying model; and
forming the bounding box specifying the object by using the object specifying model, wherein
the object specifying model is trained using at least one of logging data and a generative artificial intelligence model.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
tracking image frames obtained in real time by using a tracking algorithm; and
determining whether objects included in the frames are the same object based on the association between objects included in the image frames.

### Claim 4 (depends on 1)

The method of claim 1, wherein
the first sensor is a camera, and
the second sensor is a Lidar device.

### Claim 5 (depends on 1)

The method of claim 1, wherein
the calculating information about the object comprises calculating information about the object based on a median value of point data obtained using the second sensor.

### Claim 6 (depends on 1)

The method of claim 1, wherein in a case where the object included in the object area comprises a plurality of objects, the operation of calculating information about the object comprises:
obtaining point data of the plurality of objects using the second sensor; and
calculating information about an object close to a host ship based on a median value of dominant point data among the point data of the plurality of objects obtained.

### Claim 7 (depends on 5)

The method of claim 5, wherein the median value of the point data is calculated using a preset number or more of the point data.

### Claim 8 (depends on 1)

The method of claim 1, wherein the information of the object comprises at least one of a distance and a bearing angle of the object.

### Claim 9 (depends on 8)

The method of claim 8, further comprising at least one of:
displaying the at least one of the distance and the bearing angle of the object; and
providing a monitoring image displaying the at least one of the distance and the bearing angle of the object.

### Claim 10 (depends on 1)

The method of claim 1, wherein
the first sensor is at least one of an EO camera and an IR camera, and
the second sensor is at least one of a Radar device and an automatic identification system (AIS).

### Claim 11 (depends on 10)

The method of claim 10, wherein the obtained image frame is an image obtained by fusing individual images obtained using the EO camera and the IR camera.

### Claim 12 (depends on 10)

The method of claim 10, wherein the data obtained using the second sensor is fused data of point data obtained using the Radar device and object data obtained using the AIS.

### Claim 13 (depends on 12)

The method of claim 12, further comprising
providing a monitoring image displaying at least one of a distance, bearing angle, and speed of the object using the fused data.

### Claim 14 (depends on 1)

The method of claim 1, wherein a depth of the object area is determined based on at least one of a type of the second sensor, a length of the ship, and environmental information.

### Claim 15

A navigation assistance system using a plurality of sensors, comprising:
a display configured to provide a monitoring image;
a sensor unit comprising at least one sensor;
a memory in which at least one program is stored; and
at least one processor configured to execute the at least one program,
wherein the at least one processor is configured to
obtain image frame information of an object using a first sensor,
calculate information about the object using a second sensor from a multi-dimensional object area generated based on the image frame information of the object,
calculate a risk of collision between a host ship and the object by fusing the image frame information of the object and the information about the object, and
determine an avoidance path based on the calculated collision risk.
