# US 10,372,976 B2 · Person detection in a marine environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,372,976 B2
- Document ID: US10372976B2
- Status: Granted
- Kind: B2
- Application number: 15/147,181
- Filed: 2016-05-05
- Published: 2017-11-09
- Granted: 2019-08-06
- Earliest claimed: 2016-05-05
- Assignees: Brunswick Corporation
- Applicants: Brunswick Corporation
- Inventors: Troy J. Kollmann; Jon M. Nowick
- Web page: https://patentgrep.com/patent/US10372976B2

## CPC

- G06N 3/02: Neural networks
- B63B 79/10: using sensors, e.g. pressure sensors, strain gauges or accelerometers
- B63B 79/40: for controlling the operation of vessels, e.g. monitoring their speed, routing or maintenance schedules
- B63J 99/00: Subject matter not provided for in other groups of this subclass
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60)
- G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06T 2207/30196: Human being; Person
- G06T 2207/30244: Camera pose
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- B63B: SHIPS OR OTHER WATERBORNE VESSELS; EQUIPMENT FOR SHIPPING (arrangements of vessel ventilation, heating, cooling, or air-conditioning B63J2/00; floating substructures as supports of dredgers or soil-shifting machines E02F9/06)
- B63J: AUXILIARIES ON VESSELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An object detection system for a marine vessel having at least one marine drive includes at least one image sensor positioned on the marine vessel and configured to capture an image of a marine environment on or around the marine vessel, and a processor. The object detection system further includes an image scanning module executable on the processor that receives the image as input. The image scanning module includes an artificial neural network trained to detect patterns within the image of the marine environment associated with one or more predefined objects, and to output detection information regarding a presence or absence of the one or more predefined objects within the image of the marine environment.

## Claims

### Claim 1

An object detection system for a marine vessel having at least one marine drive, the object detection system comprising:
at least one image sensor positioned on the marine vessel and configured to capture an image of a first marine environment around the marine vessel, wherein the first marine environment is a water surface behind the marine vessel and the at least one image sensor is a visible light camera positioned on the marine vessel at an angle and distance from the water surface;
a processor; and
an image scanning module executable on the processor and configured to receive the image as input and output information regarding a presence or absence of a person located in the first marine environment, the image scanning module comprising an artificial neural network trained to detect patterns within the image of the first marine environment associated with a person located behind the marine vessel, and to output detection information regarding the presence or absence of the person within the image of the first marine environment;
wherein the artificial neural network of the image scanning module is trained using positive and negative training images of objects on a water surface having a range of view angles and fields of view that approximate the angle and distance from the water surface of the image sensor positioned on the marine vessel.

### Claim 2 (depends on 1)

The object detection system of claim 1, wherein the detection information indicates a probability value of the presence of the person within the image.

### Claim 3 (depends on 1)

The object detection system of claim 1, wherein the artificial neural network of the image scanning module is trained using positive training images of persons on a water surface having the range of view angles and/or fields of view.

### Claim 4 (depends on 1)

The object detection system of claim 1, further comprising
a second image sensor positioned on the marine vessel and configured to capture a second image of a second marine environment being a second water surface around the marine vessel, wherein the second marine environment is at a different position with respect to the marine vessel than the first marine environment and the second image sensor is positioned on the marine vessel at an angle and distance from the second water surface;
a second processor; and
a second image scanning module executable on the second processor, the second image scanning module including a second artificial neural network trained to receive the image of the second marine environment as input and output second detection information regarding the presence or absence of one or more predefined objects within the second image, wherein the second artificial neural network of the second image scanning module is trained using positive and negative training images of objects on a water surface having a range of view angles and fields of view that approximate the angle and distance from the second water surface of the second image sensor positioned on the marine vessel.

### Claim 5 (depends on 4)

The system of claim 4, further comprising a powertrain control unit configured to receive the detection information from the first image scanning module and the second image scanning module and to generate a control instruction to the marine drive based on the detection information.

### Claim 6 (depends on 4)

The system of claim 4, wherein the second image sensor is a visible light camera positioned on the marine vessel and the second marine environment is a water surface of an area in front of the marine vessel, and wherein the image scanning module is further configured to output the detection information regarding the presence of at least one of the person, a predefined marine animal, or a floating debris object of a predefined size range located in the area in front of the marine vessel.

### Claim 7

A method of detecting a presence of one or more objects in a marine environment of a marine vessel, the method comprising:
determining a range of view angles and fields of view that approximate an angle and distance from a water surface of an image sensor positioned on the marine vessel;
training an artificial neural network to detect patterns associated with a predefined object in images of a marine environment imaged by the image sensor, wherein the training includes:
providing positive training images of the predefined object on a water surface from view angles or fields of view that are within the range of view angles and fields of view of the image sensor;
providing negative training images of objects other than the predefined object on a water surface from view angles or fields of view that are within the range of view angles and/or fields of view of the image sensor;
recording an image with the image sensor positioned on the marine vessel;
providing the image as input to the artificial neural network; and
outputting detection information regarding a presence or absence of the object within the image.

### Claim 8 (depends on 7)

The method of claim 7, wherein the detection information includes a probability of the presence of the predefined object in the image.

### Claim 9 (depends on 7)

The method of claim 7, further comprising sending the detection information via a CAN bus to a powertrain control unit, and controlling at least one marine drive with the powertrain control unit based on the detection information.

### Claim 10 (depends on 7)

The method of claim 7, further comprising generating an alert on the marine vessel based on the detection information.

### Claim 11 (depends on 3)

The object detection system of claim 3, wherein the artificial neural network of the image scanning module is trained using negative training images of objects other than persons on a water surface having the range of view angles and/or fields of view.

### Claim 12

A driver detection system for a marine vessel having at least one marine drive, the driver detection system comprising:
at least one image sensor positioned on the marine vessel and configured to capture an image of a helm area on the marine vessel, wherein the at least one image sensor is a visible light camera positioned to image the helm area;
a processor; and
an image scanning module executable on the processor and configured to receive the image as input and output detection information regarding a presence or absence of a person located in the helm area, the image scanning module including an artificial neural network trained to detect patterns within the image associated with a person in the helm area, and to output the detection information regarding the presence or absence of the person located in the helm area.

### Claim 13 (depends on 12)

A driver detection system of claim 12, wherein the artificial neural network of the image scanning module is trained for person detection in a particular helm area using training images of a same vessel model as the helm area or a same arrangement of vessel control input devices as the helm area.

### Claim 14 (depends on 1)

The system of claim 1, wherein the range of view angles and fields of view of the training images approximate a view angle range or distance range of the image sensor from the water surface caused by motion of the marine vessel with respect to the first marine environment.

### Claim 15 (depends on 7)

The method of claim 7, wherein the range of view angles and fields of view of the training images approximate a view angle range or distance range of the image sensor from the water surface caused by motion of the marine vessel with respect to the first marine environment.