# US 2019/0094356 A1 · Methods and apparatuses for constructing a 3d sonar image of objects in an underwater environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0094356 A1
- Document ID: US20190094356A1
- Status: Published application
- Kind: A1
- Application number: 16/205,889
- Filed: 2018-11-30
- Published: 2019-03-28
- Earliest claimed: 2015-04-20
- Assignees: Navico Holding AS
- Applicants: Navico Holding AS
- Inventors: Matthew Hunt; Shahzad F. Kirmani
- Web page: https://patentgrep.com/patent/US20190094356A1

## CPC

- G01S 15/89: for mapping or imaging
- G01S 7/539: using analysis of echo signal for target characterisation; Target signature; Target cross-section
- G01S 7/6245: {Stereoscopic displays; Three-dimensional displays; Pseudo-three dimensional displays}
- G01S 7/6272: {producing cursor lines and indicia by electronic means}
- G01S 15/87: Combinations of sonar systems
- G01S 15/8902: {Side-looking sonar}
- G01S 15/96: for locating fish
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

Methods, apparatuses, and computer program products are therefore provided for producing a 3D image of an underwater environment. An example method for providing an image of an underwater environment includes analyzing sonar returns to identify and display objects, such as fish or debris, in a 3D view. Such an image allows for differentiation between the sea floor and objects in a 3D sonar view. Some example methods provide for real-time tracking of fish. Further, the fish or other objects may be displayed over a continuous surface geometry based on sonar returns from a lake, sea, or river “floor.”

## Claims

### Claim 1

A method for imaging an underwater environment, the method comprising:
generating, by a sonar signal processor, a 3D matrix based on sonar return data received from the underwater environment, wherein the 3D matrix comprises a plurality of sonar returns each defined by 3D positional values;
determining, based on at least the 3D positional values of each of the plurality of sonar returns, one or more groups of sonar returns among the plurality of sonar returns in the 3D matrix associated with one or more objects in the underwater environment, wherein the one or more objects are different than a bottom surface of the underwater environment, wherein determining the one or more groups of sonar returns associated with the one or more objects in the underwater environment comprises:
generating, based on the 3D positional values of each of the plurality of sonar returns, a 3D representation of all of the plurality of sonar returns in the 3D matrix such that each of the plurality of sonar returns defines an x coordinate, a y coordinate, and a z coordinate; and
determining one or more clusters among the plurality of sonar returns in the 3D representation, wherein the one or more clusters among the plurality of sonar returns in the 3D representation each comprise sonar returns that are within a threshold distance of each other within the 3D representation; and
generating a 3D image of the one or more groups of sonar returns associated with the one or more objects in the underwater environment.

### Claim 2 (depends on 1)

The method of claim 1 further comprising displaying the 3D image of the one or more groups of sonar returns associated with the one or more objects in the underwater environment.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining the one or more groups of sonar returns associated with the one or more objects in the underwater environment further comprises:
determining which corresponds to the bottom surface of the underwater environment by determining the cluster with the largest number of sonar returns; and
removing the cluster that corresponds to the bottom surface of the underwater environment from the 3D representation.

### Claim 4 (depends on 3)

The method of claim 3, wherein determining the one or more groups of sonar returns associated with the one or more objects in the underwater environment further comprises:
removing any sonar returns from the 3D representation that include a z coordinate below a corresponding z coordinate of the cluster that corresponds to the bottom surface of the underwater environment.

### Claim 5 (depends on 3)

The method of claim 3, further comprising:
generating a 3D image of the cluster that corresponds to the bottom surface of the underwater environment; and
generating a 3D image of the underwater environment based on the 3D image of the cluster that corresponds to the bottom surface of the underwater environment and the 3D image of the one or more groups of sonar returns associated with the one or more objects.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
generating a second 3D matrix based on second sonar return data received from the underwater environment;
determining one or more second groups of sonar returns among a plurality of sonar returns in the second 3D matrix associated with one or more objects in the underwater environment; and
updating the 3D image of the underwater environment by updating the 3D image of one or more groups of sonar returns associated with the one or more objects in the underwater environment without updating the 3D image of the cluster that corresponds to the bottom surface.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining the one or more groups of sonar returns associated with the one or more objects in the underwater environment further comprises:
determining at least one of a shape, a size, and a direction of movement of an object associated with each cluster.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining the one or more groups of sonar returns associated with the one or more objects in the underwater environment further comprises:
generating a first 2D representation of each sonar return in at least one cluster, wherein the first 2D representation includes the x coordinate and the y coordinate of each sonar return in the at least one cluster;
generating a second 2D representation of each sonar return in at least one cluster, wherein the second 2D representation includes the x coordinate and the z coordinate of each sonar return in the at least one cluster;
generating a third 2D representation of each sonar return in at least one cluster, wherein the third 2D representation includes the y coordinate and the z coordinate of each sonar return in the at least one cluster; and
analyzing the first 2D representation, the second 2D representation, and the third 2D representation to determine one or more characteristics of the at least one cluster, wherein the one or more characteristics of the at least one cluster include at least one of a shape, a size, and a direction of movement of an object associated with each cluster.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
analyzing the one or more groups of sonar returns associated with the one or more objects to determine movement of the one or more objects; and
generating the 3D image of the one or more sonar returns associated with the one or more objects by generating a trail that corresponds to the movement of the one or more objects.

### Claim 10 (depends on 1)

The method of claim 1, wherein generating the 3D image of the one or more groups of sonar returns associated with the one or more objects comprises generating one or more mesh images characterizing the respective one or more objects based on the sonar returns associated with the corresponding object.

### Claim 11 (depends on 1)

The method of claim 1, wherein generating the 3D image of the one or more groups of sonar returns associated with the one or more objects comprises positioning one or more predefined icons for the respective one or more objects, wherein each predefined icon is positioned within the underwater environment at a location corresponding to the sonar returns associated with the respective object.

### Claim 12 (depends on 1)

The method of claim 1, wherein generating the 3D image of the one or more groups of sonar returns associated with the one or more objects comprises generating raw sonar return data associated with the sonar returns associated with the one or more objects.

### Claim 13 (depends on 1)

The method of claim 1, wherein the 3D positional values of each sonar return comprise a distance value, an angle, and a position of capture of the sonar return.

### Claim 14

An apparatus comprising a sonar signal processor and a memory including computer program code, the memory and the computer program code configured to, with the sonar signal processor, cause the apparatus to:
generate a 3D matrix based on sonar return data received from an underwater environment, wherein the 3D matrix comprises a plurality of sonar returns defined by 3D positional values;
determine, based on the 3D positional values of each of the sonar returns, one or more groups of sonar returns among the plurality of sonar returns in the 3D matrix associated with one or more objects in the underwater environment, wherein the one or more objects are different than a bottom surface of the underwater environment, wherein the determination of the one or more groups of sonar returns associated with the one or more objects in the underwater environment comprises:
generating, based on the 3D positional values of each of the plurality of sonar returns, a 3D representation of all of the plurality of sonar returns in the 3D matrix such that each of the plurality of sonar returns defines an x coordinate, a y coordinate, and a z coordinate; and
determining one or more clusters among the plurality of sonar returns in the 3D representation, wherein the one or more clusters among the plurality of sonar returns in the 3D representation each comprise sonar returns that are within a threshold distance of each other within the 3D representation; and
generate a 3D image of the one or more groups of sonar returns associated with the one or more objects in the underwater environment.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the memory and the computer program code are further configured to, with the sonar signal processor, cause the apparatus to determine the one or more groups of sonar returns associated with the one or more objects in the underwater environment by:
determining which corresponds to the bottom surface of the underwater environment by determining the cluster with the largest number of sonar returns; and
removing the cluster that corresponds to the bottom surface of the underwater environment from the 3D representation.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the memory and the computer program code are further configured to, with the sonar signal processor, cause the apparatus to determine the one or more groups of sonar returns associated with the one or more objects in the underwater environment by:
removing any sonar returns from the 3D representation that include a z coordinate below a corresponding z coordinate of the cluster that corresponds to the bottom surface of the underwater environment.

### Claim 17 (depends on 15)

The apparatus of claim 15, wherein the memory and the computer program code are further configured to, with the sonar signal processor, cause the apparatus to:
generate a 3D image of the cluster that corresponds to the bottom surface of the underwater environment; and
generate a 3D image of the underwater environment based on the 3D image of the cluster that corresponds to the bottom surface of the underwater environment and the 3D image of the one or more groups of sonar returns associated with the one or more objects.

### Claim 18 (depends on 14)

The apparatus of claim 14, wherein the memory and the computer program code are further configured to, with the sonar signal processor, cause the apparatus to:
generate a second 3D matrix based on second sonar return data received from the underwater environment;
determine one or more second groups of sonar returns among a plurality of sonar returns in the second 3D matrix associated with one or more objects in the underwater environment; and
update the 3D image of the underwater environment by updating the 3D image of one or more groups of sonar returns associated with the one or more objects in the underwater environment without updating the 3D image of the cluster that corresponds to the bottom surface.

### Claim 19

A computer program product comprising a non-transitory computer readable medium having program code portions stored thereon, the program code portions configured, when said program product is run with a sonar signal processor, to:
generate a 3D matrix based on a sonar return data received from an underwater environment, wherein the 3D matrix comprises a plurality of sonar returns defined by 3D positional values;
determine, based on the 3D positional values of each of the plurality of sonar returns, one or more groups of sonar returns among the plurality of sonar returns in the 3D matrix associated with one or more objects in the underwater environment, wherein the one or more objects are different than a bottom surface of the underwater environment, by:
generating, based on the 3D positional values of each of the plurality of sonar returns, a 3D representation of all of the plurality of sonar returns in the 3D matrix such that each of the plurality of sonar returns defines an x coordinate, a y coordinate, and a z coordinate; and
determining one or more clusters among the plurality of sonar returns in the 3D representation, wherein the one or more clusters among the plurality of sonar returns in the 3D representation each comprise sonar returns that are within a threshold distance of each other within the 3D representation; and
generate a 3D image of the one or more groups of sonar returns associated with the one or more objects in the underwater environment.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the program code portions are further configured to determine the one or more groups of sonar returns associated with the one or more objects in the underwater environment by:
determining which corresponds to the bottom surface of the underwater environment by determining the cluster with the largest number of sonar returns; and
removing the cluster that corresponds to the bottom surface of the underwater environment from the 3D representation.
