# US 6,731,777 B1 · Object recognition system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 6,731,777 B1
- Document ID: US6731777B1
- Status: Granted
- Kind: B1
- Application number: 09/567,734
- Filed: 2000-05-10
- Granted: 2004-05-04
- Earliest claimed: 1999-06-16
- Assignees: Honda Giken Kogyo Kabushiki Kaisha
- Inventors: Morimichi Nishigaki; Masakazu Saka
- Web page: https://patentgrep.com/patent/US6731777B1

## CPC

- G01C 3/085: {with electronic parallax measurement}
- G06V 10/267: {by performing operations on regions, e.g. growing, shrinking or watersheds}
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/64: Three-dimensional [3D] objects
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An object recognition system includes one or more image sensors and a controller that is adapted for measuring the distance from the system to objects with respect to respective windows of an image captured by the sensors is provided. The controller forms clusters by uniting adjacent windows that have similar measured distances. The system includes a memory for storing data on a previously recognized physical object. The controller infers a physical object based on data on a previously recognized physical object and the speed of the vehicle relative to a previously recognized physical object and wherein said controller determines a combination of clusters that best matches the inferred physical object. The controller groups the clusters into one or more cluster groups according to the distance from the vehicle carrying the system and selects from one of the cluster groups those clusters that overlaps with a physical object inferred by the controller. The controller matches combinations of the selected clusters with the inferred physical object based on the attributes of the combined clusters and the attributes of the inferred physical object.

## Claims

### Claim 1

An object recognition system, comprising:
one or more image sensors;
a controller that is adapted for measuring the distance from the system to objects with respect to respective windows of an image captured by the sensors, wherein said controller forms clusters by uniting adjacent windows that have similar measured distances; and
a memory for storing data on a previous recognized physical object;
wherein said controller infers a physical object using the data on the previously recognized physical object and the speed of a vehicle relative to the previously recognized physical object and wherein said controller determines a combination of clusters that best matches the inferred physical object.

### Claim 2 (depends on 1)

The system of claim 1, wherein said controller groups the clusters into one or more cluster groups according to the distance from the vehicle carrying the system.

### Claim 3 (depends on 2)

The system of claim 2, wherein said controller selects from one of the cluster groups those clusters that overlaps with a physical object inferred by the controller.

### Claim 4 (depends on 3)

The system of claim 3, wherein said controller matches combinations of the selected clusters with the inferred physical object based on the attributes of the combined clusters and the attributes of the inferred physical object.

### Claim 5 (depends on 4)

The system of claim 4, wherein the controller calculates E1 according to the following equation and determines the combination of clusters that has the smallest E1 to be the clusters representing the physical object:
E1={square root over ((Xc−Xt)2+(Yc−Yt)2+(Zc−Zt)2/C·Zt)}
where Xc is x coordinate of horizontal center of combined clusters, Yc is y coordinate of vertical center of combined clusters, Zc is z coordinate indicating distance of combined clusters from the vehicle, Xt is x coordinate of horizontal center of an inferred physical object, Yt is y coordinate of vertical center of the inferred physical object, Zt is z coordinate indicating distance of the inferred physical object from the vehicle, and C is a constant.

### Claim 6 (depends on 5)

The system of claim 5, wherein Zt is determined by an equation:
previous Zt+relative speed×detection time interval.

### Claim 7 (depends on 5)

The system of claim 5, wherein the following value is added to E1:
|Wc−Wt|+|Hc−Ht|
where Wc and Wt are width of combined clusters and the inferred physical object respectively, and Hc and Ht are height of combined clusters and the inferred physical object respectively.

### Claim 8 (depends on 6)

The system of claim 6, wherein the relative speed is determined by dividing by the detection time interval the difference between a distance from the vehicle to the physical object that has been measured in the present process cycle and a distance from the vehicle to said physical object that was measured in the preceding process cycle.

### Claim 9

An object recognition system, comprising:
one or more image sensors;
means for measuring distance from the system to a physical object with respect to each window of an image captured by the sensors;
means for clustering adjacent windows that are in a predetermined distance range;
means for inferring a present position of the physical object that was recognized in the previous recognition cycle, based on previous position of the physical object and the speed of a vehicle relative to the object;
selection means for selecting clusters whose distance from the vehicle is within a predetermined tolerance relative to the distance of the inferred physical object and which overlaps with the inferred physical object;
means for recognizing, as representing the physical object, the combined clusters that comprise one or more clusters selected by said selection and that have closest attributes to the attributes of the inferred physical object.

### Claim 10 (depends on 9)

The system of claim 9, further comprising;
a memory for storing attributes of at least one sample physical object; and
means for comparing attributes of combined clusters that were not selected by said selection means with attributes of said at least one sample physical object to recognize the sample physical object having closest attributes to be the physical object corresponding to the combined clusters.

### Claim 11

A method for recognizing a physical object, comprising the steps of:
measuring distance from a vehicle to the physical object with respect to respective windows of an image captured by at least one image sensor;
uniting adjacent windows that have similar measured distances to form clusters;
inferring a present position of the physical object using data on a previously recognized physical object and the speed of the vehicle relative to the previously recognized physical object; and
determining a combination of clusters that best matches the inferred physical object.

### Claim 12 (depends on 11)

The method of claim 11, wherein said step of determining includes the step of selecting those clusters that overlaps with a physical object inferred by the controller.

### Claim 13 (depends on 12)

The method of claim 12, wherein said step of determining includes the step of matching combinations of the selected clusters with the inferred physical object based on the attributes of the combined clusters and the attributes of the inferred physical object.

### Claim 14 (depends on 13)

The method of claim 13, wherein said step of matching includes the step of calculating E1 according to the following equation to determine the combination of clusters that has the smallest E1:
E1={square root over ((Xc−Xt)2+(Yc−Yt)2+(Zc−Zt)2/C·Zt)}
where Xc is x coordinate of horizontal center of combined clusters, Yc is y coordinate of vertical center of combined clusters, Zc is z coordinate indicating distance of combined clusters from the vehicle, Xt is x coordinate of horizontal center of an inferred physical object, Yt is y coordinate of vertical center of the inferred physical object, Zt is z coordinate indicating distance of the inferred physical object from the vehicle, and C is a constant.

### Claim 15 (depends on 14)

The method of claim 14, wherein Zt is determined by an equation: previous Zt+relative speed×detection time interval.

### Claim 16 (depends on 14)

The method of claim 14, wherein the following value is added to E1:
|Wc−Wt|+|Hc−Ht|
where Wc and Wt are width of combined clusters and the inferred physical object respectively, and Hc and Ht are height of combined clusters and the inferred physical object respectively.

### Claim 17 (depends on 15)

The method of claim 15, wherein the relative speed is determined by dividing by the detection time interval the difference between a distance from the vehicle to the physical object that has been measured in the present process cycle and a distance from the vehicle to said physical object that was measured in the preceding process cycle.

### Claim 18

A method of recognizing a physical object, comprising the steps of:
measuring distance from a vehicle to the physical object with respect to each window of an image captured one or more image sensors;
clustering adjacent windows that are in a predetermined distance range;
inferring a present position of the physical object that was recognized in the previous recognition cycle, based on previous position of the physical object and the speed of the vehicle relative to the object;
selecting clusters whose distance from the vehicle is within a predetermined tolerance relative to the distance of the inferred physical object and which overlaps with the inferred physical object;
recognizing, as representing the physical object, the combined clusters that comprise one or more clusters selected by the step of selecting and that have closest attributes to the attributes of the inferred physical object.

### Claim 19 (depends on 18)

The method of claim 18, further comprising the steps of:
storing attributes of at least one sample physical object; and
comparing attributes of combined clusters that were not selected in said step of selecting with attributes of said at least one sample physical object to recognize the sample physical object having closest attributes to be the physical object corresponding to the combined clusters.
