# US 10,311,719 B1 · Enhanced traffic detection by fusing multiple sensor data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,719 B1
- Document ID: US10311719B1
- Status: Granted
- Kind: B1
- Application number: 16/201,952
- Filed: 2018-11-27
- Granted: 2019-06-04
- Earliest claimed: 2018-08-24
- Assignees: Iteris, Inc.
- Applicants: Iteris, Inc.
- Inventors: Dilip Swaminathan; Manu P. Jain; Yan Gao; Todd W. Kreter; Michael T. Whiting
- Web page: https://patentgrep.com/patent/US10311719B1

## CPC

- G01S 13/867: {Combination of radar systems with cameras}
- G01S 13/91: for traffic control (G01S13/93 takes precedence)
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G08G 1/0125: {Traffic data processing}
- G08G 1/015: with provision for distinguishing between two or more types of vehicles, e.g. between motor-cars and cycles
- G08G 1/04: using optical or ultrasonic detectors
- G08G 1/042: using inductive or magnetic detectors
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

A framework for precision traffic analysis combines traffic sensor data from multiple sensor types, combining the strengths of each sensor type within various conditions in an intersection or roadway in which traffic activity occurs. The framework calibrates coordinate systems in images taken of the same area by multiple sensors, so that fields of view from one sensor system are transposed onto fields of view from other sensor systems to fuse the images taken into a combined detection zone, and so that objects are properly detected and classified for enhanced traffic signal control.

## Claims

### Claim 1

A method, comprising:
receiving input data representing a field of view of a roadway at or near a traffic intersection, the input data including data collected from a first sensor, and data collected from a second sensor;
analyzing the input data within a computing environment in one or more data processing modules executed in conjunction with at least one specifically-configured processor, the one or more data processing modules configured to detect an object in the field of view of the roadway, by
evaluating an accuracy of the data obtained from the first sensor, by relating one or more attributes representing object characteristics in the data obtained from the first sensor to known attributes representing one or more objects, and
fusing the data from the first sensor with the data from the second sensor to create a combined detection zone in the field of view of the roadway where the one or more attributes representing object characteristics are below a threshold level for accurate detection, relative to the known attributes, by
identifying a first object detection zone in the field of view from the data collected from the first sensor, and a second object detection zone in the field of view from the data collected from the second sensor,
selecting a plurality of boundary points in the first object detection zone, and a plurality of points in the second object detection zone that correspond to the boundary points in the first detection zone,
estimating a transformation matrix for a planar area comprising the field of view by mapping the plurality of points in a plane of the road roadway as seen by the second sensor, to the plurality of boundary points in a plane of the roadway as seen by the first sensor to correct a perspective distortion between the data collected by the first sensor and the data collected by the second sensor to transpose the first object detection zone onto the second object detection zone, by applying the selected points that correspond to the plurality of boundary points to calibrate arbitrary planar coordinates between the first object detection zone and the second object detection zone, and mapping a plurality of vectors representing common data points to transform the arbitrary planar coordinates to find an equivalent zone location and an equivalent shape between the first object detection zone and the second object detection zone; and
generating an output signal to a traffic signal controller where an object is present in the combined detection zone.