# US 11,029,165 B2 · Sensor integration and synchronization unit for an autonomous driving vehicle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,029,165 B2
- Document ID: US11029165B2
- Status: Granted
- Kind: B2
- Application number: 16/158,287
- Filed: 2018-10-11
- Published: 2020-04-16
- Granted: 2021-06-08
- Earliest claimed: 2018-10-11
- Assignees: Baidu USA LLC
- Applicants: Baidu USA LLC
- Inventors: Zhou Xu; Manjiang Zhang; Xiangtao You; Weiyu Lu; Fu-Kang Liao
- Web page: https://patentgrep.com/patent/US11029165B2

## CPC

- G01C 21/26: specially adapted for navigation in a road network
- G01C 21/343: {Calculating itineraries (travelling salesman problem G06Q10/04; optimisation of routes G06Q10/047)}
- G01C 21/3407: {specially adapted for specific applications}
- G01C 21/362: {received from an external device or application, e.g. PDA, mobile phone or calendar application}
- G01C 21/3623: {using a camera or code reader, e.g. for optical or magnetic codes}
- G01S 17/931: of land vehicles
- G01S 19/46: the supplementary measurement being of a radio-wave signal type
- G01S 19/48: by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system
- G05D 1/0257: {using a radar (radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931)}
- G05D 1/0278: {using satellite positioning signals, e.g. GPS}
- G01S 13/865: {Combination of radar systems with lidar systems}
- G01S 13/867: {Combination of radar systems with cameras}
- G01S 13/931: of land vehicles
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES

## Abstract

A sensor unit includes a sensor interface and a host interface. The sensor interface can be coupled to a number of sensors mounted on various locations of the ADV. The host interface can be coupled to a host system that is configured to perceive a driving environment surrounding the ADV based on sensor data obtained from the sensors and to plan a path to autonomously drive the ADV. The sensor unit further includes a sensor processing module and a sensor control module coupled to the sensor interface, and a time module coupled to the sensor processing module and the sensor control module. The sensor processing module is configured to process sensor data received from the sensors via the sensor interface. The sensor control module is configured to control operations of the sensors. The time module is configured to generate time to synchronize timing of the operations of the sensors.

## Claims

### Claim 1

A sensor unit utilized in an autonomous driving vehicle, the sensor unit comprising:
a sensor interface to be coupled to a plurality of sensors mounted on a plurality of locations of an autonomous driving vehicle (ADV);
a host interface to be coupled to a host system, wherein the host system is configured to perceive a driving environment surrounding the ADV based on sensor data obtained from the plurality of sensors and to plan a path to autonomously drive the ADV;
a sensor control module coupled to the sensor interface, the sensor control module configured to control activation of capture of sensor data for each sensor in the plurality of sensors in accordance with a transmit timestamp for the sensor, the transmit timestamp received from a time module;
a sensor processing module coupled to the sensor interface, the sensor processing module configured to process sensor data captured by each sensor in the plurality of sensors in accordance with a receive timestamp for the sensor, the receive timestamp received from the time module;
and
the time module coupled to the sensor processing module and the sensor control module to generate the transmit timestamps to synchronize timing of transmission of control commands to each of the plurality of sensors, wherein the transmit timestamps indicate an ordering and a timing that sensor data is captured by each sensor in the plurality of sensors, and to generate receive timestamps to synchronize the timing that the sensor data captured by each sensor in the plurality of sensors is processed, wherein the receive timestamps indicate an ordering and a timing that the sensor data of each sensor in the plurality of sensors is processed.

### Claim 2 (depends on 1)

The sensor unit of claim 1, wherein the host system has to go through the sensor unit in order to access any of the plurality of sensors mounted on the ADV.

### Claim 3 (depends on 1)

The sensor unit of claim 1, wherein the time module is configured to generate receiving timestamps for the sensor processing module and to generate transmitting timestamps for the sensor control module from an identical time source.

### Claim 4 (depends on 1)

The sensor unit of claim 1, wherein the sensor control module is further configured to generate and send a same transmit timestamp to at least two sensors of the plurality of sensors, indicating that the at least two sensors are to be activated to capture sensor data at a same time indicated in the same transmit timestamp.

### Claim 5 (depends on 4)

The sensor unit of claim 4, wherein the at least two sensors are a same sensor type, and the sensor type is LIDAR.

### Claim 6 (depends on 1)

The sensor unit of claim 1, wherein the time module is configured to generate the time based on GPS signals received from a GPS sensor or pulse per second (PPS) and GPRMC signals received from a GNSS receiver via the sensor interface.

### Claim 7 (depends on 6)

The sensor unit of claim 6, wherein in response to determining that the GPS, PPS, and GPRMC signals are unavailable, the time module is configured to derive the time for synchronization from an alternative time source.

### Claim 8 (depends on 7)

The sensor unit of claim 7, wherein the alternative time source includes a real-time clock (RTC) source provided by the host system from the host interface or a network time source from an Ethernet or LTE.

### Claim 9 (depends on 1)

The sensor unit of claim 1, further comprising a data transfer module coupled to the sensor processing module to transfer the processed sensor data to the host system via the host interface.

### Claim 10 (depends on 9)

The sensor unit of claim 9, wherein the data transfer module is configured to transfer data between the sensors and the host system in a plurality of operating modes dependent upon a type of the sensors, including a low latency mode, a high bandwidth mode, and a fixed memory mode.

### Claim 11 (depends on 10)

The sensor unit of claim 10, wherein when operating in the low latency mode, the data transfer module is configured to transfer data with a minimum delay.

### Claim 12 (depends on 10)

The sensor unit of claim 10, wherein when operating in the high bandwidth mode, the data transfer module is configured to accumulate the data up to a predetermined amount, prior to sending the accumulated data in a burst to the host system to maximize a bandwidth of a connection between the data transfer module and the host system.

### Claim 13 (depends on 10)

The sensor unit of claim 10, wherein when operating in the fixed memory mode, the data module is configured to directly access a mapped memory of the host system.

### Claim 14 (depends on 9)

The sensor unit of claim 9, wherein the connection between the data transfer module and the host system is a peripheral component interconnect express (PCIe) compatible connection.

### Claim 15 (depends on 1)

The sensor unit of claim 1, wherein the sensor interface comprises an Ethernet interface to be coupled with a LIDAR device or one or more cameras.

### Claim 16 (depends on 1)

The sensor unit of claim 1, wherein the sensor interface comprises a global positioning system (GPS) interface to be coupled to at least one of a GPS receiver and an IMU device.

### Claim 17 (depends on 1)

The sensor unit of claim 1, wherein the sensor interface comprises a control area network (CAN) interface to be coupled to throttle control logic, braking control logic, and steering control logic of the ADV.

### Claim 18

An autonomous driving system, comprising:
a plurality of sensors mounted on a plurality of locations of an autonomous driving vehicle (ADV);
a host system having a perception module and a planning and control module, wherein the perception module is to perceive a driving environment surrounding the ADV based on sensor data obtained from the sensors, and wherein the planning and control module is to plan a path to autonomously drive the ADV; and
a sensor unit coupled to the plurality of sensors and the host system, wherein the sensor unit comprises
a sensor interface to be coupled to a plurality of sensors mounted on a plurality of locations of an autonomous driving vehicle (ADV),
a host interface to be coupled to a host system, wherein the host system is configured to perceive a driving environment surrounding the ADV based on sensor data obtained from the plurality of sensors and to plan a path to autonomously drive the ADV,
a sensor control module coupled to the sensor interface, the sensor control module configured to control activation of capture of sensor data for each sensor in the plurality of sensors in accordance with a transmit timestamp for the sensor, the transmit timestamp received from a time module,
a sensor processing module coupled to the sensor interface, the sensor processing module configured to process sensor data captured by each sensor in the plurality of sensors in accordance with a receive timestamp for the sensor, the receive timestamp received from a time module,
and
the time module coupled to the sensor processing module and the sensor control module to generate transmit timestamps to synchronize timing of transmission of control commands to each of the plurality of sensors, wherein the transmit timestamps indicate and ordering and a timing that sensor data is captured by each sensor in the plurality of sensors, and to generate receive timestamps to synchronize the timing that sensor data captured by each sensor in the plurality of sensors is processed, wherein the receive timestamps indicate an ordering and timing that the sensor data of each sensor in the plurality of sensors is processed.

### Claim 19 (depends on 18)

The system of claim 18, wherein the host system has to go through the sensor unit in order to access any of the plurality of sensors mounted on the ADV.

### Claim 20 (depends on 18)

The system of claim 18, wherein the time module is configured to generate receiving timestamps for the sensor processing module and to generate transmitting timestamps for the sensor control module from an identical time source.