Use of camera information for radar beamforming
AssigneeGM Cruise Holdings LLC
InventorsDaniel Flores Tapia, Jace Artichoker
Life of this patent
Estimating the term…
Abstract
Use of camera information for radio detection and ranging (RADAR) beamforming is disclosed. Camera information for a scene having a target object is received. Digital RADAR waveforms corresponding to a frame of reference including the at target object are received. Coordinates for the scene in the camera frame of reference are translated to the RADAR frame of reference. A radar cross section estimation is determined for the object based on the transformed coordinates. A kernel is selected based on the radar cross section estimation. RADAR signal processing is performed on the digital RADAR waveforms utilizing the selected kernel. A point cloud is populated based on results from the RADAR signal processing.
Claims17
An autonomous vehicle comprising one or more sensor systems to detect characteristics of an operating environment, the one or more sensor systems comprising at least a camera sensor system and a RADAR sensor system, the one or more sensor systems to:
- receive, from the camera sensor system, camera information co-registered with a RADAR sensor system field of view of a scene having a target object;
- segment the camera information into an image segment that includes the target object;
- identify the target object from the camera information;
- determine a target type and target location coordinates for the target object based on the image segment;
- receive, from the RADAR sensor system, digital RADAR waveforms corresponding to a frame of reference of the RADAR, the frame of reference including the target object;
- transform the target location coordinates for the target object to the frame of reference;
- determine a radar cross section estimation for the target object based on a ratio of a structural dimension of the image segment to a wavelength of the RADAR system, wherein the structural dimension is based on the transformed coordinates;
- select a kernel from a kernel repository based on the radar cross section estimation, target location coordinates, and the target type, wherein the selected kernel comprises a time series representation of the target type applicable to the scene;
- perform data beamforming by convolving the digital RADAR waveforms utilizing the selected kernel to maximize output resolution corresponding to the camera coordinates and maximize the angular resolution based on the phase and magnitude of the RADAR waveforms;
- and
- populate a point cloud with results from the RADAR signal processing.
A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, are configurable to cause the processors to:
- receive camera information for a scene having a target object, wherein the camera information is co-registered with a RADAR sensor system field of view of the scene;
- segment the camera information into an image segment that includes the target object;
- identify the target object in the image segment from the camera information;
- determine a target type and target location coordinates for the target object based on the image segment;
- receive digital RADAR waveforms corresponding to a frame of reference of a RADAR system, the frame of reference including the target object;
- transform the target location coordinates for the target object to the frame of reference;
- determine a radar cross section estimation for the target object based on a ratio of a structural dimension of the image segment to a wavelength of the RADAR system, wherein the structural dimension is based on the transformed coordinates;
- select a kernel from a kernel repository based on the radar cross section estimation, target location coordinates, and the target type, wherein the selected kernel comprises a time series representation of the target type applicable to the scene;
- perform data beamforming by convolving the digital RADAR waveforms with the selected kernel to maximize output resolution corresponding to the target location coordinates and the angular resolution based on the phase and magnitude of the RADAR waveforms;
- and
- populate a point cloud with results from the RADAR signal processing.
A system comprising:
- a memory system;
- and
- one or more hardware processors coupled with the memory system, the one or more processors to:
- receive camera information for a scene having a target object, wherein the camera information is co-registered with a RADAR sensor system field of view of the scene;
- segment the camera information into an image segment that includes the target object;
- identify the target object in the image segment from the camera information;
- determine a target type and target location coordinates for the target object based on the image segment;
- receive digital RADAR waveforms corresponding to a frame of reference of a RADAR system, the frame of reference including the target object;
- transform the target location coordinates for the target object to the frame of reference;
- determine a radar cross section estimation for the target object based on a ratio of a structural dimension of the image segment to a wavelength of the RADAR system, wherein the structural dimension is based on the transformed coordinates;
- select a kernel from a kernel repository based on the radar cross section estimation, target location coordinates, and the target type, wherein the selected kernel comprises a time series representation of the target type applicable to the scene;
- and
- perform data beamforming by convolving the digital RADAR waveforms with the selected kernel to maximize output resolution corresponding to the target location coordinates and the angular resolution based on the phase and magnitude of the RADAR waveforms.
Dependencies are inferred from the claim text.