Close-in Sensing Camera System

A vehicle configured to operate in an autonomous driving mode, the vehicle comprising:

• a driving system including:
• a deceleration system configured to control braking of the vehicle;
• an acceleration system configured to control acceleration of the vehicle;
• and
• a steering system configured to control wheel orientation and a direction of the vehicle;
• and
• an external sensing system including:
• a lidar sensor having a field of view configured to detect objects in at least a region of an external environment around the vehicle and within a threshold distance of the vehicle;
• an image sensor disposed adjacent to the lidar sensor and arranged along the vehicle to have an overlapping field of view of the region of the external environment within the threshold distance of the vehicle, the image sensor providing a selected resolution for objects within the threshold distance;
• and
• a control system operatively coupled to the image sensor and the lidar sensor, the control system including one or more processors configured to:
• initiate operation of the lidar sensor to obtain lidar data in the region within the threshold distance of the vehicle;
• initiate image capture by the image sensor prior to the driving system performing a driving action;
• receive the lidar data from the lidar sensor and the captured imagery from the image sensor;
• perform processing of the lidar data to detect an object in the region within the threshold distance of the vehicle;
• perform processing of the captured imagery to classify the detected object;
• and
• determine whether to cause one or more systems of the vehicle to perform the driving action based on classification of the detected object.

A method, comprising:

• initiating, by a control system of a vehicle configured to operate in an autonomous driving mode, operation of a lidar sensor of a perception system of the vehicle to obtain lidar data within a threshold distance in a region around the vehicle;
• initiating, by the control system, image capture by an image sensor of the perception system prior to the vehicle performing a driving action, the image sensor being disposed adjacent to the lidar sensor and arranged along the vehicle to have an overlapping field of view of the region around the vehicle within the threshold distance, wherein the image sensor provides a selected resolution for objects within the threshold distance;
• receiving, by the control system, the lidar data from the lidar sensor and the captured imagery from the image sensor;
• processing, by the control system, the lidar data to detect an object in the region within the threshold distance of the vehicle;
• processing, by the control system, the captured imagery to classify the detected object;
• and
• determining, by the control system, whether to cause one or more systems of the vehicle to perform the driving action based on classification of the detected object.