Perception and planning collaboration framework for autonomous driving

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

• perceiving, by a perception module, a driving environment surrounding an autonomous driving vehicle (ADV) based on sensor data obtained from a plurality of sensors, generating perception information for a current driving cycle using a plurality of perception models;
• planning, by a planning module, a trajectory for the current driving cycle based on the perception information for the current driving cycle received from the perception module;
• determining a critical region surrounding the ADV based on the trajectory in view of a current location of the ADV, including
• determining a driving scenario associated with the ADV based on the trajectory and the current location of the ADV, wherein the driving scenario is determined based on a curvature of the trajectory, a speed and a heading direction of the ADV at different points in time on the trajectory,
• performing a lookup operation in a database based on the driving scenario to obtain metadata describing the critical region corresponding to the driving scenario, wherein the lookup operation includes locating a mapping entry matching the driving scenario including driving straight scenario, right turn on intersection scenario, left turn on intersection scenario, lane changing to left scenario, and wherein the driving straight scenario corresponds to a first critical region, the right turn on intersection scenario corresponds to a second critical region, the left turn on intersection scenario corresponds to a third critical region, and the lane changing to left scenario corresponds to a fourth critical region,
• determining a polygon defining the critical region surrounding the ADV based on the metadata describing the critical region,
• calculating vertexes of the polygon based on a shape of the polygon, wherein the vertexes of the polygon are used to determine a dimension and location of the critical region, and
• in response to determining the driving scenario associated with the ADV based on the trajectory and the current location of the ADV, constructing the critical region based on the driving scenario;
• controlling the ADV to drive according to the trajectory;
• and
• transmitting metadata describing the critical region to the perception module via an application programming interface (API) such that the perception module generates, within a time limit requirement of a single driving cycle, perception information for a next driving cycle in view of the critical region of the ADV, wherein generating the perception information for the next driving cycle comprises generating first perception information perceiving the critical region using a first perception model of the perception models based on three-dimension sensor data;
• and
• generating second perception information perceiving a remaining area other than the critical region using a second perception model of the perception model based on two-dimension sensor data, wherein the first perception information perceiving the critical region describes the driving environment in a higher resolution than the second perception information perceiving the remaining area other than the critical region, and wherein the perception module consumes more processing resources to generate the first perception information than the second perception information.

A data processing system, comprising:

• a processor;
• a memory coupled to the processor;
• and
• a perception module and a planning module loaded in the memory and executed by the processor to perform operations, the operations including
• perceiving, by the perception module, a driving environment surrounding an autonomous driving vehicle (ADV) based on sensor data obtained from a plurality of sensors, generating perception information for a current driving cycle using a plurality of perception models,
• planning, by the planning module, a trajectory for the current driving cycle based on the perception information for the current driving cycle received from the perception module,
• determining a critical region surrounding the ADV based on the trajectory in view of a current location of the ADV, including
• determining a driving scenario associated with the ADV based on the trajectory and the current location of the ADV, wherein the driving scenario is determined based on a curvature of the trajectory, a speed and a heading direction of the ADV at different points in time on the trajectory,
• performing a lookup operation in a database based on the driving scenario to obtain metadata describing the critical region corresponding to the driving scenario, wherein the lookup operation includes locating a mapping entry matching the driving scenario including driving straight scenario, right turn on intersection scenario, left turn on intersection scenario, lane changing to left scenario, and wherein the driving straight scenario corresponds to a first critical region, the right turn on intersection scenario corresponds to a second critical region, the left turn on intersection scenario corresponds to a third critical region, and the lane changing to left scenario corresponds to a fourth critical region,
• determining a polygon defining the critical region surrounding the ADV based on the metadata describing the critical region,
• calculating vertexes of the polygon based on a shape of the polygon, wherein the vertexes of the polygon are used to determine a dimension and location of the critical region, and
• in response to determining the driving scenario associated with the ADV based on the trajectory and the current location of the ADV, constructing the critical region based on the driving scenario,
• controlling the ADV to drive according to the trajectory;
• and
• transmitting metadata describing the critical region to the perception module via an application programming interface (API) such that the perception module generates, within a time limit requirement of a single driving cycle, perception information for a next driving cycle in view of the critical region of the ADV, wherein generating the perception information for the next driving cycle comprises generating first perception information perceiving the critical region using a first perception model of the perception models based on three-dimension sensor data;
• and generating second perception information perceiving a remaining area other than the critical region using a second perception model of the perception model based on two-dimension sensor data, wherein the first perception information perceiving the critical region describes the driving environment in a higher resolution than the second perception information perceiving the remaining area other than the critical region, and wherein the perception module consumes more processing resources to generate the first perception information than the second perception information.