Audio-based technique to sense and detect the road condition for autonomous driving vehicles

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations of operating an autonomous driving vehicle (ADV), the operations comprising:

• receiving sound signals by one or more microphones disposed at the ADV, comprising:
• detecting the sound signals from a friction between each wheel of the ADV and a surface of a road, at least two microphones of the one or more microphones being mounted at a hub of each wheel of the ADV, the sound signals from the at least two microphones having a reduced noise by applying a noise reducing algorithm;
• analyzing the sound signals to extract a feature representing the road on which the ADV is driving, comprising:
• extracting a power spectrogram of the road based on the sound signals, the power spectrogram representing a power level of a signal of the sound signals in each frequency band over time and corresponding to a road condition of the road;
• determining the road condition of the road based on the extracted feature, comprising:
• determining a pavement condition of the road based on the extracted feature, the pavement condition including an asphalt condition, a rocky condition, or a sandy condition;
• determining a depth of a covering of the road based on the extracted feature, the depth of the covering of the road including a depth of water, snow, or sand on the road;
• and
• determining a friction coefficient of the road based on at least one of a surface type, the pavement condition, or the depth of the covering of the road by using regression models in a machine learning algorithm;
• performing a path planning and speed planning to generate a trajectory based on the road condition, comprising:
• performing at least one of changing a path or lowering a speed of the ADV, in response to that the surface type is a wet surface and the friction coefficient is less than a predetermined threshold;
• and
• controlling the ADV to drive autonomously according to the generated trajectory.

A data processing system, comprising:

• a processor;
• and
• a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations of operating an autonomous driving vehicle (ADV), the operations including
• receiving sound signals by one or more microphones disposed at the ADV, comprising:
• detecting the sound signals from a friction between each wheel of the ADV and a surface of a road, at least two microphones of the one or more microphones being mounted at a hub of each wheel of the ADV, the sound signals from the at least two microphones having a reduced noise by applying a noise reducing algorithm;
• analyzing the sound signals to extract a feature representing the road on which the ADV is driving, comprising:
• extracting a power spectrogram of the road based on the sound signals, the power spectrogram representing a power level of a signal of the sound signals in each frequency band over time and corresponding to a road condition of the road;
• determining the road condition of the road based on the extracted feature, comprising:
• determining a pavement condition of the road based on the extracted feature, the pavement condition including an asphalt condition, a rocky condition, or a sandy condition;
• determining a depth of a covering of the road based on the extracted feature, the depth of the covering of the road including a depth of water, snow, or sand on the road;
• and
• determining a friction coefficient of the road based on at least one of a surface type, the pavement condition, or the depth of the covering of the road;
• performing a path planning and speed planning to generate a trajectory based on the road condition, comprising:
• performing at least one of changing a path or lowering a speed of the ADV, in response to that the surface type is a wet surface and the friction coefficient is less than a predetermined threshold;
• and
• controlling the ADV to drive autonomously according to the generated trajectory.