# US 12,007,474 B2 · Autonomous vehicle operation using acoustic modalities > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,007,474 B2 - Document ID: US12007474B2 - Status: Granted - Kind: B2 - Application number: 16/885,385 - Filed: 2020-05-28 - Published: 2020-12-03 - Granted: 2024-06-11 - Earliest claimed: 2019-05-28 - Assignees: Motional AD LLC - Applicants: Motional AD LLC - Inventors: Aravindkumar Vijayalingam; Abhimanyu Singh - Web page: https://patentgrep.com/patent/US12007474B2 ## CPC - G01S 15/931: of land vehicles - B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering - G01S 3/80: using ultrasonic, sonic or infrasonic waves - G01S 5/0252: {Radio frequency fingerprinting} - G01S 5/02528: {Simulating radio frequency fingerprints} - G05D 1/0255: {using acoustic signals, e.g. ultra-sonic singals (sonar systems designed for anti-collision purposes G01S15/93)} - G05D 1/247: using signals provided by artificial sources external to the vehicle, e.g. navigation beacons - G06F 16/60: of audio data - G06F 16/65: Clustering; Classification - G06F 18/23: Clustering techniques - G06F 18/253: {of extracted features} - G06V 10/806: {of extracted features} - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - B60W 2420/54: Audio sensitive means, e.g. ultrasound - G06V 2201/10: Recognition assisted with metadata - H04R 2499/13: Acoustic transducers and sound field adaptation in vehicles - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract Techniques for autonomous vehicle operation using acoustic modalities include using one or more acoustic sensors of a vehicle to receive acoustic waves from one or more objects. The acoustic waves have multiple wavelengths. The acoustic waves are clustered into one or more acoustic clusters based on the plurality of wavelengths. A particular acoustic cluster of the one or more acoustic clusters is selected based on signal processing of the one or more acoustic clusters. A particular object is associated with the particular acoustic cluster. An acoustic fingerprint of the particular object is generated based on the particular acoustic cluster. Characteristics of the particular object are determined based on the acoustic fingerprint of the particular object. A control circuit of the vehicle is used to operate the vehicle to avoid a collision with the particular object based on the characteristics of the particular object. ## Claims ### Claim 1 A method comprising: receiving, using one or more acoustic sensors of a vehicle, acoustic waves from two or more objects, wherein the acoustic waves have a plurality of wavelengths; clustering, using one or more processors of the vehicle, the acoustic waves into two or more acoustic clusters based on the plurality of wavelengths; selecting, using the one or more processors, a particular acoustic cluster of the two or more acoustic clusters based on signal processing of the two or more acoustic clusters; associating, using the one or more processors, a particular object of the two or more objects with the particular acoustic cluster; generating, using the one or more processors, an acoustic fingerprint of the particular object based on the particular acoustic cluster; determining, using the one or more processors, characteristics of the particular object based on the acoustic fingerprint of the particular object; and operating, using a control circuit of the vehicle, the vehicle to avoid a collision with the particular object based on the characteristics of the particular object. ### Claim 2 (depends on 1) The method of claim 1, wherein each acoustic cluster of the ene two or more acoustic clusters corresponds to a respective object of the one two or more objects. ### Claim 3 (depends on 1) The method of claim 1, wherein the clustering of the acoustic waves comprises determining, using the one or more processors, a change in a wavelength or a frequency of the acoustic waves based on filtering the acoustic waves. ### Claim 4 (depends on 3) The method of claim 3, wherein the filtering of the acoustic waves is performed using a gradient filter. ### Claim 5 (depends on 1) The method of claim 1, wherein the clustering of the acoustic waves comprises splitting, using the one or more processors, the acoustic waves into the two or more acoustic clusters based on a spectrogram. ### Claim 6 (depends on 1) The method of claim 1, wherein the clustering of the acoustic waves comprises splitting, using the one or more processors, the acoustic waves into the two or more acoustic clusters based on a time-frequency graph. ### Claim 7 (depends on 1) The method of claim 1, wherein the generating of the acoustic fingerprint comprises determining, using the one or more processors, a zero crossing rate of the particular acoustic cluster of the two or more acoustic clusters. ### Claim 8 (depends on 1) The method of claim 1, wherein the generating of the acoustic fingerprint comprises determining, using the one or more processors, a spectral density of the particular acoustic cluster of the two or more acoustic clusters. ### Claim 9 (depends on 1) The method of claim 1, wherein the generating of the acoustic fingerprint comprises determining, using the one or more processors, a spectral flatness of the particular acoustic cluster of the two or more acoustic clusters. ### Claim 10 (depends on 1) The method of claim 1, wherein the generating of the acoustic fingerprint comprises determining, using the one or more processors, peaks in a spectrogram of the particular acoustic cluster of the two or more acoustic clusters. ### Claim 11 (depends on 1) The method of claim 1, wherein the generating of the acoustic fingerprint comprises determining, using the one or more processors, an acoustic pressure of the particular acoustic cluster of the two or more acoustic clusters. ### Claim 12 (depends on 1) The method of claim 1, wherein the generating of the acoustic fingerprint comprises determining, using the one or more processors, a particle velocity of the particular acoustic cluster of the two or more acoustic clusters. ### Claim 13 (depends on 1) The method of claim 1, wherein the generating of the acoustic fingerprint comprises determining, using the one or more processors, a particle displacement of the particular acoustic cluster of the two or more acoustic clusters. ### Claim 14 (depends on 1) The method of claim 1, wherein the generating of the acoustic fingerprint comprises determining, using the one or more processors, an acoustic intensity of the particular acoustic cluster of the two or more acoustic clusters. ### Claim 15 (depends on 1) The method of claim 1, wherein the acoustic waves are diffracted, reflected, or interfered with by the particular object of the two or more objects. ### Claim 16 (depends on 1) The method of claim 1, further comprising filtering, using the one or more processors, the acoustic waves based on peaks in a spectrogram of the acoustic waves to remove background noise. ### Claim 17 (depends on 1) The method of claim 1, wherein the characteristics comprise physical characteristics of the particular object, and wherein the determining of the physical characteristics comprises generating, using the one or more processors, a hash of peaks in a spectrogram of the acoustic waves. ### Claim 18 (depends on 17) The method of claim 17, wherein the determining of the physical characteristics further comprises searching, using the one or more processors, a database based on the hash of peaks to determine the physical characteristics. ### Claim 19 A vehicle comprising: one or more computer processors; and one or more non-transitory storage media storing instructions which, when executed by the one or more computer processors, cause the one or more computer processors to: receive, using one or more acoustic sensors of a vehicle, acoustic waves from two or more objects, wherein the acoustic waves have a plurality of wavelengths; cluster the acoustic waves into two or more acoustic clusters based on the plurality of wavelengths; select a particular acoustic cluster of the two or more acoustic clusters based on signal processing of the two or more acoustic clusters; associate a particular object of the two or more objects with the particular acoustic cluster; generate an acoustic fingerprint of the particular object based on the particular acoustic cluster; determine characteristics of the particular object based on the acoustic fingerprint of the particular object; and operate, using a control circuit of the vehicle, the vehicle to avoid a collision with the particular object based on the characteristics of the particular object. ### Claim 20 One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to: receive, using one or more acoustic sensors of a vehicle, acoustic waves from two or more objects, wherein the acoustic waves have a plurality of wavelengths; cluster the acoustic waves into two or more acoustic clusters based on the plurality of wavelengths; select a particular acoustic cluster of the two or more acoustic clusters based on signal processing of the two or more acoustic clusters; associate a particular object of the two or more objects with the particular acoustic cluster; generate an acoustic fingerprint of the particular object based on the particular acoustic cluster; determine characteristics of the particular object based on the acoustic fingerprint of the particular object; and operate, using a control circuit of the vehicle, the vehicle to avoid a collision with the particular object based on the characteristics of the particular object.