# US 11,999,354 B2 · Method and apparatus for estimation road surface type using ultrasonic signal > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,999,354 B2 - Document ID: US11999354B2 - Status: Granted - Kind: B2 - Application number: 17/186,329 - Filed: 2021-02-26 - Published: 2021-06-17 - Granted: 2024-06-04 - Earliest claimed: 2018-08-29 - Assignees: Korea Advanced Institute of Science and Technology - Applicants: Korea Advanced Institute of Science and Technology - Inventors: Sei-Bum Choi; Min-Hyun Kim; Jin-Rak Park; Seung-In Shin; Jong-Chan Park - Web page: https://patentgrep.com/patent/US11999354B2 ## CPC - B60W 40/068: Road friction coefficient - B60W 40/06: Road conditions - G01S 7/539: using analysis of echo signal for target characterisation; Target signature; Target cross-section - G01S 15/88: Sonar systems specially adapted for specific applications (seismic or acoustic prospecting or detecting G01V1/00) - G01S 15/931: of land vehicles - G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06V 10/82: using neural networks - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - B60W 2420/54: Audio sensitive means, e.g. ultrasound - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The present invention relates to a method and apparatus for estimating a road surface type by using an ultrasonic signal and, more particularly, to a method for estimating a road surface type by using an artificial neural network model machine-learned with respect to a reflected ultrasonic signal and an apparatus for performing same. According to the present invention, provided are a method and apparatus for providing highly accurate road surface information at low cost, by machine-learning both characteristics of an ultrasonic signal reflected from a road surface and a road surface state, establishing a model between the two, and then estimating the type of the road surface by utilizing the model. In particular, even a road surface where thin ice, that is, black ice, is formed, which was not detectable in the conventional method for estimating a road-surface friction coefficient, may be accurately estimated, thereby contributing to safer driving. ## Claims ### Claim 1 A method for estimation road surface type using an ultrasonic signal, comprising: (a) extracting a 1D feature vector through one or more 1D convolutional layers using an input signal derived from a reflection signal in the time domain of the ultrasonic signal reflected from the road surface after being emitted to the road surface; (b) receiving the 1D feature vector as an input and estimating a probability value for each road surface type in an artificial neural network having one or more layers; and, (c) determining the road surface type from the estimated probability value for each road surface type. ### Claim 2 (depends on 1) The method according to claim 1, wherein the input signal of the one or more 1D convolutional layers in the step (a) is a signal in the time-frequency domain which is generated by dividing the reflection signal in the time domain at predetermined time intervals and performing Short-Time Fourier Transform on each divided reflection signal. ### Claim 3 (depends on 1) The method according to claim 1, wherein the input signal of the one or more 1D convolutional layers in step (a) is the reflection signal in the time domain. ### Claim 4 (depends on 1) The method according to claim 1, wherein the road surface type determined in step (c) includes at least one of asphalt, cement, dirt, ice, marble, applied paint, snow, water, and lanes marked on the road surface. ### Claim 5 (depends on 1) The method according to claim 1, further comprising a step of estimating a road surface friction coefficient from the road surface type determined in the step (c). ### Claim 6 (depends on 1) A road surface type estimation system using ultrasonic signals, comprising: an ultrasonic transmitter that emits ultrasonic waves on the road surface; an ultrasonic receiver for receiving a ultrasonic signal reflected from the road surface and generating a reflection signal; and a road surface type estimator using the reflection signal generated by the ultrasonic receiver, the road surface type estimator performing the method according to claim 1. ### Claim 7 (depends on 6) The system according to claim 6, wherein the road surface type includes at least one of asphalt, cement, dirt, ice, marble, applied paint, snow, water, and lanes marked on the road surface. ### Claim 8 (depends on 6) The system according to claim 6, wherein the sampling frequency of the ultrasonic transmitter and the ultrasonic receiver is in a range of 20K Samples/sec to 1M Samples/sec. ### Claim 9 (depends on 6) The system according to claim 6, wherein the ultrasonic transmitter, the ultrasonic receiver and the road surface type estimator are mounted on a moving body. ### Claim 10 An apparatus for estimation road surface type using an ultrasonic signal, comprising: at least one processor; and, at least one memory storing computer-executable instructions, wherein the computer-executable instructions stored in said at least one memory, when executed by the at least one processor, causes the at least one processor to perform operations comprising: (a) extracting a 1D feature vector through one or more 1D convolutional layers using an input signal derived from a reflection signal in the time domain of the ultrasonic signal reflected from the road surface after being emitted to the road surface; (b) receiving the 1D feature vector as an input and estimating a probability value for each road surface type in an artificial neural network having one or more layers; and, (c) determining the road surface type from the estimated probability value for each road surface type. ### Claim 11 (depends on 10) The apparatus according to claim 10, wherein a road surface friction coefficient is estimated from the road surface type. ### Claim 12 An apparatus for estimation road surface type using an ultrasonic signal, comprising: at least one processor; a memory for storing a convolution filter for convolution operation; a feature extractor for extracting a 1D feature vector through one or more 1D convolutional layers using an input signal derived from a reflection signal in the time domain of the ultrasonic signal reflected from the road surface after being emitted to the road surface; and a classifier that receives the 1D feature vector as an input and estimates a probability value for each road surface type in an artificial neural network having one or more layers. ### Claim 13 (depends on 12) The apparatus according to claim 12, further comprising: a Short-Time Fourier Transform converter that divides the reflection signal in the time domain at predetermined time intervals and performs Short-Time Fourier Transform on each divided reflection signal to generate a signal in the time-frequency domain, wherein the signal in the time-frequency domain is the input signal of the one or more 1D convolutional layers. ### Claim 14 (depends on 12) The apparatus according to claim 12, wherein the input signal of the one or more 1D convolutional layers of the feature extractor is the reflection signal in the time domain. ### Claim 15 (depends on 12) The apparatus according to claim 12, wherein the road surface type includes at least one of asphalt, cement, dirt, ice, marble, applied paint, snow, water, and lanes marked on the road surface.