# US 2024/0200974 A1 · Method, apparatus, and computer program product for road surface anomaly detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0200974 A1 - Document ID: US20240200974A1 - Status: Published application - Kind: A1 - Application number: 18/067,342 - Filed: 2022-12-16 - Published: 2024-06-20 - Earliest claimed: 2022-12-16 - Assignees: HERE Global B.V. - Applicants: Here Global B.V. - Inventors: Jeremy Michael YOUNG; Jerome BEAUREPAIRE; Leon STENNETH - Web page: https://patentgrep.com/patent/US20240200974A1 ## CPC - G01C 21/3822: {Road feature data, e.g. slope data} - B60G 17/0165: to an external condition, e.g. rough road surface, side wind - B60W 10/22: including control of suspension systems - B60W 40/06: Road conditions - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - G01C 21/3848: {Data obtained from both position sensors and additional sensors} - G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) - G06V 10/764: using classification, e.g. of video objects - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - B60W 2050/0083: {Setting, resetting, calibration} - B60W 2050/143: {Alarm means} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2552/35: Road bumpiness, e.g. potholes - B60W 2556/40: High definition maps - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - B60G: VEHICLE SUSPENSION ARRANGEMENTS (air-cushion vehicles B60V; {cycle suspensions B62K25/00}) - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method, apparatus, and computer program product are provided for using visual analysis of a road surface to identify markings indicative of road surface anomalies. Methods may include, for example: receiving data from an image sensor associated with a vehicle traveling along a road segment; identifying, within the data, a visual indication of a road surface anomaly, where the visual indication is a result of the road surface anomaly; and cause at least one of: an indication of the road surface anomaly to be provided to a user of the vehicle; a vehicle operational setting to be changed responsive to the road surface anomaly; or a map update to be generated to include the road surface anomaly within a map database. ## Claims ### Claim 1 An apparatus comprising at least one processor and at least one non-transitory memory including computer program code instructions, the computer program code instructions configured to, when executed, cause the apparatus to at least: receive data from an image sensor associated with a vehicle traveling along a road segment; identify, within the data, a visual indication of a road surface anomaly, wherein the visual indication is a result of the road surface anomaly; and cause at least one of: an indication of the road surface anomaly to be provided to a user of the vehicle; a vehicle operational setting to be changed responsive to the road surface anomaly; or a map update to be generated to include at least one of the road surface anomaly or the visual indication of the road surface anomaly within a map database. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the visual indication of the road surface anomaly is not the road surface anomaly itself. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the road surface anomaly comprises a non-flat road surface, and wherein the visual indication comprises marks on a road surface caused by vehicles traversing the road surface anomaly. ### Claim 4 (depends on 3) The apparatus of claim 3, wherein the marks on the road surface caused by vehicles traversing the road surface anomaly are past the road surface anomaly in a direction of travel along the road segment. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein causing the apparatus to identify, within the data, the visual indication of the road surface anomaly comprises causing the apparatus to: process data from the image sensor using a machine learning model; and identify, within the data from the image sensor, the visual indication of the road surface anomaly using the machine learning model. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the apparatus is further caused to: receive motion sensor data from a motion sensor associated with the vehicle; and determine a severity of the road surface anomaly based, at least in part, on the motion sensor data. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the indication to be provided to the user of the vehicle comprises a recommended speed for the vehicle with which to traverse the road surface anomaly. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the vehicle operational setting to be changed responsive to the road surface anomaly comprises a change to a firmness of a suspension of the vehicle. ### Claim 9 A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to: receive data from an image sensor associated with a vehicle traveling along a road segment; identify, within the data, a visual indication of a road surface anomaly, wherein the visual indication is a result of the road surface anomaly; and cause at least one of: an indication of the road surface anomaly to be provided to a user of the vehicle; a vehicle operational setting to be changed responsive to the road surface anomaly; or a map update to be generated to include at least one of the road surface anomaly or the visual indication of the road surface anomaly within a map database. ### Claim 10 (depends on 9) The computer program product of claim 9, wherein the visual indication of the road surface anomaly is not the road surface anomaly itself. ### Claim 11 (depends on 9) The computer program product of claim 9, wherein the road surface anomaly comprises a non-flat road surface, and wherein the visual indication comprises marks on a road surface caused by vehicles traversing the road surface anomaly. ### Claim 12 (depends on 11) The computer program product of claim 11, wherein the marks on the road surface caused by vehicles traversing the road surface anomaly are past the road surface anomaly in a direction of travel along the road segment. ### Claim 13 (depends on 9) The computer program product of claim 9, wherein the program code instructions to identify, within the data, the visual indication of a road surface anomaly comprise program code instructions to: process data from the image sensor using a machine learning model; and identify, within the data from the image sensor, the visual indication of the road surface anomaly using the machine learning model. ### Claim 14 (depends on 9) The computer program product of claim 9, further comprising program code instructions to: receive motion sensor data from a motion sensor associated with the vehicle; and determine a severity of the road surface anomaly based, at least in part, on the motion sensor data. ### Claim 15 (depends on 9) The computer program product of claim 9, wherein the indication to be provided to the user of the vehicle comprises a recommended speed for the vehicle with which to traverse the road surface anomaly. ### Claim 16 (depends on 9) The computer program product of claim 9, wherein the vehicle operational setting to be changed responsive to the road surface anomaly comprises a change to a firmness of a suspension of the vehicle. ### Claim 17 A method comprising: receiving data from an image sensor associated with a vehicle traveling along a road segment; identifying, within the data, a visual indication of a road surface anomaly, wherein the visual indication is a result of the road surface anomaly; and causing at least one of: an indication of the road surface anomaly to be provided to a user of the vehicle; a vehicle operational setting to be changed responsive to the road surface anomaly; or a map update to be generated to include at least one of the road surface anomaly or the visual indication of the road surface anomaly within a map database. ### Claim 18 (depends on 17) The method of claim 17, wherein the visual indication of the road surface anomaly is not the road surface anomaly itself. ### Claim 19 (depends on 17) The method of claim 17, wherein the road surface anomaly comprises a non-flat road surface, and wherein the visual indication comprises marks on a road surface caused by vehicles traversing the road surface anomaly. ### Claim 20 (depends on 19) The method of claim 19, wherein the marks on the road surface caused by vehicles traversing the road surface anomaly are past the road surface anomaly in a direction of travel along the road segment.