# US 2022/0036097 A1 · Road surface detection device and road surface detection program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036097 A1
- Document ID: US20220036097A1
- Status: Published application
- Kind: A1
- Application number: 17/299,625
- Filed: 2019-11-29
- Published: 2022-02-03
- Earliest claimed: 2018-12-04
- Assignees: Aisin Corporation
- Applicants: Aisin Corporation
- Inventors: Atsuto OGINO; Kaisei HASHIMOTO
- Web page: https://patentgrep.com/patent/US20220036097A1

## CPC

- B60W 40/06: Road conditions
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G01B 11/245: using a plurality of fixed, simultaneously operating transducers ({G01B11/2408 - G01B11/2425, } G01B11/255 take precedence)
- G01C 7/04: involving a vehicle which moves along the profile to be traced
- G06T 5/70: Denoising; Smoothing
- G06T 7/593: from stereo images
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06V 20/653: {by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces}
- H04N 7/188: {Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253)}
- H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- H04N 13/221: using the relative movement between cameras and objects
- H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory}
- B60W 2420/403: Image sensing, e.g. optical camera
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30256: Lane; Road marking
- G06T 2219/2016: Rotation, translation, scaling
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A road surface detection device according to an embodiment includes an image acquisition unit that acquires captured image data output from a stereo camera that captures an imaging area including a road surface on which a vehicle travels, a three-dimensional model generation unit that generates a three-dimensional model of the imaging area including a surface shape of the road surface from a viewpoint of the stereo camera based on the captured image data, and a correction unit that estimates a plane from the three-dimensional model, and corrects the three-dimensional model so as to match an orientation of a normal vector of the plane and a height position of the plane with respect to the stereo camera with a correct value of an orientation of a normal vector of the road surface and a correct value of a height position of the road surface with respect to the stereo camera, respectively.

## Claims

### Claim 1

A road surface detection device comprising:
an image acquisition unit that acquires captured image data output from a stereo camera that captures an imaging area including a road surface on which a vehicle travels;
a three-dimensional model generation unit that generates a three-dimensional model of the imaging area including a surface shape of the road surface from a viewpoint of the stereo camera based on the captured image data; and
a correction unit that estimates a plane from the three-dimensional model, and corrects the three-dimensional model so as to match an orientation of a normal vector of the plane and a height position of the plane with respect to the stereo camera with a correct value of an orientation of a normal vector of the road surface and a correct value of a height position of the road surface with respect to the stereo camera, respectively.

### Claim 2 (depends on 1)

The road surface detection device according to claim 1, wherein
the correction unit matches the orientation of the normal vector of the plane with the correct value of the orientation of the normal vector of the road surface, and then corrects an entirety of the three-dimensional model so as to match the height position of the plane with respect to the stereo camera with the correct value of the height position of the road surface with respect to the stereo camera.

### Claim 3 (depends on 2)

The road surface detection device according to claim 2, wherein
the correction unit acquires the correct value of the orientation of the normal vector of the road surface and the correct value of the height position of the road surface with respect to the stereo camera based on values determined during calibration of the stereo camera.

### Claim 4

A road surface detection computer program product having a non-transitory computer readable medium including programmed instructions, wherein the instructions, when executed by a computer, cause the computer to perform:
acquiring captured image data output from a stereo camera that captures an imaging area including a road surface on which a vehicle travels;
generating a three-dimensional model of the imaging area including a surface shape of the road surface from a viewpoint of the stereo camera based on the captured image data; and
estimating a plane from the three-dimensional model, and correcting the three-dimensional model so as to match an orientation of a normal vector of the plane and a height position of the plane with respect to the stereo camera with a correct value of an orientation of a normal vector of the road surface and a correct value of a height position of the road surface with respect to the stereo camera, respectively.