# US 12,223,584 B2 · Method for simulating the effects of the optical quality of windshield

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,584 B2
- Document ID: US12223584B2
- Status: Granted
- Kind: B2
- Application number: 18/248,317
- Filed: 2021-09-28
- Published: 2023-11-30
- Granted: 2025-02-11
- Earliest claimed: 2020-10-08
- Assignees: Saint-Gobain Glass France
- Applicants: Saint-Gobain Glass France
- Inventors: Jean-Baptiste Laudereau; Mathieu Berard; Tatiana Severin-Fabiani; Nicolas Mielec
- Web page: https://patentgrep.com/patent/US12223584B2

## CPC

- G06T 15/06: Ray-tracing
- G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G06T 5/80: Geometric correction
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 15/04: Texture mapping
- G06T 15/20: Perspective computation
- G06T 15/506: {Illumination models}
- G06V 10/776: Validation; Performance evaluation
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06T 2207/20081: Training; Learning
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for simulating the effects of the optical quality of windshield onto the image recording quality of a digital image recording device, in particular a digital image recording device for an advanced or automated driver-assistance system. The method is based on a combination of an adapted stochastic ray tracing method and of a convolutional image processing to simulate the effects of the optical distortions of a windshield on the image recording quality of a digital image recording device. The simulation is performed from a measured optical quality function of the windshields related to those optical distortions.

## Claims

### Claim 1

A computer implemented method for generating a map of point spread function convolutional kernels which simulates effects of optical distortions of a windshield on an image recording quality of a digital image recording device,
wherein said method takes as input a measured optical quality function related to the optical distortions of the windshield,
wherein said method provides as output a map of point spread function convolutional kernels simulating the effects of optical distortions of a windshield on the image recording quality of the digital image recording device, wherein said method comprises the following steps:
(a) modelling at least one sheet of transparent mineral glass comprising two main parallel faces, wherein a surface of at least one of said two main parallel faces is textured with the measured optical quality function, and wherein the sheet of mineral glass is placed in front of a grid of point sources and is inclined, in respect to said grid, with an installation angle of said windshield in a transporting vehicle;
(b) calculating, with a stochastic ray tracing method, a global illuminance arriving through the modelled inclined sheet of transparent mineral glass from the grid used as a light source, and
(c) computing a projection of the global illuminance for each point source of the grid from the view frustum of a virtual camera with an optical camera model, wherein said virtual camera is placed in front of an opposite main face of the sheet of transparent mineral glass with respect to the grid used as a light source and at a position corresponding to an installation position of the digital image recording device, wherein the projected grid of point sources is a map of point spread function convolutional kernels, wherein each point spread function convolutional kernel is associated to a point source of the grid.

### Claim 2 (depends on 1)

The computer implemented method according to claim 1, further comprising, after step (c), a step of interpolating point spread function convolutional kernels for additional non projected point sources located between source points of the grid from the point spread function convolutional kernels associated to neighbour projected point sources of said non projected point sources.

### Claim 3 (depends on 2)

The computer implemented method according to claim 2, wherein the interpolating step is performed from only selected parameters of the point spread function convolutional kernels associated to neighbour projected point sources.

### Claim 4 (depends on 1)

The computer implemented method according to claim 1, further comprising step of computing a map of position shifts, wherein each position shifts of said map is the position shift between a point source of the grid and the corresponding point spread function convolutional kernel, wherein the map of point spread function convolutional kernels comprises said map of position shifts as output.

### Claim 5 (depends on 1)

The computer implemented method according to claim 1, wherein the optical camera model is represented as a recording sensor pixels in size and optical system, wherein the half size of the grid of point source is calculated with the following formulas
{
S
x
=
D
×
tan
⁢
(
H
×
π
1
⁢
8
⁢
0
)
S
y
=
D
×
tan
⁢
(
V
×
π
1
⁢
8
⁢
0
)
S
z
=
D
wherein
H
=
H
⁢
F
⁢
O
⁢
V
2
⁢
and
⁢
V
=
N
x
N
y
×
H
,
wherein HFOV is the horizontal view frustum of a virtual camera and D is a distance between the sensor and the grid of point sources.

### Claim 6 (depends on 5)

The computer implemented method according to claim 5, wherein the grid comprises a point source for each pixel of the sensor or one same point source for a given number of pixels of the sensor.

### Claim 7 (depends on 1)

The computer implemented method according to claim 1, wherein steps (a)-(c) are reiterated for grid of point sources located at different distances from the virtual camera in order to form a 3D map of point source function convolutional kernels for a given region of the view frustum of said camera.

### Claim 8 (depends on 1)

The computer implemented method according to claim 1, wherein the measured optical quality function related to the optical distortions of the windshield is the measured transmitted wavefront error of the windshield, the measured surfaces profiles and/or the measured distribution of complex refractive index.

### Claim 9 (depends on 1)

The computer implemented method according to claim 1, wherein the grid is digitally preprocessed as an environment map projected onto an inside side of an environment sphere centered on the virtual camera.

### Claim 10 (depends on 1)

The computer implemented method according to claim 1, wherein, in step (a), the sheet of transparent mineral glass is modeled so that said sheet of transparent mineral glass represents only the demarcated zone of the windshield in front of which a digital image recording device is to be placed.

### Claim 11 (depends on 1)

The computer implemented method according to claim 1, wherein, in step (a), the sheet of transparent mineral glass is modelled so that the global illuminance calculated at step (c) is only the global luminance arriving through the part of modelled sheet of transparent mineral glass which corresponds to a demarcated zone of the windshield through which a digital image recording device may record an image.

### Claim 12 (depends on 1)

The computer implemented method according to claim 1, wherein the texture of the textured surface is modeled with a bump map or a displacement map.

### Claim 13 (depends on 1)

A computer implemented method for simulating effects of optical distortions of a windshield on an image recording quality of a digital image recording device,
wherein said method takes as input an image of a landscape likely to be recorded by a digital image recording device through said windshield, and a map of point spread functions convolutional kernels which is generated with a computer implemented method according to claim 1 for said windshield, and
wherein said method provides as output an image of said input image of a landscape as viewed through said windshield,
wherein said method comprises a step of computing a convolution operation of the image of a landscape with the map of point spread functions convolutional kernels.

### Claim 14 (depends on 1)

A data processing system comprising processor and a memory coded with instructions, which when executed by the processor, carry out a method according to claim 1.

### Claim 15 (depends on 1)

A non-transitory machine readable medium comprising instructions which, when the instructions are executed by a computer, causes the computer to carry out a method according to claim 1.

### Claim 16 (depends on 1)

A process comprising evaluating the optical quality of windshields for a use with a digital image recording device using the method according to claim 1.

### Claim 17 (depends on 13)

A process for evaluating the performances of an object detection and classification algorithm of an automated driving and advanced safety system, wherein said process comprises the following steps:
(a) providing a set of measured optical quality functions related to an optical distortions of a set of windshields;
(b) providing a set of images of landscapes likely to be recorded by a digital image recording device through the windshield of said set of windshields;
(c) using a computer implemented method according to claim 13 for each image of the set of images of landscapes and each measured optical quality function of the set measured optical quality functions related to the optical distortions, in order to provide a set of images 2001 of a landscape as viewed through the windshield of said set of windshields;
(d) feeding the set of images obtained at step (c) to an object detection and classification algorithm of an automated driving and advanced safety system, and
(e) monitoring performance parameters in the object detection and classification of said algorithm while the algorithm is processing the set of images fed at step (d).

### Claim 18 (depends on 13)

A data processing system comprising a processor and a memory coded with instructions, which when executed by the processor, carry out a method according to claim 13.

### Claim 19 (depends on 13)

A non-transitory machine readable medium comprising instructions which, when the instructions are executed by a computer, causes the computer to carry out a method according to claim 13.

### Claim 20 (depends on 16)

The process according to claim 16, wherein the digital image recording device is a digital image recording device of an automated driving and advanced safety system.