# US 2020/0396394 A1 · Method for generating an output image showing a motor vehicle and an environmental region of the motor vehicle in a predetermined target view, camera system as well as motor vehicle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0396394 A1
- Document ID: US20200396394A1
- Status: Published application
- Kind: A1
- Application number: 16/753,974
- Filed: 2018-10-10
- Published: 2020-12-17
- Earliest claimed: 2017-10-10
- Assignees: Connaught Electronics Ltd.
- Applicants: Connaught Electronics Ltd.
- Inventors: Vladimir Zlokolica; Mark Patrick Griffin; Brian Michael Thomas Deegan; Barry Dever; John Maher
- Web page: https://patentgrep.com/patent/US20200396394A1

## CPC

- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- H04N 5/265: Mixing
- B60R 1/27: providing all-round vision, e.g. using omnidirectional cameras
- B60R 11/04: Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
- G06T 5/10: using non-spatial domain filtering
- G06T 5/20: using local operators
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/70: Denoising; Smoothing
- G06T 5/73: Deblurring; Sharpening
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- B60R 2300/105: using multiple cameras
- B60R 2300/303: using joined images, e.g. multiple camera images
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/20064: Wavelet transform [DWT]
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR

## Abstract

The invention relates to a method for generating an output image with a predefined target view showing a motor vehicle (1) and an environmental region (4) of the motor vehicle (1) based on at least partially overlapping raw images (RC1, RC2, RC3, RC4) captured by at least two vehicle-side cameras (5a, 5b, 5c, 5d), comprising the steps of: specifying respective camera-specific pixel density maps (PDM1a, PDM1b, PDM2a, PDM2b), which each describe an image-region dependent distribution of a number of pixels of the raw image (R1 to R4) captured by the associated camera (5a to 5d) contributing for the generation of the output image, spatially adaptive filtering of the raw images (RC1 to RC4) based on the pixel density map (PDM1a to PDM2b) specific to the associated camera (5a to 5d), which indicates an image region-dependent extent of the filtering, identifying mutually corresponding image areas (B1a, B1b, B2a, B2b, B3a, B3b, B4a, B4b) in the at least partially overlapping raw images (RC1 to RC4) of the at least two cameras (5a to 5d), spatially adaptive filtering of the image area (B1a to B4b) of the raw image (RC1 to RC4) of the one camera (5a to 5d) based on the pixel density map (PDM1a to PDM2b) specific to the respective other camera (5a to 5d) for reducing a sharpness difference between the corresponding image areas (B1a to B4b), remapping the filtered raw images (RC1 to RC4) to an image surface corresponding to the target view for generating remapped filtered raw images (R1, R2, R3, R4), generating the output image by combining the remapped filtered raw images (R1 to R4). The invention moreover relates to a camera system (3) as well as to a motor vehicle (1).

## Claims

### Claim 1

A method for generating an output image with a predefined target view showing a motor vehicle and an environmental region of the motor vehicle based on at least partially overlapping raw images captured by at least two vehicle-side cameras, the method comprising:
specifying respective camera-specific pixel density maps, which each describe an image-region dependent distribution of a number of pixels of the raw image captured by the associated camera contributing for the generation of the output image;
spatially adaptive filtering of the raw images based on the pixel density map specific to the associated camera, which indicates an image region-dependent extent of the filtering;
identifying mutually corresponding image areas in the at least partially overlapping raw images of the at least two cameras;
spatially adaptive filtering of the image area of the raw image of the one camera based on the pixel density map specific to the respective other camera for reducing a sharpness difference between the corresponding image areas;
remapping the filtered raw images to an image surface corresponding to the target view for generating remapped filtered raw images; and
generating the output image by combining the remapped filtered raw images.

### Claim 2 (depends on 1)

The method according to claim 1, wherein for each camera a horizontal and a vertical pixel density map is determined for indicating respective image regions of the raw images to be filtered.

### Claim 3 (depends on 1)

The method according to claim 1, wherein for each camera, a camera-specific sharpness mask is defined as a function of the pixel density map specific to the associated camera and as a function of the pixel density map specific to the respective other camera, wherein the raw images are spatially adaptively filtered as a function of the respective sharpness mask of the camera capturing the respective raw image.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the camera-specific sharpness masks are additionally defined as a function of at least one camera property of the associated camera.

### Claim 5 (depends on 4)

The method according to claim 4, wherein as the at least one camera property, a lens property of the respective camera and/or at least one extrinsic camera parameter of the respective camera and/or at least one intrinsic camera parameter of the respective camera is predefined.

### Claim 6 (depends on 3)

The method according to claim 3, wherein for each camera, a camera-specific, spatially adaptive filter scheme for spatially adaptively filtering the respective raw image is determined in dependence on the associated camera-specific sharpness mask.

### Claim 7 (depends on 6)

The method according to claim 6, wherein for spatially adaptive filtering of the corresponding image areas in the at least two, at least partially overlapping raw images, an image content of the image area in a first of the raw images is sharpened by means of the filter scheme specific to the camera capturing the first raw image, and an image content of the corresponding image area in a second of the raw images is blurred or not filtered by means of the filter scheme specific to the camera capturing the second raw image.

### Claim 8 (depends on 6)

The method according to claim 6, wherein the camera-specific, spatially adaptive filter scheme is determined based on a multiscale and multi-orientation gradient approach.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the spatially adaptive filter scheme is determined based on a non-decimated wavelet transform, wherein wavelet coefficients are spatially adaptively modified based on the camera-specific sharpness mask.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the wavelet coefficients are adaptively modified based on a transfer-tone-mapping function, which is aligned on the camera-specific sharpness mask.

### Claim 11 (depends on 9)

The method according to claim 9, wherein for determining the camera-specific, spatially adaptive filter scheme, at least two camera-specific sharpness masks are defined for at least two wavelet bands of the wavelet transform as a function of the pixel density map specific to the associated camera and as a function of the pixel density map specific to the respective other camera.

### Claim 12 (depends on 9)

The method according to claim 9, wherein the at least one sharpness mask is combined with spatially neighbouring area statistics in wavelet bands, wherein the statistics describes a correlation of the wavelet coefficients in spatially neighbouring areas in each wavelet band separately.

### Claim 13

A camera system for a motor vehicle, comprising:
at least two cameras for capturing at least partially overlapping raw images from an environmental region of the motor vehicle; and
an image processing device configured to generate an output image with a predefined target view showing the motor vehicle and an environmental region of the motor vehicle based on the at least partially overlapping raw images by:
specifying respective camera-specific pixel density maps, which each describe an image-region dependent distribution of a number of pixels of the raw image captured by the associated camera contributing for the generation of the output image,
spatially adaptive filtering of the raw images based on the pixel density map specific to the associated camera, which indicates an image region-dependent extent of the filtering,
identifying mutually corresponding image areas in the at least partially overlapping raw images of the at least two cameras,
spatially adaptive filtering of the image area of the raw image of the one camera based on the pixel density map specific to the respective other camera for reducing a sharpness difference between the corresponding image areas,
remapping the filtered raw images to an image surface corresponding to the target view for generating remapped filtered raw images, and
generating the output image by combining the remapped filtered raw images.

### Claim 14 (depends on 13)

A motor vehicle comprising a camera system according to claim 13.
