# US 10,504,214 B2 · System and method for image presentation by a vehicle driver assist module

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,504,214 B2
- Document ID: US10504214B2
- Status: Granted
- Kind: B2
- Application number: 16/193,650
- Filed: 2018-11-16
- Published: 2019-03-21
- Granted: 2019-12-10
- Earliest claimed: 2017-02-03
- Assignees: Harman International Industries, Incorporated
- Applicants: Harman International Industries, Incorporated
- Inventors: Pranjal Chakraborty; Rajesh Biswal; Debashis Mukherjee
- Web page: https://patentgrep.com/patent/US10504214B2

## CPC

- G06T 5/73: Deblurring; Sharpening
- B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
- B60K 35/23: Head-up displays [HUD] (optical aspects of head-up displays G02B27/01)
- B60R 1/24: in front of the vehicle
- G02B 27/0172: {characterised by optical features}
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- H04N 7/183: {for receiving images from a single remote source}
- B60K 35/22: Display screens
- B60K 35/81: for controlling displays
- B60K 2360/21: using cameras
- B60K 2360/334: Projection means
- B60R 2300/205: using a head-up display
- B60R 2300/207: using multi-purpose displays, e.g. camera image and navigation or video on same display
- B60R 2300/307: virtually distinguishing relevant parts of a scene from the background of the scene
- B60R 2300/8033: for pedestrian protection
- B60R 2300/8093: for obstacle warning
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0141: {characterised by the informative content of the display}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present disclosure relates to systems, devices and methods for image presentation for vehicle driver assistance. In one embodiment, a method includes receiving image data representing an outside of a vehicle and enhancing the image data. Image enhancement can include inverting the image data, running a dehazing operation based on scene radiance, medium transmission and atmospheric light of the image data. The method may also include generating reconstructed image data from the enhanced image data by performing a sparse representation of the enhanced image data for denoising of the enhanced image data, and performing a learning operation to identify and enhance objects of interest in the enhanced image data with the sparse representation. The method also includes outputting the reconstructed image data to a display unit of the vehicle. Vehicle driver assist systems and devices are provided to enhance image data detected by vehicle cameras and wearable devices.

## Claims

### Claim 1

A method for image presentation by a vehicle driver assist module, the method comprising:
receiving, by a driver assist module, image data representing an outside of a vehicle, wherein the image data representing an outside of a vehicle includes image data captured by a wearable device associated with an operator of the vehicle;
inverting, by the driver assist module, the image data to form inverted image data;
running, by the driver assist module, a dehazing operation on inverted image data to generate enhanced inverted image data of the outside of the vehicle, wherein the dehazing operation is based on scene radiance, medium transmission and atmospheric light of the image data;
inverting, by the driver assist module, the enhanced inverted image data to generate enhanced image data;
generating, by the driver assist module, reconstructed image data from the enhanced image data, wherein the reconstructed image data is generated by
performing a sparse representation of the enhanced image data for denoising of the enhanced image data, and
performing a learning operation to identify and enhance objects of interest in the enhanced image data with the sparse representation; and
outputting, by the driver assist module, the reconstructed image data to a display unit of the vehicle.

### Claim 2 (depends on 1)

The method of claim 1, wherein the image data representing an outside of a vehicle is wirelessly communicated to the driver assist module.

### Claim 3 (depends on 1)

The method of claim 1, wherein the device includes a first camera and a second camera mounted to an eyewear frame.

### Claim 4 (depends on 3)

The method of claim 3, wherein the first camera and the second camera provide real time video capture.

### Claim 5 (depends on 1)

The method of claim 1, wherein outputting the reconstructed image data includes superimposing outline elements associated with objects identified for the reconstructed image data.

### Claim 6 (depends on 1)

The method of claim 1, wherein outputting the reconstructed image data includes at least one of a detected object class and display element box coordinates to add bounding boxes to the reconstructed image.

### Claim 7 (depends on 1)

The method of claim 1, wherein outputting the reconstructed image data includes adding colorization to pixels labeled as associated with a detected object.

### Claim 8

A vehicle driver assist module comprising:
an input module to receive image data representing an outside of a vehicle, wherein the image data representing an outside of a vehicle includes image data captured by a wearable device associated with an operator of the vehicle;
an output module; and
an image enhancer module coupled to the input module and output module, the image enhancer module including a processor configured to
invert the image data to form inverted image data;
run a dehazing operation on inverted image data to generate enhanced inverted image data of the outside of the vehicle, wherein the dehazing operation is based on scene radiance, medium transmission and atmospheric light of the image data;
invert the enhanced inverted image data to generate enhanced image data;
generate reconstructed image data from the enhanced image data, wherein the reconstructed image data is generated by
performing a sparse representation of the enhanced image data for denoising of the enhanced image data, and
performing a learning operation to identify and enhance objects of interest in the enhanced image data with the sparse representation; and
control the output module to output the reconstructed image data to a display unit of the vehicle.

### Claim 9 (depends on 8)

The device of claim 8, wherein the image data representing an outside of a vehicle is wirelessly communicated to the driver assist module.

### Claim 10 (depends on 8)

The device of claim 8, wherein the device includes a first camera and a second camera mounted to an eyewear frame.

### Claim 11 (depends on 10)

The device of claim 10, wherein the first camera and the second camera provide real time video capture.

### Claim 12 (depends on 8)

The device of claim 8, wherein outputting the reconstructed image data includes superimposing outline elements associated with objects identified for the reconstructed image data.

### Claim 13 (depends on 8)

The device of claim 8, wherein outputting the reconstructed image data includes at least one of a detected object class and display element box coordinates to add bounding boxes to the reconstructed image.

### Claim 14 (depends on 8)

The device of claim 8, wherein outputting the reconstructed image data includes adding colorization to pixels labeled as associated with a detected object.

### Claim 15

A method for image presentation by a vehicle driver assist module, the method comprising:
receiving, by a driver assist module, image data representing an outside of a vehicle;
inverting, by the driver assist module, the image data to form inverted image data;
running, by the driver assist module, a dehazing operation on inverted image data to generate enhanced inverted image data of the outside of the vehicle, wherein the dehazing operation is based on scene radiance, medium transmission and atmospheric light of the image data;
inverting, by the driver assist module, the enhanced inverted image data to generate enhanced image data;
generating, by the driver assist module, reconstructed image data from the enhanced image data, wherein the reconstructed image data is generated by
performing a sparse representation of the enhanced image data for denoising of the enhanced image data, and
performing a learning operation to identify and enhance objects of interest in the enhanced image data with the sparse representation; and
outputting, by the driver assist module, the reconstructed image data to a display unit of the vehicle, wherein outputting the reconstructed image data includes superimposing outline elements associated with objects identified for the reconstructed image data.

### Claim 16 (depends on 15)

The method of claim 15, wherein the image data representing an outside of a vehicle is wirelessly communicated to the driver assist module.

### Claim 17 (depends on 15)

The method of claim 15, wherein the image data representing an outside of a vehicle includes image data captured by a wearable device associated with an operator of the vehicle, and wherein the device includes a first camera and a second camera mounted to an eyewear frame.

### Claim 18 (depends on 17)

The method of claim 17, wherein the first camera and the second camera provide real time video capture.

### Claim 19 (depends on 15)

The method of claim 15, wherein outputting the reconstructed image data includes at least one of a detected object class and display element box coordinates to add bounding boxes to the reconstructed image.

### Claim 20 (depends on 15)

The method of claim 15, wherein outputting the reconstructed image data includes adding colorization to pixels labeled as associated with a detected object.