# US 11,115,636 B2 · Image processing apparatus for around view monitoring

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,115,636 B2
- Document ID: US11115636B2
- Status: Granted
- Kind: B2
- Application number: 16/317,413
- Filed: 2017-07-14
- Published: 2019-08-15
- Granted: 2021-09-07
- Earliest claimed: 2016-07-14
- Assignees: LG Innotek Co., Ltd.
- Applicants: LG Innotek Co., Ltd.
- Inventors: Sung Hyun Lim; Je Yong Shin
- Web page: https://patentgrep.com/patent/US11115636B2

## CPC

- G06T 3/4015: Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
- G06F 13/38: Information transfer, e.g. on bus (G06F13/14 takes precedence)
- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/80: Geometric correction
- G06T 7/90: Determination of colour characteristics
- G06T 11/40: Filling planar surfaces by adding surface attributes, e.g. adding colours or textures
- G06T 11/60: Creating or editing images; Combining images with text
- H04N 1/387: Composing, repositioning or otherwise {geometrically} modifying originals
- H04N 1/3876: {Recombination of partial images to recreate the original image}
- H04N 9/646: {for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters}
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- H04N 23/73: by influencing the exposure time
- H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06T 2207/10024: Color image
- G06T 2207/10144: Varying exposure
- G06T 2207/20192: Edge enhancement; Edge preservation
- G06T 2207/20212: Image combination
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Disclosed are an image processing apparatus including a deserializer receiving respective Bayer image information pieces acquired from a plurality of cameras; and an image processor processing Bayer data processed and output by the deserializer to produce one stitched image from a plurality of Bayer images acquired from the plurality of cameras, wherein the one stitched image is output.

## Claims

### Claim 1

An image processing apparatus, comprising:
a deserializer configured to receive respective Bayer image information pieces acquired from a plurality of cameras; and
an image processor configured to:
process Bayer data processed and output by the deserializer to produce one stitched image from a plurality of Bayer images acquired from the plurality of cameras,
after producing the one stitched image, perform a demosaicing operation on the one stitched image to generate a stitched demosaiced image,
perform an image signal processing (ISP) operation on the stitched demosaiced image to generate an output stitched image that is demosaiced and corrected, and
adjusting an auto exposure and an auto white balance of one or more of the plurality of cameras based on the output stitched image,
wherein the output stitched image is output, and
wherein the ISP operation includes edge enhancement and at least one of gamma correction, color correction, auto exposure correction or auto white balance adjustment.

### Claim 2 (depends on 1)

The image processing apparatus of claim 1, wherein each of the Bayer image information pieces is Monochrome data, and the stitched demosaiced image corresponds to data including color of an RGB or YUV manner.

### Claim 3 (depends on 1)

The image processing apparatus of claim 1, wherein each of the Bayer image information pieces is transmitted through an independent network line.

### Claim 4

An image processing apparatus, comprising:
a reception unit configured to receive a plurality of different data obtained by serializing Bayer image information through a plurality of independent network lines; and
an image processor configured to:
process a plurality of Bayer data to output one stitched image,
after producing the one stitched image, perform a demosaicing operation on the one stitched image to generate a stitched demosaiced image,
perform an image signal processing (ISP) operation on the stitched demosaiced image to generate an output stitched image that is demosaiced and corrected, and
adjust an auto exposure and an auto white balance of one or more of a (plurality of cameras based on the output stitched image,
wherein the stitched demosaiced image is an RGB or YUV image, and
wherein the ISP operation includes edge enhancement and at least one of gamma correction, color correction, auto exposure correction or auto white balance adjustment.

### Claim 5 (depends on 4)

The image processing apparatus of claim 4, wherein the reception unit includes a deserializer for restoring and outputting the plurality of data transmitted through the plurality of independent network lines.

### Claim 6 (depends on 4)

The image processing apparatus of claim 4, wherein the reception unit includes:
a multiplexer configured to receive the plurality of data from the plurality of independent network lines; and
a deserializer configured to restore the plurality of data that are sequentially transmitted through the multiplexer to output the plurality of Bayer data.

### Claim 7 (depends on 4)

The image processing apparatus of claim 4, wherein the network line is operated using a low-voltage differential signaling (LVDS) method.

### Claim 8 (depends on 4)

The image processing apparatus of claim 4, wherein the image processor includes:
at least one interpolation correction unit configured to perform color interpolation and correction processing on the plurality of Bayer data to output a plurality of image data; and
a conversion and stitching unit configured to convert the plurality of image data to correspond to the stitched image and then stitch the converted image.

### Claim 9 (depends on 8)

The image processing apparatus of claim 8, further comprising a storage unit configured to store the plurality of image data output from the at least one interpolation correction unit and transmit the plurality of image data according to a request of the conversion and stitching unit.

### Claim 10

An image processing apparatus, comprising:
a plurality of camera devices configured to output a Bayer pattern;
an in-vehicle network (IVN) transmitting the Bayer pattern; and
an image processor configured to:
process the Bayer pattern transmitted through the IVN to produce one stitched image, and
after producing the one stitched image, perform a demosaicing operation on the one stitched image to generate a stitched demosaiced image,
perform an image signal processing (ISP) operation on the stitched demosaiced image to generate an output stitched image that is demosaiced and corrected, and
adjust an auto exposure and an auto white balance of one or more of a plurality of cameras based on the output stitched image,
wherein the ISP operation includes edge enhancement and at least one of gamma correction, color correction, auto exposure correction or auto white balance adjustment.

### Claim 11 (depends on 10)

The image processing apparatus of claim 10, wherein the Bayer pattern has an uncompressed version and is converted in a format required by the IVN.

### Claim 12 (depends on 10)

The image processing apparatus of claim 10, wherein each of the plurality of camera devices is connected to the image processor through an independent dedicated line.

### Claim 13 (depends on 10)

The image processing apparatus of claim 10, wherein the plurality of camera device comprises:
a lens assembly configured to collect optical signals;
an image sensor configured to convert the optical signals collected through the lens assembly into an electrical signal to output the Bayer pattern; and
a transmission unit configured to transmit the Bayer pattern.

### Claim 14 (depends on 13)

The image processing apparatus of claim 13, wherein the transmission unit comprises a serializer configured to convert the Bayer pattern into serial data for transmission using a low-voltage differential signaling method.

### Claim 15 (depends on 14)

The image processing apparatus of claim 14, wherein the transmission unit further comprises an encoder configured to convert the serial data in an Ethernet frame format.