# US 2020/0400453 A1 · Image processing apparatus, image processing method, computer program and computer readable recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0400453 A1
- Document ID: US20200400453A1
- Status: Published application
- Kind: A1
- Application number: 17/011,610
- Filed: 2020-09-03
- Published: 2020-12-24
- Earliest claimed: 2016-11-26
- Assignees: Thinkware Corporation
- Applicants: Thinkware Corporation
- Inventors: Ki Wook Lee; Hye Kyung Byun; Tae Kyu Han; Shin Hyoung Kim
- Web page: https://patentgrep.com/patent/US20200400453A1

## CPC

- G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera}
- G01C 21/005: {with correlation of navigation data from several sources, e.g. map or contour matching (G01C21/30 takes precedence)}
- G01C 21/3811: {Point data, e.g. Point of Interest [POI]}
- G01C 21/3841: {Data obtained from two or more sources, e.g. probe vehicles}
- G06F 16/23: Updating
- G06T 1/0007: {Image acquisition}
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/254: involving subtraction of images
- G01S 5/16: using electromagnetic waves other than radio waves
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

Provided is an image processing method. The image processing method includes receiving images acquired from a plurality of vehicles positioned on a road; storing the received images according to acquisition information of the received images; determining a reference image and a target image based on images having the same acquisition information among the stored images; performing an image registration using a plurality of feature points extracted from each of the determined reference image and target image; performing a transparency process for each of the reference image and the target image which are image-registered; extracting static objects from the transparency-processed image; and comparing the extracted static objects with objects on map data which is previously stored and updating the map data when the objects on the map data which is previously stored and the extracted static objects are different from each other.

## Claims

### Claim 1

An image processing method comprising:
acquiring images taken during driving of a moving object;
performing first image processing including image registration processing and transparency processing of the acquired images;
performing second image processing of synthesizing images on which the first image processing has been performed; and
extracting a static object from an image generated according to the second image processing.

### Claim 2 (depends on 1)

The image processing method of claim 1, wherein the method further comprising:
identifying a type of the extracted static object through deep learning or machine learning.

### Claim 3 (depends on 2)

The image processing method of claim 2,
wherein the identifying identifies the type of the static object in order to select static object to be reflected in the map data among the extracted static object.

### Claim 4 (depends on 1)

The image processing method of claim 1,
wherein the acquired images includes the static object and a dynamic object,
wherein the static object comprises at least one of a bridge, a building, a road, a sidewalk, a road construction mark, a speed bump, a crosswalk, a intersection, a traffic light, a median strip, a bus stop and a directional sign, and
wherein the dynamic object comprises at least one of a vehicle and a pedestrian.

### Claim 5 (depends on 1)

The image processing method of claim 1, wherein the image registration processing of the first image processing comprising:
determining a reference image and a target image among the acquired images;
extracting a plurality of feature points from each of the determined reference image and target image; and
matching the extracted plurality of feature points.

### Claim 6 (depends on 5)

The image processing method of claim 5, wherein the image registration processing of the first image processing further comprising:
calculating a homography, which is a transformation matrix between the reference image and the target image, using information of matched pairs between first feature point group of the reference image and the second feature point group of the target image.

### Claim 7 (depends on 1)

The image processing method of claim 1, wherein the transparency processing of the first image processing comprising:
multiplying R, G, and B pixel values of respective pixels included in the images for which the transparency processing is to be performed by a predetermined value smaller than 1, and
wherein the predetermined value is a reciprocal number of N, which is a total number of the images for which the transparency process is to be performed.

### Claim 8 (depends on 7)

The image processing method of claim 7, wherein the extracting of the static object comprising:
calculating standard deviations of a pixel value of the synthesized image, a pixel value of the reference image, and a pixel value of the target image for each of pixels; and
determining pixels of which the calculated standard deviation is a predetermined value or less as pixels for the static object.

### Claim 9 (depends on 1)

The image processing method of claim 1, wherein the transparency processing of the first image processing comprising:
performing the transparency processing by adjusting an ALPHA (A) value corresponding to a transparency level in an RGBA value of a transparency target image.

### Claim 10 (depends on 9)

The image processing method of claim 9, wherein the extracting of the static object comprising:
extracting the static object by comparing an A(ALPHA) value in an RGBA value of the synthesized image with a predetermined transparency value.

### Claim 11

An image processing method comprising:
an image acquiring unit acquiring images taken during driving of a moving object;
an processor performing first image processing including image registration processing and transparency processing of the acquired images, performing second image processing of synthesizing images on which the first image processing has been performed and extracting a static object from an image generated according to the second image processing.

### Claim 12 (depends on 11)

The image processing apparatus of claim 11, wherein the processor identifies a type of the extracted static object through deep learning or machine learning.

### Claim 13 (depends on 12)

The image processing apparatus of claim 12,
wherein the processor identifies the type of the static object in order to select static object to be reflected in the map data among the extracted static object.

### Claim 14 (depends on 11)

The image processing apparatus of claim 11,
wherein the acquired images includes the static object and a dynamic object,
wherein the static object comprises at least one of a bridge, a building, a road, a sidewalk, a road construction mark, a speed bump, a crosswalk, a intersection, a traffic light, a median strip, a bus stop and a directional sign, and
wherein the dynamic object comprises at least one of a vehicle and a pedestrian.

### Claim 15 (depends on 11)

The image processing apparatus of claim 11, wherein in the image registration processing of the first image processing, the processor determines a reference image and a target image among the acquired images, extracts a plurality of feature points from each of the determined reference image and target image and matches the extracted plurality of feature points.

### Claim 16 (depends on 15)

The image processing apparatus of claim 15, wherein in the image registration processing of the first image processing, the processor calculates a homography, which is a transformation matrix between the reference image and the target image, using information of matched pairs between first feature point group of the reference image and the second feature point group of the target image.

### Claim 17 (depends on 11)

The image processing apparatus of claim 11, wherein in the transparency processing of the first image processing, the processor multiplies R, G, and B pixel values of respective pixels included in the images for which the transparency processing is to be performed by a predetermined value smaller than 1, and
wherein the predetermined value is a reciprocal number of N, which is a total number of the images for which the transparency process is to be performed.

### Claim 18 (depends on 17)

The image processing apparatus of claim 17, wherein in the extracting of the static object, the processor calculates standard deviations of a pixel value of the synthesized image, a pixel value of the reference image, and a pixel value of the target image for each of pixels, and determines pixels of which the calculated standard deviation is a predetermined value or less as pixels for the static object.

### Claim 19 (depends on 11)

The image processing apparatus of claim 11, wherein in the transparency processing of the first image processing, the processor performs the transparency processing by adjusting an ALPHA (A) value corresponding to a transparency level in an RGBA value of a transparency target image.

### Claim 20

A non-transitory computer readable recording medium in which a program for executing an image processing method is recorded,
the image processing method comprising:
acquiring images taken during driving of a moving object;
performing first image processing including image registration processing and transparency processing of the acquired images;
performing second image processing of synthesizing images on which the first image processing has been performed; and
extracting a static object from an image generated according to the second image processing.
