# US 12,602,760 B2 · Method and apparatus for detecting defect based on phased pass/fail determination

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,760 B2
- Document ID: US12602760B2
- Status: Granted
- Kind: B2
- Application number: 17/891,302
- Filed: 2022-08-19
- Published: 2023-02-23
- Granted: 2026-04-14
- Earliest claimed: 2021-08-20
- Assignees: Hyundai Mobis Co., Ltd.
- Applicants: Hyundai Mobis Co., Ltd.
- Inventors: Tae Hyun Kim; Yu Min Cho; Kyu Ha Song
- Web page: https://patentgrep.com/patent/US12602760B2

## CPC

- G06T 7/0004: {Industrial image inspection}
- G06T 7/11: Region-based segmentation
- G06T 7/13: Edge detection
- G01N 2021/8883: {involving the calculation of gauges, generating models}
- G01N 2021/8887: {based on image processing techniques}
- G06T 2207/10152: Varying illumination
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20132: Image cropping
- G06T 2207/30108: Industrial image inspection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)

## Abstract

A method and an apparatus for detecting a defect based on a phased pass/fail determination are disclosed. According to at least one aspect of the present disclosure, a method comprising: a process of acquiring a product image which is an image of the product; a first determination process of inputting the product image into a first determination model to perform a pass/fail determination for the product; and a second determination process of inputting the product image into a second determination model to perform a pass/fail determination for the product when the product is determined to be undeterminable as a result of the pass/fail determination of the first determination process.

## Claims

### Claim 1

A method of detecting a defect of a product, comprising:
acquiring a data set including a product image of the product;
inputting the data set into a first determination model trained to perform a first pass/fail determination of the product;
extracting, from the first determination model, a first determination result;
determining whether the first determination result indicates that the first determination model failed to perform the first pass/fail determination;
in response to the first determination result indicating that the first determination model failed to perform the first pass/fail determination, inputting the data set to a second determination model trained to perform a second pass/fail determination;
extracting, from the second determination model, a second determination result of the second pass/fail determination; and
determining, based on the second determination result, whether the product has a defect,
wherein inputting the data set into the second determination model comprises inputting the data set to a deep learning-based object detection model or a deep learning-based segmentation model trained to perform the second pass/fail determination based on a detection region in the product image included the acquired data set, and
wherein the second pass/fail determination includes:
determining whether a shape of the detection region is designated by a bounding box or an edge, and
selecting the segmentation model as the second determination model in response to determining that the shape of the detection region is not designated by the bounding box or the edge.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
inputting the product image into the first determination model comprises inputting the product image to a deep learning-based classification model, and
inputting the product image into the second determination model comprises inputting the product image to a deep learning-based object detection model or a deep learning-based segmentation model.

### Claim 3 (depends on 1)

The method of claim 1, wherein inputting the product image into the second determination model comprises inputting the product image into a deep learning-based object detection model trained to:
detect one or more detection regions in the acquired product image;
determine whether a number of the detection regions meet a predetermined number threshold;
calculate a distance between the detection regions in response to the number of the detection regions meeting the predetermined number threshold; and
perform, based on the distance, the second pass/fail determination.

### Claim 4 (depends on 1)

The method of claim 1, further comprising training the first determination model and the second determination model using a data set including an image of each of a plurality of products and a label corresponding to the image.

### Claim 5 (depends on 4)

The method of claim 4, wherein training the first determination model comprises:
generating a first image in which at least one of an illuminance and contrast of the image included in the data set is randomly changed, and
including the generated first image in the data set.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
adjusting the detection region of the product image in response to the second determination result indicating that the second determination model failed to perform the second pass/fail determination of the product; and
inputting the product image having the adjusted detection region to the second determination model to reperform the second pass/fail determination.

### Claim 7 (depends on 1)

The method of claim 1, further comprising, pre-processing the product image to transform a data set representing the product image to be suitable for the first determination model or the second determination model in response to the second determination result indicating that the second determination model failed to perform the second pass/fail determination of the product.

### Claim 8 (depends on 1)

A system for detecting a defect of a product, comprising:
a processor; and
a computer-readable medium in communication with the processor and storing instructions that, when executed by the processor, cause the processor to control the system to perform the method of claim 1.

### Claim 9

A method of detecting a defect of a product, comprising:
acquiring a data set including a product image of the product;
inputting the data set into a first determination model trained to perform a first pass/fail determination of the product;
extracting, from the first determination model, a first determination result;
determining whether the first determination result indicates that the first determination model failed to perform the first pass/fail determination;
in response to the first determination result indicating that the first determination model failed to perform the first pass/fail determination, inputting the data set to a second determination model trained to perform a second pass/fail determination;
extracting, from the second determination model, a second determination result of the second pass/fail determination; and
determining, based on the second determination result, whether the product has a defect,
wherein inputting the data set into the second determination model comprises inputting the data set to a deep learning-based object detection model or a deep learning-based segmentation model trained to perform the second pass/fail determination based on a detection region in the product image included in the acquired data set, and
wherein inputting the data set into the second determination model comprises inputting the data set to the second determination model trained to:
extract the detection region from the product image included in the data set;
extract a detection pattern from the detection region; and
perform, based on the extracted detection pattern, the second pass/fail determination.