# US 12,217,498 B2 · Defect inspection system having human-machine interaction function

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,498 B2
- Document ID: US12217498B2
- Status: Granted
- Kind: B2
- Application number: 17/887,871
- Filed: 2022-08-15
- Published: 2023-04-20
- Granted: 2025-02-04
- Earliest claimed: 2021-10-20
- Assignees: Kapito Inc.; Feng-Tso Sun; Yi-Ting Yeh; Feng-Yu Sun
- Applicants: Kapito Inc.; Feng-Tso Sun; Yi-Ting Yeh; Feng-Yu Sun
- Inventors: Feng-Tso Sun; Yi-Ting Yeh; Feng-Yu Sun; Jyun-Tang Huang; Po-Han Chou
- Web page: https://patentgrep.com/patent/US12217498B2

## CPC

- G06V 10/945: {User interactive design; Environments; Toolboxes}
- G01N 21/8851: {Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges (G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002)}
- G01N 21/8983: {for testing textile webs, i.e. woven material}
- G01N 33/367: {Fabric or woven textiles}
- G06T 7/0004: {Industrial image inspection}
- G06T 7/001: {using an image reference approach}
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06V 10/443: {by matching or filtering}
- G01N 2021/8883: {involving the calculation of gauges, generating models}
- G01N 2021/8887: {based on image processing techniques}
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30124: Fabrics; Textile; Paper
- G06T 2207/30164: Workpiece; Machine component
- G06V 2201/06: Recognition of objects for industrial automation
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A defect inspection system is disclosed, and comprises a linear light source, N number of cameras, a display device, a tag reader, and a modular electronic device, in which the linear light source, the cameras and the modular electronic device are used for conducting a defect inspection of an article. On the other hand, the display device, the tag reader and the modular electronic device are adopted for conducting in production of at least one labeled example. Therefore, the modular electronic device is allowed to apply a machine learning process to an image classifier under using a training dataset containing the labeled examples, thereby producing at least one new defect recognition model or updating the existing defect recognition model.

## Claims

### Claim 1

A defect inspection system, comprising: a linear light source, being disposed at a first position for facing a photographing region; N number of cameras, being disposed at a second position for facing the photographing region, wherein N is a positive integer; a display device; a tag reader; and a modular electronic device, being coupled to the linear light source, the N number of cameras, the display device, and the tag reader, and comprising a processor, a memory and a human machine interface (HMI), wherein the memory stores an application program, and the processor being coupled to the memory and the human machine interface; wherein the application program includes instructions, such that in case the application program is executed, the processor being configured for: controlling the linear light source to emit a detection light for irradiating the photographing region during an article being transferred by an article transfer system; controlling the N number of cameras to acquire an article image frame from the article in case of there being a segment of the article in the photographing region; applying at least one image process to the article image frame, so as to generate an article feature image; extracting a plurality of defect features from the article feature image, and then determining whether there are existing any strange defects in the article feature image by matching the plurality of defect features with a plurality of reference defect features; stopping at least one motor of the article transfer system from running in case of there being one strange defect detected from the article feature image, so as to make the segment of the article be positioned on a manual inspection region; generating a plurality of defect tags correlated to the strange defect, and controlling the display device to display the plurality of defect tags; labeling the article feature image so as to produce a labeled example after the tag reader is operated to read a specific defect tag from the plurality of defect tags; and restarting the at least one motor to run after receiving a transfer restarting command transmitted by the human machine interface.

### Claim 2 (depends on 1)

The defect inspection system of claim 1, wherein the tag reader is selected from a group consisting of 2D barcode reader, QR code reader, matrix barcode reader, smart phone, smart glasses, and tablet computer.

### Claim 3 (depends on 1)

The defect inspection system of claim 1, wherein said specific defect tag is selected from a group consisting of 2D barcode tag, QR code tag, matrix barcode tag, tag containing graphic patterns, tag containing text messages, tag containing alphabets, and tag containing numeric figures.

### Claim 4 (depends on 1)

The defect inspection system of claim 1, wherein the application program consists of a plurality of subprograms, and the plurality of subprograms comprising: a first subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to control the linear light source to emit the detection light for irradiating the photographing region, and to control the N number of cameras to acquire the article image frame from the article in case of there being a segment of the article in the photographing region; a second subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to apply the at least one image process to the article image frame for generating the article feature image; a third subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to extract the plurality of defect features from the article feature image, and to determine whether there are existing any strange defects in the article feature image by matching the plurality of defect features with the plurality of reference defect features; a fourth subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to stop the at least one motor from running, or to restart the at least one motor to run; a fifth subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to generate the plurality of defect tags correlated to the strange defect, to control the display device to display the plurality of defect tags, and to produce said labeled example by labeling the article feature image after the tag reader is operated to read said specific defect tag from the plurality of defect tags.

### Claim 5 (depends on 4)

The defect inspection system of claim 4, wherein the processor conducts a plurality of processing steps to produce the plurality of defect tags, and the plurality of processing steps comprises: extracting a plurality of strange defect features from the article feature image; matching the plurality of strange defect features with the plurality of reference defect features; correlating the plurality of strange defect features to multiple types of reference article defect; and generating the plurality of defect tags correlated to the multiple types of reference article defect.

### Claim 6 (depends on 5)

The defect inspection system of claim 5, wherein in case the processor fails to correlate the plurality of strange defect features to at least one type of reference article defect, the processor controlling the display device to show a combo box, such that the plurality of strange defect features succeed in being correlated to at least one type of defined article defect after the combobox is operated manually.

### Claim 7 (depends on 1)

The defect inspection system of claim 1, wherein each said camera has a photographic coverage, the N number of cameras having a total photographic coverage, and the total photographic coverage having a first width greater than a second width of the article.

### Claim 8 (depends on 5)

The defect inspection system of claim 5, wherein the plurality of subprograms further comprising a sixth subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to stitching N number of sub-image frames that are transmitted by the N number of cameras to the article image frame.

### Claim 9 (depends on 1)

The defect inspection system of claim 1, wherein the processor stores the labeled example in the memory, and the reference defect features being also stored in the memory.

### Claim 10 (depends on 8)

The defect inspection system of claim 8, wherein the plurality of subprograms further comprising a seventh subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to apply a machine learning process to an image classifier under using a training dataset containing the labeled example, so as to update a first defect recognition model integrated in the third subprogram or produce at least one second defect recognition model for being integrated in the third subprogram.

### Claim 11 (depends on 1)

The defect inspection system of claim 1, further comprising: at least one rotation speed sensor, being connected to the at least one motor of the article transfer system, and being used for monitoring a rotation speed of said motor.

### Claim 12 (depends on 11)

The defect inspection system of claim 11, wherein after the human machine interface transmits the transfer restarting command to the processor, the processor restarting the at least one motor to run by the rotation speed.

### Claim 13 (depends on 1)

The defect inspection system of claim 1, wherein the N number of cameras acquire the article image frame from the article by a shutter speed, and the shutter speed being positively correlated to a transferring speed of the article.

### Claim 14 (depends on 10)

The defect inspection system of claim 10, wherein the plurality of subprograms further comprising an eighth subprogram, being compiled to be integrated in the application program by one type of programming language, and including instructions for configuring the processor to calculate a displacement of the article based on the rotation speed.