# US 2015/0185731 A1 · Work-in-process inspection system using motion detection, and method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0185731 A1
- Document ID: US20150185731A1
- Status: Published application
- Kind: A1
- Application number: 14/559,547
- Filed: 2014-12-03
- Published: 2015-07-02
- Earliest claimed: 2013-12-26
- Assignees: Hyundai Motor Company
- Applicants: Hyundai Motor Company
- Inventors: Se-Hoon Ham; Dong-Ho Kwak; Suyoung Kim; Kwang Joon Lee; Yong Tae Kim
- Web page: https://patentgrep.com/patent/US20150185731A1

## CPC

- G05B 19/41875: {characterised by quality surveillance of production}
- G05B 2219/31442: Detect if operation on object has been executed correctly in each station
- Y02P 90/02: Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS

## Abstract

A work-in-process inspection system is provided and includes a moving image imaging unit that captures a working motion of a worker and an imaging device that captures an image of a vehicle part that is operated on by a worker. A controller determines a difference between the working motion of the worker captured by the moving image imaging device and a predetermined reference motion, and determines a difference between the image captured by the imaging device and a predetermined reference image. Accordingly, whether a predetermined working process has been performed is determined by comparing an image captured through a 3D scanner to a reference motion.

## Claims

### Claim 1

A work-in-process inspection system comprising:
a moving image imaging device configured to capture a working motion of a worker;
an imaging device configured to capture an image of a part that is operated on; and
a controller configured to determine a difference between the working motion of the worker captured by the moving image imaging device and a predetermined reference motion, and determine a difference between the image captured by the imaging device and a predetermined reference image.

### Claim 2 (depends on 1)

The work-in-process inspection system of claim 1, wherein the controller is further configured to extract a joint motion from the working motion by the moving image imaging device;
determine a difference between the extracted joint motion and a predetermined reference motion; and
determine a difference between the image captured by the imaging device and a predetermined reference image.

### Claim 3 (depends on 2)

The work-in-process inspection system of claim 2, wherein the controller is further configured to generate an alarm when a difference between the extracted joint motion and the predetermined reference motion exceeds a predetermined first level, or when a difference between the image captured by the imaging device and the reference image exceeds a predetermined second level.

### Claim 4 (depends on 1)

The work-in-process inspection system of claim 1, further comprising:
an infrared capture device configured to capture an infrared body image in an infrared wavelength band,
wherein the controller is configured to extract a working region of the worker from the image captured by the infrared capture device, and extract a joint motion of the worker in the extracted working region.

### Claim 5 (depends on 1)

The work-in-process inspection system of claim 1,
wherein the controller is configured to determine whether a reference working process is performed from a retention time that the working motion of the worker captured by the moving image imaging device remains in a predetermined virtual working region.

### Claim 6 (depends on 5)

The work-in-process inspection system of claim 5, wherein the controller is configured to determine the retention time that the extracted joint motion remains in the predetermined virtual working region.

### Claim 7 (depends on 6)

The work-in-process inspection system of claim 6, wherein the retention time is measured from an entering time that the joint motion extracted from the motion extractor enters the virtual working region until an exit time that the joint motion exits the virtual working region.

### Claim 8 (depends on 6)

The work-in-process inspection system of claim 6, wherein the controller is configured to determine that the worker performs the reference working process when the retention time is within a range of a predetermined reference time.

### Claim 9 (depends on 6)

The work-in-process inspection system of claim 6, wherein the controller is configured to generate an alarm when the retention time is different from the predetermined reference time by a threshold amount.

### Claim 10 (depends on 2)

The work-in-process inspection system of claim 2, wherein the controller is configured to determine whether a reference working process is performed from a retention time that the working motion of the worker captured by the moving image imaging device remains in a predetermined virtual working region.

### Claim 11 (depends on 10)

The work-in-process inspection system of claim 10, wherein the controller is configured to determine the retention time that the extracted joint motion remains in the predetermined virtual working region.

### Claim 12 (depends on 11)

The work-in-process inspection system of claim 11, wherein the retention time is measured from an entering time that the joint motion extracted from the motion extractor enters the virtual working region until an escaping time that the joint motion exits the virtual working region.

### Claim 13 (depends on 11)

The work-in-process inspection system of claim 11, wherein the controller is configured to determine that the worker performs the reference working process when the retention time is within a range of a predetermined reference time.

### Claim 14 (depends on 11)

The work-in-process inspection system of claim 11, wherein the controller is configured to generate an alarm when the retention time is different from the predetermined reference time by a threshold amount.

### Claim 15

A work-in-process inspection method, comprising:
capturing, by a moving image imaging device, an image of a working motion of a worker;
extracting, by a controller, a joint motion of the worker from the working motion of the worker;
comparing, by the controller, the extracted joint motion to a predetermined reference motion;
capturing an image, by an imaging device, a part that is operated on; and
comparing, by the controller, the image to a reference image.

### Claim 16 (depends on 15)

The work-in-process inspection method of claim 15, further comprising:
generating, by the controller, an alarm in response to a difference between the joint motion extracted by the motion extractor and a predetermined reference motion is exceeds a predetermined first level.

### Claim 17 (depends on 15)

The work-in-process inspection method of claim 15, further comprising:
generating, by the controller, an alarm when the difference between the image captured by the imaging device and the predetermined reference image exceeds a predetermined second level.

### Claim 18 (depends on 15)

The work-in-process inspection method of claim 15, further comprising:
capturing an infrared image, by an infrared imaging device, a body image in an infrared wavelength band;
extracting, by the controller, a working region of a worker from the infrared image, and
extracting a joint motion, by a motion extractor, of the worker from the working region.

### Claim 19 (depends on 15)

The work-in-process inspection method of claim 15, further comprising:
measuring, by the controller, a retention time that the extracted joint motion remains in a predetermined virtual working region; and
determining, by the controller, whether a reference working process is performed by comparing the retention time with a predetermined reference time.

### Claim 20 (depends on 19)

The work-in-process inspection method of claim 19, wherein the retention time is measured from an entering time that the joint motion extracted from the motion extractor enters the virtual working region until an exit time that the joint motion exits the virtual working region.

### Claim 21 (depends on 19)

The work-in-process inspection method of claim 19, further comprising:
determining, by the controller, that the reference working process is performed when the retention time is within a range of the predetermined reference time.

### Claim 22 (depends on 19)

The work-in-process inspection method of claim 19, further comprising
generating alarm when the retention time is different from the predetermined time.
