# US 2024/0391048 A1 · In-tool audio and visual behavior monitoring for process control

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0391048 A1
- Document ID: US20240391048A1
- Status: Published application
- Kind: A1
- Application number: 18/321,895
- Filed: 2023-05-23
- Published: 2024-11-28
- Earliest claimed: 2023-05-23
- Assignees: Applied Materials, Inc.
- Applicants: Applied Materials, Inc.
- Inventors: Justin H. WONG; Devin PON
- Web page: https://patentgrep.com/patent/US20240391048A1

## CPC

- B24B 37/005: Control means for lapping machines or devices
- B24B 49/003: {involving acoustic means}
- B24B 49/12: involving optical means
- G06T 7/0004: {Industrial image inspection}
- G06T 7/001: {using an image reference approach}
- G06T 7/12: Edge-based segmentation
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30148: Semiconductor; IC; Wafer
- B24B: MACHINES, DEVICES, OR PROCESSES FOR GRINDING OR POLISHING (grinding of gear teeth B23F, of screw-threads B23G1/36; by electro-erosion B23H; abrasive or related blasting B24C; tools for grinding, buffing or sharpening B24D; polishing compositions C09G1/00; abrasives C09K3/14; electrolytic etching or polishing C25F3/00; grinding arrangements for use on assembled railway tracks E01B31/17); DRESSING OR CONDITIONING OF ABRADING SURFACES; FEEDING OF GRINDING, POLISHING, OR LAPPING AGENTS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Embodiments of the disclosure provided herein include an apparatus and method for more efficient and effective in-tool monitoring of chemical mechanical polishing systems. The polishing apparatus includes a controller coupled to the at least one camera. The controller is configured to receive a media data stream from the at least one camera, process the media data stream into processed media data, and detect an anomaly in the polishing apparatus based on the processed media data. A method of monitoring a polishing apparatus includes receiving an image or audio data stream, processing the data stream, receiving a subsequent image or audio data stream, and comparing the second set of processed data to the first set of processed data.

## Claims

### Claim 1

A polishing apparatus, comprising:
an arm rotatably secured to a base of the polishing apparatus;
a conditioning head coupled to the arm and configured to rotate over a platen;
at least one camera positioned within the polishing apparatus; and
a controller coupled to the at least one camera, the controller comprising a memory and a media analysis processor, the controller configured to:
receive a media data stream from the at least one camera;
process the media data stream into processed media data; and
detect an anomaly in the polishing apparatus based on the processed media data.

### Claim 2 (depends on 1)

The polishing apparatus of claim 1, wherein the media analysis processor is a video analysis processor, the media data stream is an image data stream, and the processed media data is processed image data.

### Claim 3 (depends on 2)

The polishing apparatus of claim 2, wherein the video analysis processor is configured to process the image data stream using object contouring.

### Claim 4 (depends on 3)

The polishing apparatus of claim 3, wherein the processed image data includes a first set of processed image data and a second set of processed image data, and wherein the controller is configured to detect the anomaly by comparing the second set of processed image data to the first set of processed image data.

### Claim 5 (depends on 1)

The polishing apparatus of claim 1, wherein the media analysis processor is an audio analysis processor, the media data stream is an audio data stream, and the processed media data is processed audio media data.

### Claim 6 (depends on 5)

The polishing apparatus of claim 5, wherein the audio analysis processor is configured to process the audio data stream using Fast Fourier Transform analysis.

### Claim 7 (depends on 6)

The polishing apparatus of claim 6, wherein the processed audio data includes a first set of processed audio data and a second set of processed audio data, and wherein the controller is configured to detect the anomaly by comparing the second set of processed audio data to the first set of processed audio data.

### Claim 8

A method of monitoring a polishing apparatus, comprising:
receiving a first image data stream from a camera disposed in the polishing apparatus;
processing the first image data stream using a video analysis processor of a controller coupled to the polishing apparatus to produce a first set of processed image data;
receiving a second image data stream from the camera;
processing the second image data stream using the video analysis processor to produce a second set of processed image data;
comparing the second set of processed image data to the first set of processed image data to determine if a predetermined threshold is exceeded; and
upon determining that the predetermined threshold is exceeded, transmitting an alert to a user interface coupled to the controller.

### Claim 9 (depends on 8)

The method of claim 8, further comprising passing the first set of processed image data and the second set of processed image data through a gating mechanism of a long short-term memory neural network.

### Claim 10 (depends on 8)

The method of claim 8, wherein processing the first image data stream includes performing an object contouring process such that reconstructed contours are defined by components of the polishing apparatus.

### Claim 11 (depends on 8)

The method of claim 8, wherein processing the first image data stream includes performing image segmentation using a line-segment based approach.

### Claim 12 (depends on 8)

The method of claim 8, wherein the predetermined threshold corresponds to abnormal environment conditions within the polishing apparatus.

### Claim 13 (depends on 8)

The method of claim 8, wherein the predetermined threshold corresponds to excessive vibration of a component of the polishing apparatus.

### Claim 14 (depends on 8)

The method of claim 8, wherein processing the first image data stream includes using cross-classification clustering to track interrelated objects in the polishing apparatus.

### Claim 15

A method of monitoring a polishing apparatus, comprising:
receiving a first audio data stream from a camera disposed in the polishing apparatus;
processing the first audio data stream using an audio analysis processor of a controller coupled to the polishing apparatus to produce a first set of processed audio data;
receiving a second audio data stream from the camera;
processing the second audio data stream using the audio analysis processor to produce a second set of processed audio data;
comparing the second set of processed audio data to the first set of processed audio data to determine if a predetermined threshold is exceeded; and
upon determining that the predetermined threshold is exceeded, transmitting an alert to a user interface coupled to the controller.

### Claim 16 (depends on 15)

The method of claim 15, further comprising passing the first set of processed audio data and the second set of processed audio data through a gating mechanism of a long short-term memory neural network.

### Claim 17 (depends on 15)

The method of claim 15, wherein processing the first audio data stream includes performing a Fast Fourier Transform analysis.

### Claim 18 (depends on 15)

The method of claim 15, wherein processing the first audio data stream includes performing harmonic percussive source separation, and wherein the first set of processed audio data comprises harmonic spectrogram data, percussive spectrogram data, or both.

### Claim 19 (depends on 15)

The method of claim 15, wherein the first set of audio data corresponds to a state of a consumable within the polishing apparatus.

### Claim 20 (depends on 15)

The method of claim 15, wherein the predetermined threshold corresponds to tool failure within the polishing apparatus.
