# US 10,872,406 B2 · Hot spot defect detecting method and hot spot defect detecting system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,406 B2 - Document ID: US10872406B2 - Status: Granted - Kind: B2 - Application number: 16/116,899 - Filed: 2018-08-29 - Published: 2019-10-17 - Granted: 2020-12-22 - Earliest claimed: 2018-04-13 - Assignees: Taiwan Semiconductor Manufacturing Company, Ltd. - Applicants: Taiwan Semiconductor Manufacturing Co., Ltd. - Inventors: Chien-Huei Chen; Pei-Chao Su; Xiaomeng Chen; Chan-Ming Chang; Shih-Yung Chen; Hung-Yi Chung; Kuang-Shing Chen; Li-Jou Lee; Yung-Cheng Lin; Wei-Chen Wu; Shih-Chang Wang; Chien-An Lin - Web page: https://patentgrep.com/patent/US10872406B2 ## CPC - G06T 7/001: {using an image reference approach} - G01N 21/8851: {Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges (G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002)} - G01N 21/9505: {Wafer internal defects, e.g. microcracks} - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06T 7/0006: {using a design-rule based approach} - G01N 2021/8854: {Grading and classifying of flaws} - G01N 2021/8887: {based on image processing techniques} - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30141: Printed circuit board [PCB] - G06T 2207/30148: Semiconductor; IC; Wafer - G06T 2207/30168: Image quality inspection - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A hot spot defect detecting method and a hot spot defect detecting system are provided. In the method, hot spots are extracted from a design of a semiconductor product to define a hot spot map comprising hot spot groups, wherein local patterns in a same context of the design yielding a same image content are defined as a same hot spot group. During runtime, defect images obtained by an inspection tool performing hot scans on a wafer manufactured with the design are acquired and the hot spot map is aligned to each defect image to locate the hot spot groups. The hot spot defects in each defect image are detected by dynamically mapping the hot spot groups located in each defect image to a plurality of threshold regions and respectively performing automatic thresholding on pixel values of the hot spots of each hot spot group in the corresponding threshold region. ## Claims ### Claim 1 A method for detecting hot spot defects, adapted to an electronic apparatus, the method comprising: extracting a plurality of local patterns from a design of a semiconductor product as a plurality of hot spots to define a hot spot map comprising a plurality of hot spot groups, wherein a plurality of local patterns in a same context of the design yielding a same image content are defined as the hot spots of a same hot spot group; acquiring a plurality of defect images obtained by an inspection tool performing hot scans on a wafer manufactured with the design during runtime and aligning the hot spot map to each of the defect images to locate the hot spot groups in each defect image; and for each of the defect images, detecting the hot spot defects in the respective defect image by dynamically mapping each of the hot spot groups located in the respective defect image to one of a plurality of threshold regions and performing, for each of the threshold regions, automatic thresholding on pixel values of the hot spots corresponding to the hot spot group dynamically mapped to the respective threshold region of the plurality of threshold regions. ### Claim 2 (depends on 1) The method according to claim 1, further comprising: acquiring a plurality of defect images obtained by the inspection tool performing hot scans, under a plurality of optical modes, on the wafer manufactured with the design; and selecting an optimal optical mode for detecting the hot spot defects from among the plurality of optical modes based on a separability of defects to nuisances in the defect images obtained under each optical mode, wherein the defect images obtained by the inspection tool performing hot scans on the wafer manufactured with the design under the selected optimal optical mode are acquired for detecting the hot spot defects. ### Claim 3 (depends on 2) The method according to claim 2, wherein the step of selecting the optimal optical mode comprises: aligning the hot spot map to the defect image of each optical mode to locate the hot spot defects; computing a signal level and a noise level of each of the hot spot defects in the defect image of each optical mode; computing the separability for each optical mode by summarizing ratios of the signal level to the noise level of the hot spot defects; and ranking the optical modes according to the computed separabilities to select the optimal optical mode. ### Claim 4 (depends on 2) The method according to claim 2, further comprising: training a machine learning model for classifying the defects from the nuisances with the defect images of the selected optical mode to evaluate optimal filters for detecting the hot spot defects. ### Claim 5 (depends on 4) The method according to claim 4, wherein the step of training the machine learning model for classifying the defects from the nuisances with the defect images of the selected optical mode to evaluate the optimal filters for detecting the hot spot defects comprises: creating the machine learning model with convolution filters for processing the defect images; feeding a plurality of defect images of the selected optical mode to the machine learning model to train the machine learning model for classifying the defects from the nuisances in the defect images; and adopting the convolution filters of the trained machine learning model as the optimal filters for detecting the hot spot defects, wherein the optimal filters are applied to the pixel values of the hot spots of each hot spot group in the corresponding threshold region before performing the automatic thresholding. ### Claim 6 (depends on 1) The method according to claim 1, wherein the performing, for each of the threshold regions, automatic thresholding on pixel values of the hot spots corresponding to the hot spot group dynamically mapped to the respective threshold region of the plurality of threshold regions comprises: determining at least a detection threshold for each of the threshold region based on noise levels of the pixels of the hot spots of each hot spot group in the respective threshold region of the plurality of threshold regions; and determining the pixels having the pixel values deviating from the detection threshold as the hot spot defect. ### Claim 7 (depends on 1) The method according to claim 1, wherein defects detected in each of the threshold region are mapped to the corresponding hot spot groups as the hot spot defects in the defect image. ### Claim 8 A system for detecting hot spot defects, comprising: a connecting device, configured to connect an inspection tool; a storage medium, configured to store images acquired by the connecting device; a processor, coupled to the connecting device and the storage medium, and configured to execute instructions to perform steps of: extracting a plurality of local patterns from a design of a semiconductor product as a plurality of hot spots to define a hot spot map comprising a plurality of hot spot groups, wherein a plurality of local patterns in a same context of the design yielding a same image content are defined as the hot spots of a same hot spot group; acquiring a plurality of defect images obtained by an inspection tool performing hot scans on a wafer manufactured with the design during runtime and aligning the hot spot map to each of the defect images to locate the hot spot groups in each defect image; and for each of the defect images, detecting the hot spot defects in the respective defect image by dynamically mapping each of the hot spot groups located in the respective defect image to one of a plurality of threshold regions and performing, for each of the threshold regions, automatic thresholding on pixel values of the hot spots corresponding to the hot spot group dynamically mapped to the respective threshold region of the plurality of threshold regions. ### Claim 9 (depends on 8) The system according to claim 8, wherein the processor is further configured to execute instructions to perform steps of: acquiring a plurality of defect images obtained by the inspection tool performing hot scans, under a plurality of optical modes, on the wafer manufactured with the design; and selecting an optimal optical mode for detecting the hot spot defects from among the plurality of optical modes based on a separability of defects to nuisances in the defect images obtained under each optical mode, wherein the defect images obtained by the inspection tool performing hot scans on the wafer manufactured with the design under the selected optimal optical mode are acquired for detecting the hot spot defects. ### Claim 10 (depends on 9) The system according to in claim 9, wherein the processor is further configured to execute instructions to perform steps of: aligning the hot spot map to the defect image of each optical mode to locate the hot spot defects; computing a signal level and a noise level of each of the hot spot defects in the defect image of each optical mode; computing the separability for each optical mode by summing ratios of the signal level to the noise level of the hot spot defects; and ranking the optical modes according to the computed separabilities to select the optimal optical mode. ### Claim 11 (depends on 9) The system according to claim 9, wherein the processor is further configured to execute instructions to perform a step of: training a machine learning model for classifying the defects from the nuisances with the defect images of the selected optical mode to evaluate optimal filters for detecting the hot spot defects. ### Claim 12 (depends on 11) The system according to claim 11, wherein the processor is further configured to execute instructions to perform steps of: creating the machine learning model with convolution filters for processing the defect images; feeding a plurality of defect images of the selected optical mode to the machine learning mode to train the machine learning model for classifying the defects from the nuisances in the defect images; and adopting the convolution filters of the trained machine learning model as optimal filters for detecting the hot spot defects, wherein the optimal filters are applied to the pixel values of the hot spots of each hot spot group in the corresponding threshold region before performing the automatic thresholding. ### Claim 13 (depends on 8) The system according to claim 8, wherein the processor is further configured to execute instructions to perform steps of: determining at least a detection threshold for each of the threshold region based on noise levels of the pixels of the hot spots of each hot spot group in the respective threshold region of the plurality of threshold regions; and determining the pixels having the pixel values out of the detection threshold as the hot spot defect. ### Claim 14 (depends on 8) The system according to claim 8, wherein the processor maps defects detected in each of the threshold region to the corresponding hot spot groups as the hot spot defects in the defect image. ### Claim 15 A method for detecting hot spot defects, adapted to an electronic apparatus, the method comprising: acquiring a plurality of defect images obtained by an inspection tool performing hot scans, under a plurality of optical modes, on a wafer manufactured with a design of a semiconductor product and selecting an optimal optical mode for detecting the hot spot defects from among the plurality of optical modes based on a separability of defects to nuisances in the defect images obtained under each optical mode; training a machine learning model for classifying the defects from the nuisances with the defect images obtained under the selected optimal optical mode to evaluate optimal filters for detecting the hot spot defects for the optimal optical mode; and acquiring a plurality of defect images obtained by the inspection tool performing hot scans, under the optimal optical mode, on the wafer in runtime and aligning a hot spot map comprising a plurality of hot spot groups extracted from the design to each of the defect images to locate the hot spot groups in each defect image, wherein a plurality of local patterns are extracted from the design as a plurality of hot spots, and a plurality of local patterns in a same context of the design yielding a same image content are defined as the hot spots of a same hot spot group; and for each of the defect images, detecting the hot spot defects in the respective defect image by dynamically mapping each of the hot spot groups located in the respective defect image to one of a plurality of threshold regions, applying the optimal filters to the pixel values of the hot spots of each hot spot group in the respective corresponding threshold region, and performing, for each of the threshold regions, automatic thresholding on pixel values of the hot spots corresponding to the hot spot group dynamically mapped to the respective threshold region of the plurality of threshold regions. ### Claim 16 (depends on 15) The method according to claim 15, wherein the step of selecting the optimal optical mode comprises: aligning the hot spot map to the defect image of each optical mode to locate the hot spot defects; computing a signal level and a noise level of each of the hot spot defects in the defect image of each optical mode; computing the separability for each optical mode by summing ratios of the signal level to the noise level of the hot spot defects; and ranking the optical modes according to the computed separabilities to select the optimal optical mode. ### Claim 17 (depends on 15) The method according to claim 15, wherein the step of training the machine learning model for classifying the defects from the nuisances with the defect images obtained under the selected optical mode to evaluate the optimal filters for detecting the hot spot defects for the optimal optical mode comprises: creating the machine learning model with convolution filters for processing the defect images; feeding a plurality of defect images of the selected optical mode to the machine learning mode to train the machine learning model for classifying the defects from the nuisances in the defect images; and adopting the convolution filters of the trained machine learning model as optimal filters for detecting the hot spot defects, wherein the optimal filters are applied to the pixel values of the hot spots of each hot spot group in the corresponding threshold region before performing the automatic thresholding. ### Claim 18 (depends on 15) The method according to claim 15, wherein the step of performing, for each of the threshold regions, automatic thresholding on pixel values of the hot spots corresponding to the hot spot group dynamically mapped to the respective threshold region of the plurality of threshold regions comprises: determining at least a detection threshold for each of the threshold region based on noise levels of the pixels of the hot spots in the hot spot group dynamically mapped to the respective threshold region of the plurality of threshold regions; and determining the pixels having the pixel values out of the detection threshold as the hot spot defect. ### Claim 19 (depends on 15) The method according to claim 15, wherein defects detected in each of the threshold region are mapped to the corresponding hot spot groups as the hot spot defects in the defect image. ### Claim 20 (depends on 15) The method according to claim 15, wherein the machine learning model comprises a convolution neural network (CNN) model.