# US 10,769,761 B2 · Generating high resolution images from low resolution images for semiconductor applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,761 B2 - Document ID: US10769761B2 - Status: Granted - Kind: B2 - Application number: 16/019,422 - Filed: 2018-06-26 - Published: 2019-01-03 - Granted: 2020-09-08 - Earliest claimed: 2017-06-30 - Assignees: KLA-Tencor Corporation - Applicants: KLA-Tencor Corporation - Inventors: Saurabh Sharma; Amitoz Singh Dandiana; Mohan Mahadevan; Chao Fang; Amir Azordegan; Brian Duffy - Web page: https://patentgrep.com/patent/US10769761B2 ## CPC - G06T 3/4046: using neural networks - G06T 5/90: Dynamic range modification of images or parts thereof - G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/20208: High dynamic range [HDR] image processing - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Methods and systems for generating a high resolution image for a specimen from a low resolution image of the specimen are provided. One system includes one or more computer subsystems configured for acquiring a low resolution image of a specimen. The system also includes one or more components executed by the one or more computer subsystems. The one or more components include a deep convolutional neural network that includes one or more first layers configured for generating a representation of the low resolution image. The deep convolutional neural network also includes one or more second layers configured for generating a high resolution image of the specimen from the representation of the low resolution image. The second layer(s) include a final layer configured to output the high resolution image and configured as a sub-pixel convolutional layer. ## Claims ### Claim 1 A system configured to generate a high resolution image for a specimen from a low resolution image of the specimen, comprising: one or more computer subsystems configured for acquiring a low resolution image of a specimen; and one or more components executed by the one or more computer subsystems, wherein the one or more components comprise: a deep convolutional neural network, wherein the deep convolutional neural network comprises: one or more first layers configured for generating a representation of the low resolution image; and one or more second layers configured for generating a high resolution image for the specimen from the representation of the low resolution image, wherein the one or more second layers comprise a final layer configured to output the high resolution image, and wherein the final layer is further configured as a sub-pixel convolution layer. ### Claim 2 (depends on 1) The system of claim 1, wherein the deep convolutional neural network is configured such that the high resolution image generated by the one or more second layers has less noise than the low resolution image. ### Claim 3 (depends on 1) The system of claim 1, wherein the deep convolutional neural network is configured such that the high resolution image generated by the one or more second layers retains structural and spatial features of the low resolution image. ### Claim 4 (depends on 1) The system of claim 1, wherein the one or more components further comprise a context aware loss module configured to train the deep convolutional neural network, wherein during training of the deep convolutional neural network, the one or more computer subsystems input the high resolution image generated by the one or more second layers and a corresponding, known high resolution image for the specimen into the context aware loss module and the context aware loss module determines context aware loss in the high resolution image generated by the one or more second layers compared to the corresponding, known high resolution image. ### Claim 5 (depends on 4) The system of claim 4, wherein the context aware loss comprises content loss, style loss, and total variation regularization. ### Claim 6 (depends on 5) The system of claim 5, wherein the content loss comprises loss in low level features of the corresponding, known high resolution image. ### Claim 7 (depends on 5) The system of claim 5, wherein the style loss comprises loss in one or more abstract entities that qualitatively define the corresponding, known high resolution image. ### Claim 8 (depends on 4) The system of claim 4, wherein the context aware loss module comprises a pre-trained VGG network. ### Claim 9 (depends on 4) The system of claim 4, wherein the one or more components further comprise a tuning module configured to determine one or more parameters of the deep convolutional neural network based on the context aware loss. ### Claim 10 (depends on 1) The system of claim 1, wherein the one or more computer subsystems are further configured to perform one or more metrology measurements for the specimen based on the high resolution image generated by the one or more second layers. ### Claim 11 (depends on 1) The system of claim 1, wherein the deep convolutional neural network functions independently of an imaging system that generated the low resolution image. ### Claim 12 (depends on 1) The system of claim 1, wherein the low resolution image is generated by one imaging system having a first imaging platform, wherein the one or more computer subsystems are further configured for acquiring another low resolution image generated for another specimen by another imaging system having a second imaging platform that is different than the first imaging platform, wherein the one or more first layers are configured for generating a representation of the another low resolution image, and wherein the one or more second layers are further configured for generating a high resolution image for the another specimen from the representation of the another low resolution image. ### Claim 13 (depends on 12) The system of claim 12, wherein the first imaging platform is an electron beam imaging platform, and wherein the second imaging platform is an optical imaging platform. ### Claim 14 (depends on 12) The system of claim 12, wherein the first and second imaging platforms are different optical imaging platforms. ### Claim 15 (depends on 12) The system of claim 12, wherein the first and second imaging platforms are different electron beam imaging platforms. ### Claim 16 (depends on 1) The system of claim 1, wherein the low resolution image is generated by an electron beam based imaging system. ### Claim 17 (depends on 1) The system of claim 1, wherein the low resolution image is generated by an optical based imaging system. ### Claim 18 (depends on 1) The system of claim 1, wherein the low resolution image is generated by an inspection system. ### Claim 19 (depends on 1) The system of claim 1, wherein the specimen is a wafer. ### Claim 20 (depends on 1) The system of claim 1, wherein the specimen is a reticle. ### Claim 21 (depends on 1) The system of claim 1, wherein the deep convolutional neural network outputs the high resolution image at a throughput that is higher than a throughput for generating the high resolution image with a high resolution imaging system. ### Claim 22 A system configured to generate a high resolution image for a specimen from a low resolution image of the specimen, comprising: an imaging subsystem configured for generating a low resolution image of a specimen; one or more computer subsystems configured for acquiring the low resolution image of the specimen; and one or more components executed by the one or more computer subsystems, wherein the one or more components comprise: a deep convolutional neural network, wherein the deep convolutional neural network comprises: one or more first layers configured for generating a representation of the low resolution image; and one or more second layers configured for generating a high resolution image for the specimen from the representation of the low resolution image, wherein the one or more second layers comprise a final layer configured to output the high resolution image, and wherein the final layer is further configured as a sub-pixel convolution layer. ### Claim 23 A non-transitory computer-readable medium, storing program instructions executable on one or more computer systems for performing a computer-implemented method for generating a high resolution image for a specimen from a low resolution image of the specimen, wherein the computer-implemented method comprises: acquiring a low resolution image of a specimen; generating a representation of the low resolution image by inputting the low resolution image into one or more first layers of a deep convolutional neural network; and generating a high resolution image for the specimen based on the representation, wherein generating the high resolution image is performed by one or more second layers of the deep convolutional neural network, wherein the one or more second layers comprise a final layer configured to output the high resolution image, wherein the final layer is further configured as a sub-pixel convolution layer, wherein said acquiring, said generating the representation, and said generating the high resolution image are performed by the one or more computer systems, wherein one or more components are executed by the one or more computer systems, and wherein the one or more components comprise the deep convolutional neural network. ### Claim 24 A computer-implemented method for generating a high resolution image for a specimen from a low resolution image of the specimen, comprising: acquiring a low resolution image of a specimen; generating a representation of the low resolution image by inputting the low resolution image into one or more first layers of a deep convolutional neural network; and generating a high resolution image for the specimen based on the representation, wherein generating the high resolution image is performed by one or more second layers of the deep convolutional neural network, wherein the one or more second layers comprise a final layer configured to output the high resolution image, wherein the final layer is further configured as a sub-pixel convolution layer, wherein said acquiring, said generating the representation, and said generating the high resolution image are performed by one or more computer systems, wherein one or more components are executed by the one or more computer systems, and wherein the one or more components comprise the deep convolutional neural network.