# US 12,217,402 B2 · Deep learning based image enhancement for additive manufacturing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,402 B2
- Document ID: US12217402B2
- Status: Granted
- Kind: B2
- Application number: 17/535,766
- Filed: 2021-11-26
- Published: 2022-06-02
- Granted: 2025-02-04
- Earliest claimed: 2020-12-01
- Assignees: BWXT Advanced Technologies LLC
- Applicants: BWXT Advanced Technologies LLC
- Inventors: Simon Mason; Ryan Scott Kitchen; Travis McFalls
- Web page: https://patentgrep.com/patent/US12217402B2

## CPC

- G06T 3/4046: using neural networks
- G06T 5/73: Deblurring; Sharpening
- B33Y 50/00: Data acquisition or data processing for additive manufacturing
- G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/0004: {Industrial image inspection}
- G06T 2207/10048: Infrared image
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30108: Industrial image inspection
- Y02P 10/25: Process efficiency
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B33Y: ADDITIVE MANUFACTURING, i.e. MANUFACTURING OF THREE-DIMENSIONAL [3D] OBJECTS BY ADDITIVE DEPOSITION, ADDITIVE AGGLOMERATION OR ADDITIVE LAYERING, e.g. BY 3D PRINTING, STEREOLITHOGRAPHY OR SELECTIVE LASER SINTERING
- Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS

## Abstract

A method is provided for enhancing image resolution for sequences of 2-D images of additively manufactured products. For each of a plurality of additive manufacturing processes, the process obtains a respective plurality of sequenced low-resolution 2-D images of a respective product during the respective additive manufacturing process and obtains a respective high-resolution 3-D image of the respective product after completion of the respective additive manufacturing process. The process selects tiling maps that subdivide the low-resolution 2-D images and the high-resolution 3-D images into low-resolution tiles and high-resolution tiles, respectively. The process also builds an image enhancement generator iteratively in a generative adversarial network using training inputs that includes ordered pairs of low-resolution and high-resolution tiles. The process stores the image enhancement generator for subsequent use to enhance sequences of low-resolution 2-D images captured for products during additive manufacturing.

## Claims

### Claim 1

A method for enhancing image resolution for sequences of 2-D images of additively manufactured products, the method comprising:
for each of a plurality of additive manufacturing processes:
obtaining a respective plurality of sequenced low-resolution 2-D images of a respective product during the respective additive manufacturing process;
obtaining a respective high-resolution 3-D image of the respective product after completion of the respective additive manufacturing process, wherein the high-resolution 3-D image comprises a plurality of high-resolution 2-D images corresponding to the low resolution 2-D images;
selecting one or more tiling maps that subdivide each of the low-resolution 2-D images into a plurality of LR tiles and subdivide each of the corresponding high-resolution 2-D images into a plurality of corresponding HR tiles;
building an image enhancement generator iteratively in a generative adversarial network using training input comprising ordered pairs of corresponding LR tiles and HR tiles; and
storing the image enhancement generator for subsequent use to enhance sequences of low-resolution 2-D images captured for products during additive manufacturing,
wherein the method further comprises cropping and aligning the low-resolution 2-D images with the high-resolution 2-D images prior to subdividing into tiles.

### Claim 2 (depends on 1)

The method of claim 1, wherein each of the plurality of sequenced low-resolution 2-D images is a near-infrared (NIR) image of the respective product captured in a temporal sequence during the respective additive manufacturing process.

### Claim 3 (depends on 1)

The method of claim 1, wherein each of the high-resolution 3-D images is generated based on performing a micro-CT scan of the respective product after the respective additive manufacturing process is complete.

### Claim 4 (depends on 1)

The method of claim 1, wherein the generative adversarial network includes a first neural network comprising the image enhancement generator and a second neural network comprising a discriminator.

### Claim 5 (depends on 4)

The method of claim 4, wherein building the image enhancement generator iteratively comprises:
training the image enhancement generator to produce candidate high-resolution 2-D images based on low-resolution 2-D images; and
training the discriminator to distinguish between the candidate high-resolution 2-D images and 2-D slices of the obtained high-resolution 3-D images.

### Claim 6 (depends on 5)

The method of claim 5, wherein building the image enhancement generator ceases when output of the image enhancement generator is classified by the discriminator as a real high-resolution 3-D image for 50 percent of the candidate high-resolution 2-D images during multiple successive training iterations.

### Claim 7 (depends on 1)

The method of claim 1, further comprising augmenting the LR tiles and HR tiles in the training input by performing a warp transformation on some of the 2-D images.

### Claim 8 (depends on 1)

The method of claim 1, wherein the one or more tiling maps comprise a plurality of tiling maps, each subdividing according to a different pattern.

### Claim 9 (depends on 1)

An electronic device for enhancing image resolution for sequences of 2-D images of additively manufactured products, comprising:
one or more processors; and
memory storing one or more programs configured for execution by the one or more processors, the one or more programs including instructions for performing the method of claim 1.

### Claim 10 (depends on 1)

A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of an electronic device, the one or more programs including instructions for performing the method of claim 1.

### Claim 11

A method for enhancing image resolution for sequences of 2-D images of additively manufactured products, the method comprising:
obtaining a plurality of temporally sequenced low-resolution 2-D images of a product during an in-progress additive manufacturing process;
obtaining an image enhancement generator previously trained as part of a generative adversarial network, wherein the image enhancement generator is configured to accept input images of a fixed 2-dimensional size;
selecting one or more tiling maps that subdivide each of the low-resolution 2-D images into a plurality of LR tiles;
for each of the LR tiles, applying the image enhancement generator to generate a high-resolution 2-D artificial image tile of the product;
stitching together the high-resolution 2-D artificial image tiles to form a set of high-resolution 2-D artificial layers corresponding to the low-resolution images; and
stacking together the high-resolution 2-D artificial layers to form a 3-D artificial volume of the product,
wherein the stitching comprises:
for each pixel included in two or more overlapping regions of the tiling map, generating a respective output image for the respective pixel by computing a respective weighted sum of values in the corresponding high-resolution 2-D artificial image tiles, and
wherein computing the respective weighted sum comprises:
associating, for each tile's contribution to the respective weighted sum, a respective weight that is linearly proportional to a distance from a center of the respective tile.

### Claim 12 (depends on 11)

The method of claim 11, wherein the generative adversarial network includes a first neural network comprising the image enhancement generator and a second neural network comprising a discriminator.

### Claim 13 (depends on 12)

The method of claim 12, wherein, during training:
the image enhancement generator is trained to generate candidate high-resolution 2-D images based on low-resolution 2-D images; and
the discriminator is trained to discriminate between the candidate high-resolution 2-D images and slices of real high-resolution 3-D images captured after additive manufacturing processes are complete.

### Claim 14 (depends on 11)

The method of claim 11, wherein obtaining the plurality of temporally sequenced low-resolution 2-D images comprises capturing a respective low-resolution 2-D image for each layer of the product during the in-progress additive manufacturing process.

### Claim 15 (depends on 11)

The method of claim 11, further comprising resizing the plurality of sequenced low-resolution 2-D images.

### Claim 16 (depends on 11)

The method of claim 11, wherein each tiling map subdivides each of the low-resolution 2-D images into non-overlapping tiles.

### Claim 17 (depends on 11)

The method of claim 11, wherein the one or more tiling maps comprise a plurality of tiling maps, each subdividing the low-resolution 2-D images according to a different pattern.

### Claim 18 (depends on 11)

The method of claim 11, further comprising:
converting the 3-D artificial volume of the product into a native CT-scan format.

### Claim 19 (depends on 11)

The method of claim 11, further comprising:
interpolating, using a trained neural network, between print layers of the in-progress additive manufacturing process.

### Claim 20 (depends on 11)

The method of claim 11, further comprising using the 3-D artificial volume to identify post-build effects or defects in the product.

### Claim 21 (depends on 11)

An electronic device for enhancing image resolution for sequences of 2-D images of additively manufactured products, comprising:
one or more processors; and
memory storing one or more programs configured for execution by the one or more processors, the one or more programs including instructions for performing the method of claim 11.

### Claim 22 (depends on 11)

A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of an electronic device, the one or more programs including instructions for performing the method of claim 11.