# US 2026/0253406 A1 · Accurate and multi-scale material selection in digital images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0253406 A1
- Document ID: US20260253406A1
- Status: Published application
- Kind: A1
- Application number: 19/059,847
- Filed: 2025-02-21
- Published: 2026-08-27
- Earliest claimed: 2025-02-21
- Assignees: Adobe Inc.
- Applicants: Adobe Inc.
- Inventors: Julia Guerrero Viu; Valentin Deschaintre; Iliyan Georgiev; Elena Garces Garcia
- Web page: https://patentgrep.com/patent/US20260253406A1

## CPC

- G06V 10/945: {User interactive design; Environments; Toolboxes}
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/52: Scale-space analysis, e.g. wavelet analysis (multi-scale boundary representations G06V10/42)
- G06V 10/75: Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/82: using neural networks
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure is directed toward systems, methods, and non-transitory computer readable media that generate, for display via a graphical user interface, a digital image portraying a variety of materials, wherein one of the materials includes a first sub-material and a second sub-material which are visually distinct. The disclosed systems receive, via an interaction with the graphical user interface, a selection of a location on the digital image corresponding to the first sub-material. Furthermore, the disclosed systems generate, based on the selection and utilizing a material detection neural network, a first material selection that includes an indication of pixels of the digital image displaying the material and a second material selection that includes an indication of pixels of the digital image displaying the first sub-material. In addition, the disclosed systems provide, via the graphical user interface, the first material selection and the second material selection.

## Claims

### Claim 1

A method comprising:
displaying, via a graphical user interface, a digital image portraying a plurality of materials, wherein a material of the plurality of materials comprises a first sub-material and a second sub-material, wherein the first sub-material is visually distinct from the second sub-material;
receiving, via an interaction with the graphical user interface, a selection of a location on the digital image, wherein the first sub-material of the material is at the location;
generating, based on the selection and utilizing a material detection neural network, a first material selection comprising an indication of pixels of the digital image comprising the material;
generating, based on the selection and utilizing the material detection neural network, a second material selection comprising an indication of pixels of the digital image comprising the first sub-material; and
providing, via the graphical user interface, the first material selection and the second material selection.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
generating the first material selection comprises utilizing a first channel of a material selection head of the material detection neural network trained utilizing images with material annotations at a spatially varying bidirectional reflectance distribution level; and
generating the second material selection comprises utilizing a second channel of a material selection head of the material detection neural network trained utilizing images with material annotations at a bidirectional reflectance distribution level.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
the digital image comprises an additional material that shares a semantic class with the material; and
generating the first material selection comprises selecting the pixels of the digital image comprising the material and excluding pixels of the digital image comprising the additional material.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
receiving, via an additional interaction with the graphical user interface, an additional selection of an additional location on the digital image, wherein the second sub-material of the material is at the additional location;
generating, based on the additional selection and utilizing the material detection neural network, a third material selection comprising an indication of pixels of the digital image comprising the second sub-material; and
providing, via the graphical user interface, the third material selection.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
receiving a user input indicating a modification to a material matching threshold;
generating, based on the modification to the material matching threshold, a modification to the first material selection; and
providing, via the graphical user interface, the modification to the first material selection.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
receiving an additional user input indicating a modification to a sub-material matching threshold;
generating, based on the modification to the sub-material matching threshold, a modification to the second material selection; and
providing, via the graphical user interface, the modification to the second material selection.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
generating, from the digital image at a first resolution, a second digital image by scaling the digital image to a second resolution;
generating image partitions by partitioning the second digital image into non-overlapping image patches; and
providing the digital image and the image partitions to the material detection neural network;
wherein generating, utilizing the material detection neural network, the first material selection and the second material selection comprises utilizing the digital image and the image partitions.

### Claim 8 (depends on 7)

The method of claim 7, wherein generating, utilizing the material detection neural network, the first material selection and the second material selection comprises:
generating, utilizing an encoder of the material detection neural network, a first set of multi-scale image features from the digital image;
generating, utilizing the encoder of the material detection neural network, a second set of multi-scale image features from the image partitions; and
aggregating the first set of multi-scale image features and the second set of multi-scale image features.

### Claim 9

A system comprising:
a memory component; and
one or more processing devices coupled to the memory component, the one or more processing devices to perform operations comprising:
generating, utilizing an encoder of a material detection neural network, a first set of multi-scale image features from a digital image at a first resolution;
generating, utilizing the encoder of the material detection neural network, a second set of multi-scale image features from the digital image at a second resolution;
generating, utilizing an aggregation module of the material detection neural network, multi-scale aggregated features by aggregating the first set of multi-scale image features and the second set of multi-scale image features; and
generating, from the multi-scale aggregated features and utilizing a decoder of the material detection neural network, a material selection comprising an indication of pixels of the digital image comprising a material portrayed in the digital image.

### Claim 10 (depends on 9)

The system of claim 9, further comprising training the material detection neural network utilizing training images comprising hierarchical materials, wherein the hierarchical materials comprise a material composed of a first sub-material and a second sub-material, wherein the first sub-material is visually distinct from the second sub-material.

### Claim 11 (depends on 9)

The system of claim 9, further comprising generating, from the multi-scale aggregated features and utilizing the decoder of the material detection neural network, a second material selection comprising an indication of pixels of the digital image comprising a first sub-material, wherein the material comprises the first sub-material and a second sub-material and the first sub-material is visually distinct from the second sub-material.

### Claim 12 (depends on 9)

The system of claim 9, further comprising generating, from the digital image at the second resolution, image partitions by partitioning the digital image at the second resolution into non-overlapping image patches;
wherein generating, utilizing the encoder of the material detection neural network, the second set of multi-scale image features comprises generating the second set of multi-scale image features from the image partitions.

### Claim 13 (depends on 9)

The system of claim 9, wherein generating the multi-scale aggregated features comprises concatenating the first set of multi-scale image features and the second set of multi-scale image features along a feature dimension.

### Claim 14 (depends on 9)

The system of claim 9, further comprising generating, utilizing the encoder of the material detection neural network, a third set of multi-scale image features from the digital image at a third resolution;
wherein generating, utilizing the aggregation module of the material detection neural network, the multi-scale aggregated features comprises aggregating the first set of multi-scale image features, the second set of multi-scale image features, and the third set of multi-scale image features.

### Claim 15 (depends on 9)

The system of claim 9, wherein generating the material selection further comprises:
generating an overlay which visually distinguishes the pixels of the digital image comprising the material portrayed in the digital image from other pixels of the digital image; or
generating a binary mask which portrays the pixels of the digital image comprising the material portrayed in the digital image using a first value and portrays the other pixels of the digital image using a second value.

### Claim 16

A non-transitory computer readable medium storing executable instructions which, when executed by a processing device, cause the processing device to perform operations comprising:
displaying, via a graphical user interface, a digital image portraying a plurality of materials, wherein a material of the plurality of materials comprises a first sub-material and a second sub-material, wherein the first sub-material is visually distinct from the second sub-material;
receiving, via an interaction with the graphical user interface, a selection of a location on the digital image, wherein the first sub-material of the material is at the location;
generating, based on the selection and utilizing a material detection neural network, a first material selection comprising an indication of pixels of the digital image comprising the material;
generating, based on the selection and utilizing the material detection neural network, a second material selection comprising an indication of pixels of the digital image comprising the first sub-material; and
providing, via the graphical user interface, the first material selection and the second material selection.

### Claim 17 (depends on 16)

The non-transitory computer readable medium of claim 16, wherein generating the first material selection comprises utilizing a first detection channel of the material detection neural network trained utilizing images with material annotations at a spatially varying bidirectional reflectance distribution level.

### Claim 18 (depends on 16)

The non-transitory computer readable medium of claim 16, wherein generating the second material selection comprises utilizing a second detection channel of the material detection neural network trained utilizing images with material annotations at a bidirectional reflectance distribution level.

### Claim 19 (depends on 16)

The non-transitory computer readable medium of claim 16, wherein generating, utilizing the material detection neural network, the first material selection and the second material selection comprises generating the first material selection and the second material selection from features extracted from non-overlapping image patches generated by upscaling and partitioning the digital image.

### Claim 20 (depends on 16)

The non-transitory computer readable medium of claim 16, further comprising:
receiving user input indicating a modification to a material matching threshold;
generating, based on the modification to the material matching threshold, a modification to the first material selection and a modification to the second material selection; and
providing, via the graphical user interface, the modification to the first material selection and the modification to the second material selection.
