US 2023/0141395 A1 · MODIFYING MATERIALS OF THREE-DIMENSIONAL DIGITAL SCENES UTILIZING A VISUAL NEURAL NETWORK

Abstract

Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a visual neural network to replace materials in a three-dimensional scene with visually similar materials from a source dataset. Specifically, the disclosed system utilizes the visual neural network to generate source deep visual features representing source texture maps from materials in a plurality of source materials. Additionally, the disclosed system utilizes the visual neural network to generate deep visual features representing texture maps from materials in a digital scene. The disclosed system then determines source texture maps that are visually similar to the texture maps of the digital scene based on visual similarity metrics that compare the source deep visual features and the deep visual features. Additionally, the disclosed system modifies the digital scene by replacing one or more of the texture maps in the digital scene with the visually similar source texture maps.

Claims20

Dependent claims

A computer-implemented method comprising:

generating, utilizing a visual neural network, one or more deep visual features for one or more texture maps of one or more objects in a digital scene;
determining, by at least one processor, one or more corresponding texture maps from a set of source texture maps extracted from a plurality of source materials based on the one or more deep visual features and source deep visual features representing the set of source texture maps in the plurality of source materials;
and
modifying, by the at least one processor, the digital scene by replacing the one or more texture maps of the one or more objects in the digital scene with the one or more corresponding texture maps.

1 more element

A system comprising:

one or more computer memory devices comprising a digital scene and a plurality of source materials;
and
one or more servers configured to cause the system to:
generate, utilizing a visual neural network, source deep visual features for source texture maps of a plurality of source materials;
generate, utilizing the visual neural network, a plurality of deep visual features for a plurality of texture maps of a plurality of objects in the digital scene;
determine a plurality of corresponding texture maps from the source texture maps by comparing the source deep visual features and the plurality of deep visual features;
and
modify the digital scene by replacing the plurality of texture maps of the plurality of objects in the digital scene with the plurality of corresponding texture maps.

5 more elements

A non-transitory computer readable storage medium comprising instructions that, when executed by at least one processor, cause a computing device to:

generate, utilizing a visual neural network, source deep visual features for a plurality of source texture maps from a plurality of source materials;
generate, utilizing the visual neural network, one or more deep visual features for one or more texture maps of one or more objects in a digital scene;
determine one or more visual similarity metrics by comparing the one or more deep visual features and the plurality of source texture maps;
select one or more corresponding texture maps from the plurality of source texture maps based on the one or more visual similarity metrics;
and
modify the digital scene by replacing the one or more texture maps of the one or more objects in the digital scene with the one or more corresponding texture maps.

3 more elements

Dependencies are inferred from the claim text.

Modifying materials of three-dimensional digital scenes utilizing a visual neural network

Life of this patent

Modifying materials of three-dimensional digital scenes utilizing a visual neural network

Life of this patent

Abstract

Claims20