Systems and methods for generating voxel-based three-dimensional objects in a virtual space based on natural language processing (NLP) of a user-provided description

A system configured to use natural language processing (NLP) to automatically generate three-dimensional objects in a virtual space, the system comprising:

• electronic storage configured to electronically store information, wherein the stored information includes a set of three-dimensional objects, wherein the set includes an annotated object that includes a textual description of the annotated object and an unannotated object that lacks a description;
• one or more hardware processors configured by machine-readable instructions to:
• obtain the set of three-dimensional objects;
• convert one or more three-dimensional objects in the set into one or more objects using a three-dimensional voxelized format that natively supports voxels, responsive to the one or more three-dimensional objects in the set using three-dimensional formats that fail to support voxels, wherein the voxelized three-dimensional objects include a first voxelized object based on the annotated object;
• encode, using an autoencoder that includes an encoder and a decoder, the voxelized three-dimensional objects into pairs of multi-dimensional vectors, wherein individual pairs include a vector of means and a vector of standard deviations;
• sample from the individual pairs of multi-dimensional vectors such that a first sampled multi-dimensional vector having a particular dimensionality is created based on an encoding of the first voxelized object;
• decode, using the decoder, the first sampled multi-dimensional vector into a first decoded voxelized three-dimensional object;
• determine loss information for the first decoded voxelized three-dimensional object by comparison to the first voxelized object;
• use the determined loss information to train the autoencoder;
• fine-tune a pretrained text-based system to generate multi-dimensional vectors that are subsequently decoded into decoded voxelized three-dimensional objects, having the particular dimensionality, from textual descriptions included in annotated objects in the set of three-dimensional objects, wherein one or more prompts for the pretrained text-based system are based on a combination of the textual description of the annotated object and the first sampled multi-dimensional vector;
• receive, from a user, a user input, wherein the user input includes user-provided text describing a three-dimensional object;
• generate, using the fine-tuned pretrained text-based system, a multi-dimensional vector from the user-provided text in the user input;
• and
• decode, using the decoder, the generated multi-dimensional vector into a voxelized three-dimensional object that is based on the user-provided text.