Voice driven 3D static asset creation in computer simulations
AssigneeSony Interactive Entertainment Inc.
InventorsSudha Krishnamurthy, Michael Taylor
Life of this patent
Estimating the term…
Abstract
A 3D scene is generated consisting of one or more objects from a natural language description that may consist of text or voice. Relevant keywords like asset attributes and placement are extracted from the description. Using these keywords, a 2D image is generated using a generative model. Another neural model is used to reconstruct the 3D objects from the 2D. The 3D objects can be assembled to meet the placement specifications. Alternatively, the 3D object is generated by either transforming existing 3D objects or by using a 3D generative model to meet the specifications in the description.
Claims22
A non-transitory, computer readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform operations comprising:
- receiving text;
- executing or requesting execution of an artificial intelligence (AI) engine, wherein, the text is a first input to the AI engine, and wherein the AI engine outputs, based on the text, a two-dimensional (2D) image of a computer simulation asset;
- converting, the 2D image to a three-dimensional (3D) asset;
- receiving a second input indicating a change to the 3D asset;
- modifying an encoding for the 3D asset based on the second input to generate a 3D model;
- and
- presenting, the 3D model within a computer simulation environment that includes another computer simulation asset.
A method comprising:
- receiving a first input, wherein the input comprises at least an image or text, corresponding to a two-dimensional (2D) object;
- generating a 2D synthetic representation of the 2D object;
- converting the 2D synthetic representation to a 3D asset;
- receiving a second input indicating a change to the 3D asset;
- modifying an encoding for the 3D asset based on the second input to generate a 3D model;
- and
- presenting the 3D model within a computer simulation environment that includes another computer simulation asset.
An apparatus comprising:
- at least one processor;
- at least one computer output device controlled by the at least one processor;
- and
- a non-transitory computer readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
- identifying a two-dimensional (2D) image;
- generating a synthetic representation of the 2D image via an artificial intelligence (AI) engine;
- converting the synthetic representation of the 2D image to a reconstructed mesh of a 3D asset, wherein the 3D asset comprises at least asset characteristics or location;
- receiving an input indicating a change to the 3D asset;
- modifying an encoding for the 3D asset based on the input;
- and
- presenting the reconstructed mesh of the 3D asset as an object within a computer simulation environment at the location and relative to other computer simulation assets.
Dependencies are inferred from the claim text.