Method of generating data by using artificial neural network model having encoder-decoder structure

A method of training a model, the method comprising:

• pre-training an encoder layer of an artificial neural network model having an encoder-decoder structure;
• inserting a latent vector layer between the encoder layer and a decoder layer of the artificial neural network model, wherein the inserting the of the latent vector layer between the encoder layer and the decoder layer of the artificial neural network model includes:
• transforming an output of the encoder layer into a pooling vector;
• transforming the pooling vector into a latent variable vector;
• and
• using the latent variable vector as an input value of the decoder layer through a projection matrix, wherein the using of the latent variable vector as the input value of the decoder layer through the projection matrix includes:
• performing cross attention of the decoder layer based on the latent variable vector and the projection matrix;
• and
• tuning the artificial neural network model so that the artificial neural network denoises input data, wherein the tuning comprises inducing output data to approach the input data based on the determined weight.

A computer program stored in a non-transitory computer readable storage medium including instructions for causing a computing device to perform operations, the operations comprising:

• an operation of generating new data based on a probability distribution of input data by using an artificial neural network model having an encoder-decoder structure,
• wherein the artificial neural network model having the encoder-decoder structure corresponds to an artificial neural network model trained based on:
• pre-training an encoder layer and an decoder layer of the artificial neural network model, wherein the pre-training step includes determining a weight;
• inserting a latent vector layer between the encoder layer and the decoder layer of the artificial neural network model, wherein the inserting the of the latent vector layer between the encoder layer and the decoder layer of the artificial neural network model includes:
• transforming an output of the encoder layer into a pooling vector;
• transforming the pooling vector into a latent variable vector;
• and
• using the latent variable vector as an input value of the decoder layer through a projection matrix, wherein the using of the latent variable vector as the input value of the decoder layer through the projection matrix includes:
• performing cross attention of the decoder layer based on the latent variable vector and the projection matrix;
• and
• tuning the artificial neural network so that the artificial neural network denoises the input data, wherein the tuning comprises inducing output data to approach the input data based on the determined weight.

A computing device, comprising:

• a processor including one or more cores;
• a network unit for receiving one or more data;
• and
• a memory,
• wherein the processor configured to generate new data based on a probability distribution of input data by using an artificial neural network model having an encoder-decoder structure, and
• the artificial neural network model having the encoder-decoder structure corresponds to an artificial neural network model trained based on:
• pre-training an encoder layer and an decoder layer of the artificial neural network model, wherein the pre-training step includes determining a weight;
• inserting a latent vector layer between the encoder layer and the decoder layer of the artificial neural network model, wherein the inserting the of the latent vector layer between the encoder layer and the decoder layer of the artificial neural network model includes:
• transforming an output of the encoder layer into a pooling vector;
• transforming the pooling vector into a latent variable vector;
• and
• using the latent variable vector as an input value of the decoder layer through a projection matrix, wherein the using of the latent variable vector as the input value of the decoder layer through the projection matrix includes:
• performing cross attention of the decoder layer based on the latent variable vector and the projection matrix;
• and
• tuning the artificial neural network so that the artificial neural network denoises the input data, wherein the tuning comprises inducing output data to approach the input data based on the determined weight.