Back to results
US 11,379,722 B2Granted
Method for training generative adversarial network (GAN), method for generating images by using GAN, and computer readable storage medium
AssigneeHTC Corporation
InventorsEdward Chang, Che-Han Chang, Chun-Hsien Yu, Szu-Ying Chen
Life of this patent
Estimating the term…
Abstract
The disclosure provides a method for training generative adversarial network (GAN), a method for generating images by using GAN, and a computer readable storage medium. The method may train the first generator of the GAN with available training samples belonging to the first type category and share the knowledge learnt by the first generator to the second generator. Accordingly, the second generator may learn to generate (fake) images belonging to the second type category even if there are no available training data during training the second generator.
Claims20
A method for training a generative adversarial network (GAN), wherein the GAN comprises a first generator, a second generator, a discriminator, and a prediction network, comprising:
- receiving, by the first generator, a first random input and a first category indication and accordingly generating a first output image, wherein the first generator and the second generator are both characterized by a plurality of first neural network weightings, the first category indication indicates that the first output image corresponds to a first type category, and the first type category has available training samples;
- predicting, by the prediction network, a first semantic embedding vector corresponding to the first output image;
- generating a first comparing result by comparing the first semantic embedding vector with a second semantic embedding vector corresponding to the first type category;
- receiving, by the second generator, a second random input and a second category indication and accordingly generating a second output image, wherein the second category indication indicates that the second output image corresponds to a second type category;
- predicting, by the prediction network, a third semantic embedding vector corresponding to the second output image;
- generating a second comparing result by comparing the third semantic embedding vector with a fourth semantic embedding vector corresponding to the second type category;
- generating, by the discriminator, a discriminating result via discriminating between the first output image and at least one reference image belonging to the first type category, wherein the discriminator is characterized by a plurality of second neural network weightings;
- updating the second neural network weightings based on the discriminating result;
- updating the first neural network weightings based on the discriminating result, the first comparing result and the second comparing result.
6 more elements
A non-transitory computer readable storage medium, recording an executable computer program to be loaded by a training system for training a generative adversarial network (GAN) comprising a first generator, a second generator, a discriminator and a prediction network to execute steps of:
- receiving, by the first generator, a first random input and a first category indication and accordingly generating a first output image, wherein the first generator and the second generator are both characterized by a plurality of first neural network weightings, the first category indication indicates that the first output image corresponds to a first type category, and the first type category has available training samples;
- predicting, by the prediction network, a first semantic embedding vector corresponding to the first output image;
- generating a first comparing result by comparing the first semantic embedding vector with a second semantic embedding vector corresponding to the first type category;
- receiving, by the second generator, a second random input and a second category indication and accordingly generating a second output image, wherein the second category indication indicates that the second output image corresponds to a second type category;
- predicting, by the prediction network, a third semantic embedding vector corresponding to the second output image;
- generating a second comparing result by comparing the third semantic embedding vector with a fourth semantic embedding vector corresponding to the second type category;
- generating, by the discriminator, a discriminating result via discriminating between the first output image and at least one reference image belonging to the first type category, wherein the discriminator is characterized by a plurality of second neural network weightings;
- updating the second neural network weightings based on the discriminating result;
- updating the first neural network weightings based on the discriminating result, the first comparing result and the second comparing result.
6 more elements
A method for generating images by using a generative adversarial network (GAN) comprising a first generator and a second generator, comprising:
- receiving, by the first generator, a first random input and a first category indication and accordingly generating a first output image, wherein the first generator and the second generator are both characterized by a plurality of first neural network weightings, the first category indication indicates that the first output image corresponds to a first type category;
- receiving, by the second generator, a second random input and a second category indication and accordingly generating a second output image, wherein the second category indication indicates that the second output image corresponds to a second type category, and only a plurality of training samples belonging to the second type category are previously used to train the first generator and the second generator.
A method for training a generative adversarial network (GAN), wherein the GAN comprises a first generator, a second generator, a discriminator and a color estimator, comprising:
- receiving, by the first generator, a first input image and a category indication and accordingly generating a first output image via replacing a first color of a first specific region in the first input image with a first target color, wherein the first target color belongs to a first type category having a plurality of training color samples, and the first generator and the second generator are partially characterized by a plurality of first neural network weightings;
- generating, by the discriminator, a discriminating result and a classification result based on the first output image;
- receiving, by the second generator, a second input image and a target color indication and accordingly generating a second output image via replacing a second color of a second specific region in the second input image with a second target color, wherein the second target color corresponds to the target color indication, and the second target color does not belonging to the first type category;
- estimating, by the color estimator, a region color corresponding to the second specific region in the second output image and generating a color comparing result by comparing the region color with the target color;
- generating, by the first generator, a cycle image according to the second output image and an original category indication and generating a cycle-consistency result by comparing the cycle image with the second input image;
- updating the discriminator based on the discriminating result and the classification result;
- updating the first generator and the second generator based on the discriminating result, the color comparing result, and the cycle-consistency result.
4 more elements
A non-transitory computer readable storage medium, recording an executable computer program to be loaded by a training system for training a generative adversarial network (GAN) comprising a first generator, a second generator, a discriminator, and a color estimator to execute steps of:
- receiving, by the first generator, a first input image and a category indication and accordingly generating a first output image via replacing a first color of a first specific region in the first input image with a first target color, wherein the first target color belongs to a first type category having a plurality of training color samples, and the first generator and the second generator are partially characterized by a plurality of first neural network weightings;
- generating, by the discriminator, a discriminating result and a classification result based on the first output image;
- receiving, by the second generator, a second input image and a target color indication and accordingly generating a second output image via replacing a second color of a second specific region in the second input image with a second target color, wherein the second target color corresponds to the target color indication, and the second target color does not belonging to the first type category;
- estimating, by the color estimator, a region color corresponding to the second specific region in the second output image and generating a color comparing result by comparing the region color with the target color;
- generating, by the first generator, a cycle image according to the second output image and an original category indication and generating a cycle-consistency result by comparing the cycle image with the second input image;
- updating the discriminator based on the discriminating result and the classification result;
- updating the first generator and the second generator based on the discriminating result, the color comparing result, and the cycle-consistency result.
4 more elements
A method for generating images by using a generative adversarial network (GAN) comprising a first generator and a second generator, comprising:
- receiving, by the first generator, a first input image and a category indication and accordingly generating a first output image via replacing a first color of a first specific region in the first input image with a first target color, wherein the first target color belongs to a first type category having a plurality of training color samples, and the training color samples are previously used to train the first generator and the second generator;
- receiving, by the second generator, a second input image and a target color indication and accordingly generating a second output image via replacing a second color of a second specific region in the second input image with a second target color, wherein the second target color corresponds to the target color indication, and the second target color does not belonging to the first type category.
Dependencies are inferred from the claim text.