Generating realistic images from specified semantic maps

A method for generating images from a semantic map, which assigns to each pixel of the images a semantic meaning of an object to which this pixel belongs, the method comprising the following steps:

• providing the semantic map as a map tensor including a plurality of channels, each of the channels of the map tensor indicating all the pixels of the images to be generated, to which the semantic map assigns a specific semantic meaning;
• generating at least one set of variable pixels of the images to be generated, which are to vary from one image to the next image;
• generating, using values taken from at least one random distribution, a noise tensor with a plurality of channels, those values of the noise tensor which relate to the set of variable pixels being reused for each image to be generated;
• merging the channels of the map tensor with the channels of the noise tensor to yield an input tensor;
• and
• mapping the input tensor by a trained generator onto at least one image of the images to be generated.

A method for training a generator, comprising the following steps:

• providing respective real training images, and associated semantic training maps, which assign a semantic meaning to each pixel of the respective training image;
• generating images from at least one semantic training map by:
• providing the semantic map as a map tensor including a plurality of channels, each of the channels of the map tensor indicating all the pixels of the images to be generated, to which the semantic map assigns a specific semantic meaning,
• generating at least one set of variable pixels of the images to be generated, which are to vary from one image to the next image,
• generating, using values taken from at least one random distribution, a noise tensor with a plurality of channels, those values of the noise tensor which relate to the set of variable pixels being reused for each image to be generated,
• merging the channels of the map tensor with the channels of the noise tensor to yield an input tensor, and
• mapping the input tensor by a trained generator onto at least one image of the images to be generated;
• ascertaining at least one real training image for the same at least one semantic training map;
• supplying the images generated by the generator, and the at least one real training image, to a discriminator, which is configured to distinguish images generated by the generator from real images of scenery specified by the semantic training map;
• optimizing generator parameters, which characterize a behavior of the generator, with a goal that the realistic images generated by the generator are misclassified by the discriminator as real images;
• and
• optimizing discriminator parameters, which characterize a behavior of the discriminator, with a goal of improving accuracy when distinguishing between generated images and real images.

A method for training an image classifier which assigns an input image, and/or pixels of the input image, to a semantic meaning, the method comprising the following steps:

• generating images, using a trained generator, from semantic maps, by:
• providing each semantic map as a map tensor including a plurality of channels, each of the channels of the map tensor indicating all the pixels of the images to be generated, to which the semantic map assigns a specific semantic meaning,
• generating at least one set of variable pixels of the images to be generated, which are to vary from one image to the next image,
• generating, using values taken from at least one random distribution, a noise tensor with a plurality of channels, those values of the noise tensor which relate to the set of variable pixels being reused for each image to be generated,
• merging the channels of the map tensor with the channels of the noise tensor to yield an input tensor, and
• mapping the input tensor by a trained generator onto at least one image of the images to be generated;
• ascertaining nominal semantic meanings from each of the semantic, onto which meanings the trained image classifier is intended in each case to map the images;
• expanding a training data set for the image classifier, which contains real training images and associated nominal semantic meanings, by the generated images and associated nominal semantic meanings;
• and
• training the image classifier with the expanded training data set.

A method, comprising:

• training an image classifier by:
• generating images, using a trained generator, from semantic maps, by:
• providing each semantic map as a map tensor including a plurality of channels, each of the channels of the map tensor indicating all the pixels of the images to be generated, to which the semantic map assigns a specific semantic meaning,
• generating at least one set of variable pixels of the images to be generated, which are to vary from one image to the next image,
• generating, using values taken from at least one random distribution, a noise tensor with a plurality of channels, those values of the noise tensor which relate to the set of variable pixels being reused for each image to be generated,
• merging the channels of the map tensor with the channels of the noise tensor to yield an input tensor, and
• mapping the input tensor by a trained generator onto at least one image of the images to be generated,
• ascertaining nominal semantic meanings from each of the semantic, onto which meanings the trained image classifier is intended in each case to map the images,
• expanding a training data set for the image classifier, which contains real training images and associated nominal semantic meanings, by the generated images and associated nominal semantic meanings, and
• training the image classifier with the expanded training data set;
• assigning, using the image classifier, images which were captured with at least one sensor carried by a vehicle, a semantic meaning;
• ascertaining an actuation signal from the semantic meaning ascertained by the image classifier;
• actuating the vehicle using the actuation signal.

A non-transitory machine-readable data storage medium on which is stored a computer program for generating images from a semantic map, which assigns to each pixel of the images a semantic meaning of an object to which this pixel belongs, the computer program, when executed by one or more computers, causing the one or more computers to perform the following steps:

• providing the semantic map as a map tensor including a plurality of channels, each of the channels of the map tensor indicating all the pixels of the images to be generated, to which the semantic map assigns a specific semantic meaning;
• generating at least one set of variable pixels of the images to be generated, which are to vary from one image to the next image;
• generating, using values taken from at least one random distribution, a noise tensor with a plurality of channels, those values of the noise tensor which relate to the set of variable pixels being reused for each image to be generated;
• merging the channels of the map tensor with the channels of the noise tensor to yield an input tensor;
• and
• mapping the input tensor by a trained generator onto at least one image of the images to be generated.

A computer configured to generate images from a semantic map, which assigns to each pixel of the images a semantic meaning of an object to which this pixel belongs, the computer configured to:

• provide the semantic map as a map tensor including a plurality of channels, each of the channels of the map tensor indicating all the pixels of the images to be generated, to which the semantic map assigns a specific semantic meaning;
• generate at least one set of variable pixels of the images to be generated, which are to vary from one image to the next image;
• generate, using values taken from at least one random distribution, a noise tensor with a plurality of channels, those values of the noise tensor which relate to the set of variable pixels being reused for each image to be generated;
• merge the channels of the map tensor with the channels of the noise tensor to yield an input tensor;
• and
• map the input tensor by a trained generator onto at least one image of the images to be generated.