US 11,694,089 B1Granted

Deep-learned photorealistic geo-specific image generator with enhanced spatial coherence

AssigneeRockwell Collins, Inc.

InventorsDaniel J. Lowe, Rishabh Kaushik

Life of this patent

Estimating the term…

Loading patent…

Abstract

A simulation environment is disclosed. In embodiments, the simulation environment includes deep learning neural networks trained to generate photorealistic geotypical image content while preserving spatial coherence. The deep learning networks are trained to correlate geo-specific datasets with input images of increasing detail and resolution to iteratively generate output images until the desired level of detail is reached. The correlation of image input with geo-specific data (e.g., labeled data elements, land use data, biome data, elevational data, near-IR imagery) preserves spatial coherence of objects and image elements between output images, e.g., between adjacent levels of detail and/or between adjacent image tiles sharing a common level of detail.

Claims14

Dependent claims

A simulation environment, comprising:

one or more graphics generation (GG) processors;
at least one non-transitory memory unit in communication with the GG processors, the memory unit configured for storing one or more of:
a set of spatially coherent basemap images associated with a geographical area and with a minimal detail level;
and
a geo-specific dataset associated with one or more characteristics of the geographical area;
and
one or more deep learning (DL) neural networks configured to execute on the GG processors, the one or more DL neural networks configured to generate at least one spatially coherent target image, the target image associated with at least one location within the geographical area and a target detail level D by, for at least one detail level N greater than the minimal detail level:
receiving one or more input images corresponding to the at least one location and to a detail level M lower than the detail level N;
receiving the one or more geo-specific datasets corresponding to the at least one location;
generating one or more spatially coherent primary output images corresponding to the at least one location and associated with the detail level N by correlating the one or more geo-specific datasets to the one or more input images;
and
while the detail level N is lower than the target detail level D, generating, in real time while the geographical area is being traversed in the simulation environment, one or more spatially coherent secondary output images corresponding to the at least one location and associated with a detail level N + 1 by correlating the one or more geo-specific datasets to the one or more spatially coherent primary output images,
wherein the one or more geo-specific datasets include at least one of:
geographical vector data;
ecological data;
infrared imagery data configured to be used to improve an accuracy of generated foliage in the one or more spatially coherent primary output images;
near-infrared imagery data configured to be used to improve the accuracy of generated foliage in the one or more spatially coherent primary output images;
or
land-use data,
and
wherein each of the one or more geo-specific datasets includes data indicating a presence of one or more types of features, and wherein the one or more spatially coherent primary and secondary output images depict the one or more types of features and the one or more types of features are spatially coherent across the one or more spatially coherent primary and secondary output images.

18 more elements

A method for generating spatially coherent photorealistic geo-specific imagery, the method comprising:

providing to at least one deep learning (DL) neural network, the DL neural network configured to execute on one or more graphics generator (GG) processors, one or more input images corresponding to a location and associated with a detail level M lower than a desired detail level D;
providing to the at least one DL neural network one or more geo-specific datasets corresponding to the location;
generating, via the at least one DL neural network, one or more spatially coherent primary output images by correlating the one or more geo-specific datasets to the one or more input images, the one or more spatially coherent primary output images associated with a detail level N greater than the detail level M;
and
while the detail level N is lower than the desired detail level D, generating, via the at least one DL neural network in real time while a geographical area is being traversed in the simulation environment, one or more spatially coherent secondary output images by correlating the one or more geo-specific datasets to the one or more spatially coherent primary output images , the one or more secondary output images associated with a detail level N + 1,
wherein the one or more spatially coherent primary and secondary output images depict the one or more types of features and the one or more types of features are spatially coherent across the one or more spatially coherent primary and secondary output images.
wherein the one or more geo-specific datasets include at least one of:
geographical vector data;
ecological data;
infrared imagery data configured to be used to improve an accuracy of generated foliage in the one or more spatially coherent primary output images;
near-infrared imagery data configured to be used to improve the accuracy of generated foliage in the one or more spatially coherent primary output images;
or
land-use data;
and
wherein each of the one or more geo-specific datasets includes data indicating a presence of one or more types of features, and wherein the one or more spatially coherent primary output images depict the one or more types of features and the one or more types of features are spatially coherent across the one or more spatially coherent primary output images.

12 more elements

Dependencies are inferred from the claim text.