Generating synthetic image data
AssigneeSynapse Technology Corporation
InventorsIan Cinnamon, Bruno Brasil Ferrari Faviero, Simanta Gautam
Life of this patent
Estimating the term…
Abstract
According to an aspect, a method comprises: generating a 2D projection from a 3D representation of an object, generating, based on the 2D projection, a simulated image of the object, wherein the simulated image appears as though the object has been scanned by a detection device, combining the simulated object with a background image to form a synthesized image, wherein the background image was captured by a detection device, and outputting the synthesized image.
Claims20
A method for generating synthesized images for use in a baggage security screening system, the method comprising:
- receiving, by a processor, an image of a bag, wherein the image of the bag was captured by a scanning device of the baggage security screening system;
- receiving, by the processor, an image of an object, wherein the image of the object is an image of the object not placed inside the bag;
- inputting an edgemapped version of the image of the object into a first generative adversarial neural network (GAN);
- generating, by the first GAN and based on the inputted edgemapped version of the image, a synthesized image of the object that has image characteristics of images of a type of the object captured by the scanning device;
- overlaying the image of the bag with the synthesized image of the object to produce an overlaid image, wherein the overlaying comprises:
- determining an inside perimeter of the bag from the image of the bag;
- selecting a region of the image of the bag inside the perimeter to be overlaid by the synthesized image of the object generated by the first GAN;
- and
- overlaying, by the processor, the image of the bag with the synthesized image of the object generated by the first GAN to form the overlaid image having the synthesized image of the object completely inside the perimeter of the bag;
- refining, with a second generative adversarial neural network (GAN), the selected region of the overlaid image that contains the synthesized image of the object to form a refined overlaid image,
- wherein refining the selected region of the overlaid image comprises:
- after forming the overlaid image:
- inputting the selected region of the overlaid image into the second GAN;
- and
- refining, by the second GAN and based on the inputted selected region of the overlaid image, the selected region of the overlaid image to match image characteristics of the image of the bag;
- and
- after forming the refined overlaid image:
- training an object detection machine learning algorithm with the refined overlaid image.
A system for generating for generating synthesized images for use in a baggage security screening system, the system comprising:
- a memory storing instructions thereon;
- a first generative adversarial neural network (GAN);
- a second generative adversarial neural network (GAN);
- and
- at least one processor,
- wherein the instructions stored on the memory, when executed, cause the at least one processor to:
- receive an image of a bag, wherein the image of the bag was captured by a scanning device of the baggage security screening system
- receive an image of an object, wherein the image of the object is an image of the object not placed inside the bag;
- input an edgemapped version of the image of the object into the first GAN;
- generate, by the first GAN and based on the inputted edgemapped version of the image, a synthesized image of the object that matches image characteristics of images of a type of the object captured by the scanning device;
- overlay the image of the bag with the synthesized image of the object generated by the first GAN to produce an overlaid image,
- wherein the instructions that cause the at least one processor to overlay the image of the bag with the synthesized image of the object comprise instructions that, when executed, cause the at least one processor to:
- determine an inside perimeter of the bag from the image of the bag;
- select a region of the image of the bag inside the perimeter to be overlaid by the synthesized image of the object generated by the first GAN;
- and
- overlay the image of the bag with the synthesized image of the object generated by the first GAN to form the overlaid image having the synthesized image of the object completely inside the perimeter of the bag;
- refine, with the second GAN the selected region of the overlaid image that contains the synthesized image of the object to form a refined overlaid image,
- wherein the instructions that cause the at least one processor to refine the selected region of the overlaid image comprise instructions that, when executed, cause the at least one processor to:
- input the selected region of the overlaid image into the second GAN;
- and
- refine, by the second GAN and based on the inputted selected region of the overlaid image, the selected region of the overlaid image to match image characteristics of the image of the bag;
- and
- after forming the refined overlaid image:
- train an object detection machine learning algorithm with the refined overlaid image.
A non-transitory computer-readable storage medium comprising instructions stored thereon for generating synthesized images for use in a baggage security system, wherein the instructions, when executed, cause at least one processor to:
- receive an image of a bag, wherein the image of the bag was captured by a scanning device;
- receive an image of an object, wherein the image of the object is an image of the object not placed inside the bag;
- input an edgemapped version of the object into a first generative adversarial neural network (first GAN);
- generate, by the first GAN and based on the inputted edgemapped version of the image, a synthesized image of the object that has image characteristics of images of a type of the object captured by the scanning device;
- overlay the image of the bag with the synthesized image of the object generated by the first GAN to produce an overlaid image,
- wherein the instructions that cause the at least one processor to overlay the image of the bag with the synthesized image of the object generated by the first GAN to produce the overlaid image comprise instructions that, when executed, cause the at least one processor to:
- determine an inside perimeter of the bag from the image of the bag;
- select a region of the image of the bag inside the perimeter to be overlaid by the synthesized image of the object generated by the first GAN;
- and
- overlay the image of the bag with the synthesized image of the object generated by the first GAN and the image of the object to form the overlaid image having the synthesized image of the object completely inside the perimeter of the bag;
- refine, with a second generative adversarial neural network (second GAN), a portion of the overlaid image that contains the synthesized image of the object to form a refined overlaid image,
- wherein the instructions that cause the at least one processor to refine the selected region of the overlaid image comprise instructions that, when executed, cause the at least one processor to:
- after forming the overlaid image:
- input the selected region of the overlaid image into the second GAN;
- refine, by the second GAN and based on the inputted selected region of the overlaid image, the selected region of the overlaid image to match image characteristics of the image of the bag;
- and
- after forming the refined overlaid image:
- train an object detection machine learning algorithm with the refined overlaid image.
Dependencies are inferred from the claim text.