Method and system of image based anomaly localization for vehicles through generative contextualized adversarial network
AssigneeHong Kong Applied Science and Technology Research Institute Company Limited
InventorsWai Kai Arvin TANG, Pak Kan WONG
Life of this patent
Estimating the term…
Abstract
The present invention provides an anomaly detection method and apparatus based on a neural network which can be trained on undamaged normal vehicle images and able to detect unknown/unseen vehicle damages of stochastic types and extents from images which are taken in various contexts. The provided method and apparatus are implemented with functional units which are trained to perform the anomaly detection under a GCAN model with a training dataset containing images of undamaged vehicles, intact-vehicle frame images and augmented vehicle frame images of the vehicles.
Claims11
An apparatus for image-based anomaly detection for vehicle damage assessment, comprising:
- an image processor configured for segmenting one or more input vehicle images;
- a texture encoder configured for generating texture feature maps from the segmented vehicle images;
- a frame extractor configured for extracting one or more vehicle frame images from the segmented vehicle images;
- an intact-vehicle frame (IVF) reconstructor configured for reconstructing one or more IVF images from the extracted vehicle frame images;
- an image reconstructor configured for reconstructing one or more intact-vehicle images from the reconstructed IVF images and the texture feature maps;
- and
- an anomaly locator configured for comparing the segmented vehicle images and their corresponding reconstructed intact-vehicle images to detect one or more anomaly regions;
- wherein the texture encoder, the frame extractor, the IVF reconstructor, the image reconstructor, and the anomaly locator are generative contextualized adversarial network (GCAN) generators trained under a GCAN model with a training dataset comprising a plurality of images of undamaged vehicles, IVF images, and augmented vehicle frame images of the vehicles.
(canceled)
A method for image-based anomaly detection for vehicle damage assessment, comprising:
- performing image segmentation on an image of a vehicle image to generate a segmented vehicle image;
- generating a texture feature map for a damaged vehicle from the segmented vehicle image;
- extracting a damaged-vehicle frame image from the segmented vehicle image;
- reconstructing an IVF image from the damaged-vehicle frame image;
- reconstructing an intact-vehicle image from the reconstructed IVF image;
- comparing the segmented vehicle image and the reconstructed intact-vehicle image pixel by pixel;
- generating an anomaly region detection score for each pixel and classifying each of compared pixels of the reconstructed intact-vehicle image as anomaly region if its anomaly region detection score is larger than a threshold;
- and
- annotating the classified anomaly region on the reconstructed intact-vehicle image to produce an anomaly-annotated vehicle image;
- wherein the method is performed by generative contextualized adversarial network (GCAN) generators trained under a GCAN model with a training dataset comprising a plurality of images of undamaged vehicles, IVF images, and augmented vehicle frame images of the vehicles.
(canceled)
A method for training an apparatus for image-based anomaly detection for vehicle damage assessment, comprising:
- extracting an IVF image from an input vehicle image by a frame extractor, wherein the input vehicle image is a real or augmented image of a normal vehicle;
- extracting a texture feature map from the input vehicle image by a texture encoder;
- generating a damage-augmented vehicle frame image by performing a damage augmentation to the IVF image by an image processor;
- generating a first IVF feature map from the damage-augmented vehicle frame image by a frame encoder;
- reconstructing an IVF image from the first IVF feature map by a frame decoder;
- reconstructing an intact-vehicle image from the reconstructed IVF image and the texture feature map by an image reconstructor;
- extracting a second reconstructed IVF image from the reconstructed intact-vehicle image by the frame extractor;
- generating a second IVF feature map from the second reconstructed IVF image by the frame encoder;
- generating a score of likelihood between the first IVF feature map and the second IVF feature map by a frame encoder discriminator;
- and
- using the score of likelihood between the first IVF feature map and the second IVF feature map to optimize the parameterization of the frame encoder and the frame extractor.
Dependencies are inferred from the claim text.