Scene recognition based on labeled feature vectors of an image
AssigneeBlack Sesame Technologies Inc.
InventorsZi Sian Wong, Fangwen Tu, Bo Li
Life of this patent
Estimating the term…
Abstract
The present invention discloses a system and method for image processing and recognizing a scene of an image. The system utilizes a Multi-mode scalable network system and regrouping pipeline. The system is AI based system which uses neuro network. The system includes a pre-processing, processing and a post-processing unit. The system uses optical information recorded from the camera of a mobile device to extract and analyze the content in an image such as a photo or video clip. Based on the retrieved information, a label is given to best describe the scene of the image.
Claims13
An image processing system for recognizing a scene of an image, the image processing system comprising:
- a pre-processing unit for processing the image, wherein the pre-processing includes resizing the image, generating an attention region image and a background region image from the resized image, and re-stamping the attention region image on the background region image to generate a fused image;
- a processing unit, for processing the fused image, comprising:
- an encoder for generating a plurality of distinctive feature vectors of the fused image;
- a labeler for assigning a label from a set of pre-defined labels to each of the plurality of distinctive feature vectors to generate labelled vectors;
- a deep neural network for predicting a probability for each of the labelled vectors;
- and
- a class re-grouper for grouping the labelled vectors based on probability values of the probability to generate one or more parent labelled vectors;
- and
- a post-processing unit for generating scores for the one or more parent labelled vectors;
- wherein a final label is generated based on the scores in order to recognize a scene of the image.
A method for recognizing a scene of an image comprising:
- pre-processing the image, wherein the pre-processing comprises:
- resizing the image and generating an attention region image and a background region image from the resized image to generate a fused image;
- re-stamping the attention region image on the background region image to generate a fused image;
- processing the fused image, wherein the processing comprises:
- generating a plurality of distinctive feature vectors of the fused image;
- assigning a label from a set of pre-defined labels to each of the plurality of distinctive feature vectors to generate labelled vectors;
- predicting a probability for each of the labelled vectors;
- and
- regrouping the labelled vectors based on probability values of the probability to generate one or more parent labelled vectors;
- and
- generating scores based on the one or more parent labelled vector, and generating a final label to recognize the scene of the image based on the scores.
An AI based multi-media processing system for recognizing a scene from a multi-media, comprising:
- a pre-processing unit for processing the multi-media, wherein the pre-processing comprises resizing the multi-media, generating an attention region image and a background region image of the resized multi-media, and re-stamping the attention region image on the background region image to generate a fused multi-media;
- a processing unit for processing the fused multi-media, wherein the processing unit comprises:
- an encoder for generating a plurality of distinctive feature vectors of the fused multi-media;
- a labeler for assigning a label from a set of pre-defined labels to each of the plurality of distinctive feature vectors to generate labelled vectors;
- a deep neural network for predicting a probability for each of the labelled vectors;
- and
- a class re-grouper for grouping the labelled vectors based on probability values of the probability to generate one or more parent labelled vectors;
- and
- a post-processing unit for generating scores for the one or more parent labelled vectors, and generating a final label based on the scores to recognize the scene of the multi-media.
Dependencies are inferred from the claim text.