Image processing neural network systems and methods with scene understanding
AssigneeRutgers, The State University of New Jersey
InventorsDimitris Metaxas, Zachary A. Daniels
Life of this patent
Estimating the term…
Abstract
An image processing neural network system includes a base net of at least one convolutional layer and at least one pooling layer; and a scenario block layer. The scenario block layer performs scene classification and generates a dictionary of scenarios and a vector of scenario encoding coefficients to output a probabilistic scene class assignment and the vector of scenario encoding coefficients. The vector of scenario encoding coefficients corresponds to reasoning for the scene classification.
Claims20
An image processing neural network system comprising:
- at least one convolutional layer;
- at least one pooling layer;
- and
- a scenario block layer that performs scene classification and generates a dictionary of scenarios and a vector of scenario encoding coefficients to output a probabilistic scene class assignment and the vector of scenario encoding coefficients, wherein the vector of scenario encoding coefficients corresponds to reasoning for the scene classification, the reasoning for the scene classification comprising information of scenarios, including objects associated with each scenario, supporting the scene classification.
A method of image classification with scene understanding, comprising:
- receiving an image at an image processing neural network system;
- processing the image using the image processing neural network system to determine a scene assignment and a confidence score for the scene assignment and generate a dictionary of scenarios and a vector of scenario encoding coefficients;
- identifying scenarios associated with the scene assignment using the vector of scenario encoding coefficients for the image and the dictionary of scenarios;
- and
- outputting the scene assignment and the confidence score along with scenarios identified from the vector of scenario encoding coefficients such that both the scene assignment and information of scenarios, including objects associated with each scenario, supporting the scene assignment are output.
A storage medium having an application stored thereon, that when executed by a computing system directs the computing system to:
- provide at least one input image to an image processing neural network system;
- receive a probabilistic scene class assignment and a vector of scenario encoding coefficients from the image processing neural network system;
- and
- provide a reasoning for a scene classification of the probabilistic scene class assignment using the vector of scenario encoding coefficients as output such that both the probabilistic scene class assignment and information of scenarios, including objects associated with each scenario, supporting the probabilistic scene class assignment are output.
Dependencies are inferred from the claim text.