System and method for learning scene embeddings via visual semantics and application thereof
AssigneeYahoo Assets LLC
InventorsPaloma de Juan, Aasish Pappu
Life of this patent
Estimating the term…
Abstract
The present teaching relates to method, system, and programming for responding to an image related query. Information related to each of a plurality of images is received, wherein the information represents concepts co-existing in the image. Visual semantics for each of the plurality of images are created based on the information related thereto. Representations of scenes of the plurality of images are obtained via machine learning, based on the visual semantics of the plurality of images, wherein the representations capture concepts associated with the scenes.
Claims20
A method, implemented on a machine having at least one processor, storage, and a communication platform for responding to an image related query for an image, comprising:
- receiving the image related query with an associated annotation;
- establishing, based on the annotation, visual semantics of the image, wherein the visual semantics of the image comprises a hierarchy of abstraction including a first level of abstraction naming a type of the image abstracted based on what are observed in an entire scene of the image and a second level of abstraction with a category name for each group of objects observed in the image, wherein the first level of abstraction does not directly describe any object in the image;
- and
- providing, based on the visual semantics, a response to the image related query.
A non-transitory, computer-readable medium having information recorded thereon for responding to an image related query for an image, when read by at least one processor, effectuate operations comprising:
- receiving the image related query with an associated annotation;
- establishing, based on the annotation, visual semantics of the image, wherein the visual semantics of the image comprises a hierarchy of abstraction including a first level of abstraction naming a type of the image abstracted based on what are observed in an entire scene of the image and a second level of abstraction with a category name for each group of objects observed in the image, wherein the first level of abstraction does not directly describe any object in the image;
- and
- providing, based on the visual semantics, a response to the image related query.
A system for online user profiling, the system comprising:
- memory storing computer program instructions;
- and
- one or more processors that, in response to executing the computer program instructions, effectuate operations comprising:
- receiving the image related query with an associated annotation;
- establishing, based on the annotation, visual semantics of the image, wherein the visual semantics of the image comprises a hierarchy of abstraction including a first level of abstraction naming a type of the image abstracted based on what are observed in an entire scene of the image and a second level of abstraction with a category name for each group of objects observed in the image, wherein the first level of abstraction does not directly describe any object in the image;
- and
- providing, based on the visual semantics, a response to the image related query.
Dependencies are inferred from the claim text.