Method and apparatus for integration of detected object identifiers and semantic scene graph networks for captured visual scene behavior estimation

A computer system executing code of visual scene feature extractor and feature classifier system comprising:

• a processor executing object recognition on a captured image of a visual scene of interest to identify a first object and a second object in the visual scene of interest and a set of feature attributes captured for the first and second object, wherein the feature attributes include an image view of the first object and an image view of the second object;
• the processor executing code instructions for the visual scene feature extractor and classifier to generate a semantic scene graph including a object candidate node for the first object and a plurality of object candidate nodes for the second object and a location attribute node for the visual scene of interest;
• the processor associating a multi-feature graph for a plurality of object candidate nodes, wherein the multi-feature graph includes feature vector values for feature attributes of the first object or the second object in feature spaces each including a neighborhood of labeled, previously-observed feature vector values for assessment of nearness to an observed feature vector value of the first object or the second object;
• the processor to generate an association edge between the first object candidate node and a second object candidate node and to generate an affinity label for the association edge;
• the processor to generate a likelihood for accuracy of that affinity label for the association edge based on probabilistic graph analysis of other object candidate nodes, the attribute location node, and association edges in the semantic scene graph including at least one multi-feature graph for at least the first object or the second object;
• and
• the processor transmitting a graphical output indication of the likelihood for accuracy for the affinity label for the association edge.

A computer system executing code of visual scene feature extractor and feature classifier system comprising:

• a network interface for receiving a captured image of a visual scene of interest including a plurality of objects in the visual scene of interest and a position identifier of a location of the visual scene of interest;
• a processor executing object recognition to identify a first object and a second object of the plurality of objects within the visual scene of interest for analysis in a semantic scene graph and a visual scene feature extractor to classify the first object by association with a first set of feature attributes in a multi-feature graph wherein the feature attributes include an image view of the first object at the location;
• the processor executing code to access available contextual data for a plurality of object candidate nodes corresponding to at least the identified first object and second object within the visual scene of interest and generate an association edge with labeled affinity between the object candidate nodes for the first object and the second object;
• the processor to generate a weighted likelihood for object candidate nodes as being the first object from analysis of nearness of previously-observed feature vector nodes to currently observed vector nodes within feature spaces for the first set of feature attributes, frequency of previously-observed feature vector nodes within neighborhoods of the feature spaces, and association edges within the semantic scene graph relating to the first object.