Conflation of geospatial points of interest and ground-level imagery

Memory encoding instructions that, when executed by a data processing apparatus, cause the data processing apparatus to perform operations comprising:

• accessing information about points of interest within an area of interest and information about scene images within the area of interest, wherein information about each of the points of interest comprises at least a point-of-interest category, and information about each of the scene images comprises M scene tags and respective scene-tag probabilities;
• encoding the information about each of the scene images as a respective scene-image vector with M coefficients in one-to-one correspondence with the scene tags, wherein coefficients of the scene-image vector are
• the respective scene-tag probabilities for a predetermined number of coefficients N<M corresponding to N-highest scene-tag probabilities, and
• zero for the remaining (M−N) coefficients;
• encoding the information about each of the points of interest as a respective point-of-interest vector with M coefficients in one-to-one correspondence with the scene tags, wherein the coefficients of the point-of-interest vector are respective scores of a word similarity between the point-of-interest category and the scene tags;
• and
• constructing a joint semantic graph comprising nodes and edges by performing operations comprising
• attributing to each node a respective point-of-interest vector or a respective scene-image vector,
• determining semantic distances between pairs of point-of-interest vectors, pairs of scene-image vectors, and pairs formed from a point-of-interest vector and a scene-image vector, and
• connecting each node with respective edges to at least some of a predetermined number of nearest-neighbor nodes K having respective vectors with the K-lowest semantic distances to each other.