Determining location information for images using landmark, caption, and metadata location data

A system comprising:

• a data processing apparatus;
• and
• a data store storing instructions that, when executed by the data processing apparatus, cause the data processing apparatus to perform operations comprising:
• obtaining landmark location data for an image, the landmark location data specifying one or more landmark locations derived from content depicted in the image and, for each landmark location, a landmark confidence score that indicates a measure of confidence that the image depicts content associated with the landmark location;
• obtaining caption location data for the image, the caption location data specifying one or more caption locations derived from user input and, for each caption location, a caption confidence score that indicates a measure of confidence that the image depicts content associated with the caption location;
• obtaining metadata location data for the image, the metadata location data specifying a metadata location derived from data provided by an image capturing device that captured the image;
• identifying location pairs from the landmark location data, caption location data, and metadata location data, each location pair including two of a landmark location, caption location, and metadata location;
• generating, for each location pair, a geographic consistency score that indicates a measure of consistency between locations specified in the location pair;
• ranking each location pair according to a predetermined hierarchy of location types, the predetermined hierarchy specifying a first priority for landmark locations, a second priority for caption locations, and third priority for metadata locations, wherein the ranking of each location pair is based on a highest priority of the locations specified in the location pair;
• selecting a location pair based at least in part on the geographic consistency scores and location pair rankings;
• selecting an image location for the image from the locations specified in the selected location pair;
• determining an image location score for the image location based on a confidence score for one of the locations specified in the selected location pair;
• and
• associating the image location and image location score with the image.

A computer readable storage medium comprising instructions that, when executed by a data processing apparatus, cause the data processing apparatus to perform operations comprising:

• obtaining landmark location data for an image, the landmark location data specifying one or more landmark locations derived from content depicted in the image and, for each landmark location, a landmark confidence score that indicates a measure of confidence that the image depicts content associated with the landmark location;
• obtaining caption location data for the image, the caption location data specifying one or more caption locations derived from user input and, for each caption location, a caption confidence score that indicates a measure of confidence that the image depicts content associated with the caption location;
• obtaining metadata location data for the image, the metadata location data specifying a metadata location derived from data provided by an image capturing device that captured the image;
• identifying location pairs from the landmark location data, caption location data, and metadata location data, each location pair including two of a landmark location, caption location, and metadata location;
• generating, for each location pair, a geographic consistency score that indicates a measure of consistency between locations specified in the location pair;
• ranking each location pair according to a predetermined hierarchy of location types, the predetermined hierarchy specifying a first priority for landmark locations, a second priority for caption locations, and third priority for metadata locations, wherein the ranking of each location pair is based on a highest priority of the locations specified in the location pair;
• selecting a location pair based at least in part on the geographic consistency scores and location pair rankings;
• selecting an image location for the image from the locations specified in the selected location pair;
• determining an image location score for the image location based on a confidence score for one of the locations specified in the selected location pair;
• and
• associating the image location and image location score with the image.