3D reference point detection for survey for venue model construction

A system, comprising:

• a computer system configured to:
• receive a plurality of survey images of a venue for an event and locations for a set of fiducial points for the venue in a real world coordinate system, where the location of each of the fiducial points in the real world coordinate system are determined in a survey of the venue and each of the survey images including one or more fiducial markers each located a corresponding one of the fiducial points and;
• process the survey images and the set of fiducial points to generate locations of the set of fiducial markers in the real world coordinate system by identifying corresponding fiducial markers within each of the survey images, determining a two dimensional position for each of the identified fiducial markers within the corresponding survey image, matching the identified fiducial markers between different ones of the survey images, and determining real world coordinates for the identified fiducial markers from the matching and from the locations of the fiducial points;
• receive from each of a plurality of mobile devices image data and corresponding image metadata, each of the mobile devices including a camera configured to generate the image data and a display to present a view of the event at the venue, and each of the mobile devices independently maintaining a corresponding internal coordinate system and the corresponding image metadata including information on a location and an orientation of the corresponding mobile device within a venue for an event in the internal coordinate system of the corresponding mobile device when capturing the image data;
• transmit to each of the mobile devices a corresponding transformation between the mobile device's corresponding internal coordinate system and the real world coordinate system;
• receive from each of the mobile devices receiving requests for content including graphics to be displayed by the mobile device over the view of the event at the venue as specified by location and orientation in the real world coordinate system;
• and
• transmit to each of the mobile devices the requested content;
• and
• one or more servers configured to access data from one or more databases and to:
• retrieve, from one or more databases, point features of the venue in a first coordinate system;
• determine the point features of the venue in the real world coordinate system from the set of fiducial markers for the venue in the real world coordinate system and the point features of the venue in the first coordinate system;
• build a model of the venue in the real world coordinate system from the point features of the venue and the locations of the set of fiducial markers in the real world coordinate system, the model comprising a reference map including location data of a set reference features in the real world coordinate system;
• generate for each of the mobile devices the corresponding transformation between the mobile device's corresponding internal coordinate system and the real world coordinate system from the image data and corresponding image metadata from each of the mobile devices and the model of the venue in the real world coordinate system.

A method, comprising:

• placing each of a plurality of fiducial markers at a corresponding fiducial point in a venue for an event;
• performing a survey of the venue, the survey including determining a location of each of the fiducial points in a real world coordinate system;
• subsequent to placing the fiducial markers in the venue, capturing survey images of the venue, each of the survey images including one or more fiducial markers;
• and
• processing the survey images and the fiducial points to generate locations of the fiducial markers in the real world coordinate system, including identifying corresponding fiducial markers within each of the survey images, determining a two dimensional position for each of the identified fiducial markers within the corresponding survey image, matching the identified fiducial markers between different ones of the survey images, and determining real world coordinates for the identified fiducial markers from the matching and from the locations of the fiducial points;
• receiving from each of a plurality of mobile devices image data and corresponding image metadata, each of the mobile devices including a camera configured to generate the image data and a display to present a view of the event at the venue, and each of the mobile devices independently maintaining a corresponding internal coordinate system and the corresponding image metadata including information on a location and an orientation of the corresponding mobile device within a venue for an event in the internal coordinate system of the corresponding mobile device when capturing the image data;
• generating for each of the mobile devices a corresponding transformation between the mobile device's corresponding internal coordinate system and the real world coordinate system from the image data and corresponding image metadata from each of the mobile devices and from the point features of the venue and the locations of the set of fiducial markers in the real world coordinate system;
• transmitting to each of the mobile devices the corresponding transformation between the mobile device's corresponding internal coordinate system and the real world coordinate system;
• receiving from each of the mobile devices receiving requests for content including graphics to be displayed by the mobile device over the view of the event at the venue as specified by location and orientation in the real world coordinate system;
• and
• transmitting to each of the mobile devices the requested content.