Method and System for Constructing a Virtual Image Anchored onto a Real-World Object

A system for generating and displaying visual content comprising:

• a sensor set configured to obtain one or more data frames comprising spatial data of a 3D real-world space, the sensor set comprising one or more sensors, at least one sensor in the sensor set having an associated sensing field in the 3D real-world space and a sensor view ray, the sensor view ray being a vector extending from the sensor into the 3D real-world space in a 3D real-world coordinate system, at least one sensor in the sensor set having an associated up-vector that is fixed relative to the sensor, one or more of the sensor up-vectors defining an up-vector of a data frame;
• a viewing or display device;
• a computer processing unit comprising a processor and a memory, wherein:
• the memory stores data indicative of one or more objects to be detected in a data frame, and further stores, for each object to be detected, a predetermined real-world reference point that is fixed relative to the object;
• and the processor is configured to:
• a. analyze one or more the data frames and determine whether one or more of the real-world objects to be detected is in a current real-world view frustum;
• b. if at least a portion of one of the real-world objects is detected in the current frustum, determine from the current data frame the coordinates in the real-world of the real-world reference point associated with the detected real-world object in a real-world coordinate system associated with the real-world viewing frustum;
• c. determine a real-world object vector emanating from a predetermined point in the real-world viewing frustum, to the real-world reference point of the detected real-world object;
• d. determine a position of a gravitational vector associated with the data frame relative to the real-world viewing frustum;
• e. construct a virtual 3D space having a 3D virtual space coordinate system and position in the 3D virtual space a virtual viewing pyramid having an associated virtual world viewpoint, an associated virtual view frustum positioned symmetrically around a virtual view ray emanating from the virtual viewpoint, an associated virtual viewing surface forming a surface of the virtual view frustum and an associated virtual up vector where the x and y axes in a virtual coordinate system of the virtual up vector are parallel to the virtual viewing surface and the z axis is parallel to the virtual view ray;
• f. construct in the 3D virtual space a base plane passing through the virtual object reference point, the base plane having an associated 2D-coordinate system (x′, z′);
• g. place one or more planar content elements on the base plane or parallel to the base plane in any one or more predetermined permissible locations on the base plane;
• h. project the content elements positioned in the 3D virtual space onto the virtual viewing surface to form a projected image;
• i. form a final image the final image comprising the projected image;
• and
• j. display the final image the viewing or display device.