Intelligent Object Magnification for Augmented Reality Displays

A computer-implemented method of intelligent object magnification the method comprising:

• capturing by a digital camera a real-time image stream;
• accessing the image stream by a first computer processor having at least one computer vision algorithm to classify a target of a pre-defined category from the image stream, the image stream of the classified target having a foreground representing the target and a background in which the target exists;
• segmenting into an image region the foreground of the classified target by the first computer processor or a second computer processor from the background such that substantially only those pixels showing the image region are further processed;
• transmitting the image region to a rendering engine to generate an output for a plurality of augmented reality see-through displays;
• creating in the rendering engine a first mathematical transformation defining the relative spatial difference between the digital camera and the augmented reality see-through displays;
• further creating in the rendering engine, a second mathematical transformation defining the factor by which the segmented image region should be up-scaled compared to the unmodified, classified target;
• generating in the rendering engine pixels of the image region of the classified target, the image region modified by the first and second transformations;
• and
• presenting the pixels of the image region superimposed over the otherwise unmagnified background through the augmented reality see-through displays whereby each augmented reality see-through display registers its position and orientation so that individual users of the augmented reality see-through displays looking in different directions have a consistent spatial representation of the classified target relative to each individual user's line of sight.

An apparatus for intelligent object magnification, the apparatus comprising:

• a digital camera for capturing a real-time image stream;
• a first computer processor accessing at least one computer vision algorithm to classify a target of a pre-defined category from the image stream, the image stream of the classified target having a foreground representing the target and a background in which the target exists;
• a segmenting module communicatively coupled to the first computer processor for segmenting into an image region the foreground of the classified target by the first computer processor or a second computer processor from the background such that substantially only those pixels showing the image region are further processed;
• a rendering engine communicatively coupled to the first computer processor, the rendering engine receiving the image region to generate an output for an augmented reality see-through display;
• a first mathematical transformation module communicatively coupled to the rendering engine, the first mathematical transformation module defining the relative spatial difference between the digital camera and the augmented reality see-through display;
• a second mathematical transformation module communicatively coupled to the rendering engine, the second mathematical transformation module defining the factor by which the segmented image region should be up-scaled compared to the unmodified, classified target;
• a pixel generating module communicatively coupled to the rendering engine, the pixel generating module generating the image region of the classified target, the image region modified by the first and second transformations;
• and
• a presentation module communicatively coupled to the rendering engine, the presentation module presenting the pixels of the image region superimposed over the otherwise unmagnified background through the augmented reality see-through display.