Method and apparatus for inserting a high resolution image into a low resolution interactive image to produce a realistic immersive experience

A method of capturing images for image overlay comprising the steps of

• capturing a wide angle interactive image at a first camera location,
• capturing at least one high resolution image portion in the wide angle interactive image;
• dewarping the captured wide angle interactive image to provide a perspective corrected image portion of the captured wide angle interactive image using an image transformation process;
• magnifying said perspective corrected portion in response to user input;
• as a user input degree of magnification of said perspective corrected image portion increases and a predetermined magnification threshold is exceeded, matching pixel values representing at least three different reference points in said perspective corrected image portion with pixel values representing at least three corresponding reference points in said high resolution image, said three different reference points defining a correspondingly similar planar area;
• and
• based on said match, automatically overlaying the high resolution image portion to replace said planar area of the perspective corrected image portion at said predetermined magnification threshold.

A method of processing a wide angle interactive image and a high resolution image portion comprising the steps of:

• storing said wide angle interactive image;
• storing said high resolution image portion;
• receiving input comprising a value of magnification;
• perspective correcting a wide angle interactive image portion using an image transformation process;
• matching pixel values representing at least three different reference points in said perspective corrected image portion with pixel values representing at least three corresponding reference points in said high resolution image portion, said three different reference points defining a planar area to be overlaid;
• and
• based on said match, automatically overlaying said planar area of said high resolution image portion on said perspective corrected image portion when said value of magnification exceeds a predetermined magnification threshold value during an image zooming process for increasing a user input degree of magnification.

A method of capturing images for image overlay comprising the steps of:

• graphically rendering a wide angle interactive image;
• capturing at least one high resolution image portion with a camera for overlay in said graphically rendered image;
• dewarping the graphically rendered wide angle image portion to provide a perspective corrected image portion of the graphically rendered wide angle image using an image transformation process;
• receiving user input degree of magnification until a predetermined magnification threshold is exceeded;
• matching pixel values representing at least three different reference points in said perspective corrected image portion with pixel values representing at least three corresponding reference points in said high resolution image portion, said three different reference points defining a planar area to be overlaid;
• and
• based on said match, automatically overlaying said planar area of said high resolution image portion in the perspective corrected image portion during a process of increasing said user input degree of magnification of said perspective corrected image portion at said predetermined magnification threshold.

A method of capturing images for image overlay comprising the steps of:

• capturing a wide angle interactive image with a camera;
• graphically rendering at least one high resolution image portion for overlay in said wide angle interactive image;
• providing a perspective corrected image portion of the captured wide angle interactive image using an image transformation process;
• matching pixel values representing at least three different reference points in said perspective corrected image portion with at least three corresponding reference points in said graphically rendered high resolution image portion, said three different point defining a planar area to be overlaid;
• and
• based on said match, automatically overlaying said planar area of said graphically rendered high resolution image portion in the perspective corrected image portion at a predetermined degree of magnification when a predetermined magnification threshold is exceeded during a zooming process controlled by user input for increasing a degree of magnification.

A method for displaying a realistic immersive image sequence comprising the steps of:

• displaying a hemispherical interactive image;
• receiving user input zooming commands for zooming into a perspective corrected interactive image portion of said hemispherical interactive image, said zooming controlled by user input degree of magnification;
• predetermining locations of at least three different corner pixels in said perspective corrected interactive image portion;
• matching pixel values representing said corner pixels in said perspective corrected interactive image portion with pixel values representing corresponding corner pixels in a high resolution image portion, said three different corner pixels defining a planar area to be overlaid;
• based on said match, displaying an automatic overlay of said planar area of said perspective corrected interactive image portion with said planar area of said high resolution image portion when a predetermined magnification threshold is exceeded during the image zooming step such that the realistic immersive image sequence is displayed.