Method and system for enhancing virtual stage experience

An apparatus for augmenting visual images of an audio-visual entertainment system comprising:

• (a) one or a plurality of means for capturing facial images from video input image sequences of a user or a plurality of users,
• (b) means for displaying output,
• (c) means for enhancing said facial images of said user or said plurality of users from said video input image sequences by superimposing virtual object images to said facial images,
• (d) means for processing dynamically changing virtual background images according to body movements of said user or said plurality of users,
• (e) means for simulating a virtual stage environment image by composing the enhanced facial and body image of said user or said plurality of users, virtual stage images, and virtual objects images, and
• (f) means for handling interaction between said user or said plurality of users and said audio-visual entertainment system,
• (g) a sound system, and
• (h) a microphone,
• whereby the means for enhancing facial images processes the facial image enhancement at the level of local facial features on said facial images of said user or said plurality of users, and
• whereby examples of the facial features can be eyes, nose, and mouth of said user or said plurality of users.

A method for augmenting images on a means for displaying output of an audio-visual entertainment system, comprising the following steps of:

• (a) capturing a plurality of images for a user or a plurality of users with a single or a plurality of means for capturing images,
• (b) processing a single image or a plurality of images from the captured plurality of images in order to obtain facial features and body movement information of said user or said plurality of users,
• (c) processing selection by said user or said plurality of users for virtual object images on a means for displaying output,
• (d) augmenting facial feature images of said user or said plurality of users with the selected virtual object images,
• (e) simulating a virtual stage environment image, and
• (f) displaying the augmented facial images with said facial feature images of said user or said plurality of users and the simulated virtual stage environment image on said means for displaying output,
• whereby the step for augmenting facial feature images is processed at the level of local facial features on face images of said user or said plurality of users,
• whereby examples of the local facial features can be eyes, nose, and mouth of said user or said plurality of users, and
• whereby the step for augmenting facial feature images of said user or said plurality of users with the selected virtual object images is processed automatically, dynamically, and in real-time.