System and method of real time insertions into video using adaptive occlusion with a synthetic reference image

A system for merging an inserted image with a current image in a live broadcast of an event while compensating for occluding objects in the foreground of the current image, the system comprising:

• camera means for obtaining a reference image of a scene, said scene including an intended insertion region;
• processor means for processing the reference image to obtain a synthetic reference image;
• warping means for warping the synthetic reference image into the current image using warp parameters which transform the synthetic reference image to the same size and perspective as the current image;
• updating means for updating the color and intensity of the warped, synthetic reference image to reflect camera and lighting conditions in the current image;
• comparator means for creating a difference image by subtracting the updated, warped, synthetic reference image from the current image;
• masking means for creating an occlusion mask from said difference image for deciding whether to broadcast the pixels of the current image or the pixels of the inserted image;
• and
• mixing means for mixing the inserted image and the current image according to the occlusion mask to obtain a broadcast image which seamlessly blends the inserted image into a portion of the current image without obscuring the action of the event.