Entertainment system providing dynamically augmented game surfaces for interactive fun and learning

A system for visually enhancing a game that includes a game structure and physical game objects that may be moved by a player relative to a game surface on the game structure, comprising:

• a projector projecting augmentation images onto the game surface;
• a tracking mechanism generating tracking data defining positions of the game objects relative to the game surface;
• and
• a controller processing the tracking data to determine first and second ones of the positions of the game objects and modifying the augmentation images in response to the positions of the game objects,
• wherein the controller further operates to generate a first augmentation content based on the first positions, to first operate the projector to project the first augmentation content on the game surface, to generate a second augmentation content based on the second positions, and to second operate the projector to project the second augmentation content on the game surface,
• wherein the tracking mechanism tracks movement of at least one of the game objects from a corresponding one of the first positions to a corresponding one of the second positions,
• wherein the controller further operates to generate third augmentation content providing animation for the one of the game objects based on the tracked movement, and
• wherein the augmentation images comprise a composite digital video stream including a base image mapped to the game surface and at least one game object image mapped to one of the game objects and a corresponding one of the positions, whereby the one game object image is projected onto the one of the game objects.