Toy system for augmented reality

A toy system, the toy system comprising a data processing system and one or more toys, the plurality of toys including at least one reference toy having a visual appearance recognisable by the data processing system in a captured image of a real-world scene including said at least one reference toy, the data processing system comprising an image capturing device, a processing unit, and a display, wherein the data processing system has stored thereon information associated with the at least one reference toy, the information including at least one predetermined reference position defined relative to the at least one reference toy;

• and wherein the data processing system is configured to:
• capture a sequence of images of a real-world scene, the real-world scene including said at least one reference toy;
• process the captured images to detect and recognize said at least one reference toy within the real-world scene;
• retrieve the at least one predetermined reference position from the stored information associated with the recognized at least on reference toy;
• process the captured images to identify the at least one predetermined reference position within at least a first image of the sequence of captured images;
• generate and render computer-generated perceptual information prompting the user to manipulate the real-world scene at the identified at least one reference position;
• process the captured images to detect a user manipulation of the real-world scene at the identified at least one predetermined reference position;
• responsive to detecting the user manipulation, generate and render computer-generated perceptual information associated with the detected user manipulation;
• wherein the data processing system is configured to selectively look for a detectable user manipulation at the reference position during a limited time window after prompting the user to perform the manipulation at the reference position.