Motion compensation in an interactive display system

An interactive display system, comprising:

• a computer for generating display image data to be displayed on a display;
• graphics output circuitry for generating graphics output signals corresponding to the display image data in a format suitable for display;
• a pointing device, comprising:
• a hand-held housing;
• an image sensor disposed in the housing;
• one or more motion sensors;
• and
• image capture circuitry for capturing image data obtained by the image sensor;
• and
• positioning circuitry for determining a location at the display at which the pointing device is aimed by performing a plurality of operations comprising:
• generating visual payload image frame data;
• combining, with the visual payload image frame data, at least one positioning target pattern corresponding to complementary intensity variances at one or more selected pixel locations of the visual payload image frame data, applied in first and second consecutive frames;
• displaying the combined visual payload image frame data and positioning target pattern on a display;
• capturing, at the handheld human interface device, image data representative of at least a portion of the display including the positioning target, from each of the first and second frames;
• detecting relative motion of the device between the times at which image data is captured from the first and second frames, based on measurements acquired from one or more motion sensors in the device;
• aligning the image data of the first and second frames based on the detected relative motion;
• subtracting the aligned image data from the first and second frames from one another to recover the positioning target pattern as viewed at the device;
• and
• determining a location of the display at which the device is pointing at the time of one of the first and second frames, responsive to the recovered positioning target pattern.