Back to results
US 9,024,876 B2Granted
Absolute and relative positioning sensor fusion in an interactive display system
AssigneeInterphase Corporation
InventorsYoram Solomon, Branislav Kisacanin
Life of this patent
Estimating the term…
Abstract
An interactive display system including a wireless pointing device, and positioning circuitry capable of determining absolute and relative positions of the display at which the pointing device is aimed. An error value between the absolute position and an estimated or actual relative position at the point in time of the absolute position is determined, and a compensation factor is determined from this error value that is applied to subsequent relative positioning results.
Claims54
A method of operating a computer system including a handheld human interface device, comprising:
- displaying image frame data on a display;
- capturing, at a handheld human interface device, image data representative of at least a portion of the display;
- determining an absolute location at the display at which the device is pointing at an image capture time;
- evaluating motion of the handheld device based on measurements acquired from one or more sensors in the device;
- for each of a plurality of sample times including a first sample time earlier in time than the image capture time and a second sample time later in time than the image capture time, determining a relative location at the display at which the device is pointing based on the evaluated motion of the device relative to a prior sample time;
- calculating, from the relative locations determined at the first and second sample times, a calculated relative location at the image capture time;
- determining an error value between the absolute location at the image capture time and the calculated relative location;
- and
- then adjusting one or more relative locations by a compensation value corresponding to the error value.
6 more elements
A method of operating a computer system including a handheld human interface device, comprising:
- displaying image frame data on a display;
- capturing, at a handheld human interface device, image data representative of at least a portion of the display including a positioning target pattern, over a sequence of frames, the positioning target pattern corresponding to complementary intensity variances at one or more selected pixel locations of consecutive frames in the sequence;
- determining an absolute location at the display at which the device is pointing at an image capture time;
- evaluating motion of the handheld device based on measurements acquired from one or more sensors in the device;
- for each of a plurality of sample times, determining a relative location at the display by:
- capturing image data including the positioning target pattern at sample times during display of a first frame and a third frame in the sequence, wherein the first, a second, and the third frames are consecutive in time in the sequence;
- and
- detecting a change in position in the positioning target pattern by comparing the locations of the positioning target pattern in the captured image data from the first and third frames;
- determining an error value between the absolute location at the image capture time and a calculated relative location at the image capture time calculated from the relative locations at first and second sample times;
- and
- then adjusting one or more relative locations by a compensation value corresponding to the error value.
8 more elements
An interactive display system, comprising:
- a computer for generating display image data to be displayed on a display;
- graphics output circuitry for generating graphics output signals corresponding to the display image data in a format suitable for display;
- a pointing device, comprising:
- a hand-held housing;
- an image sensor disposed in the housing;
- means for sensing motion of the handheld device;
- and
- image capture circuitry for capturing image data obtained by the image sensor;
- and
- positioning circuitry for determining a location at the display at which the pointing device is aimed by performing a plurality of operations comprising:
- determining an absolute location at the display at which the device is pointing at an image capture time;
- evaluating motion of the handheld device based on measurements acquired from the motion sensing means;
- for each of a plurality of sample times including a first sample time earlier in time than the image capture time and a second sample time later in time than the image capture time, determining a relative location at the display at which the device is pointing based on the evaluated motion of the device relative to a prior sample time;
- calculating, from the relative locations determined at the first and second sample times, a calculated relative location at the image capture time;
- determining an error value between the absolute location at the image capture time and the calculated relative location;
- and
- then adjusting one or more relative locations by a compensation value corresponding to the error value.
14 more elements
An interactive display system comprising:
- a computer for generating display image data to be displayed on a display;
- graphics output circuitry for generating graphics output signals corresponding to the display image data in a format suitable for display;
- a pointing device, comprising:
- a hand-held housing;
- an image sensor disposed in the housing;
- means for sensing motion of the handheld device;
- and
- image capture circuitry for capturing image data obtained by the image sensor representative of at least a portion of the display including a positioning target pattern, over a sequence of frames, the positioning target pattern corresponding to complementary intensity variances at one or more selected pixel locations of consecutive frames in the sequence;
- and
- positioning circuitry for determining a location at the display at which the pointing device is aimed by performing a plurality of operations comprising:
- determining an absolute location at the display at which the device is pointing at an image capture time;
- evaluating motion of the handheld device based on measurements acquired from the motion sensing means;
- for each of a plurality of sample times, determining a relative location at the display at which the device is pointing based on the evaluated motion of the device relative to a prior sample time, from captured image data including the positioning target pattern at sample times during display of a first frame and a third frame in the sequence, wherein the first, a second, and the third frames are consecutive in time in the sequence, by detecting a change in position in the positioning target pattern by comparing the locations of the positioning target pattern in the captured image data from the first and third frames;
- determining an error value between the absolute location at the image capture time and a calculated relative location at the image capture time calculated from the relative locations at first and second sample times;
- and
- then adjusting one or more relative locations by a compensation value corresponding to the error value.
13 more elements
Dependencies are inferred from the claim text.