US 6,982,697 B2 · System and process for selecting objects in a ubiquitous computing environment

Abstract

A system and process for selecting objects in an ubiquitous computing environment where various electronic devices are controlled by a computer via a network connection and the objects are selected by a user pointing to them with a wireless RF pointer. By a combination of electronic sensors onboard the pointer and external calibrated cameras, a host computer equipped with an RF transceiver decodes the orientation sensor values transmitted to it by the pointer and computes the orientation and 3D position of the pointer. This information, along with a model defining the locations of each object in the environment that is associated with a controllable electronic component, is used to determine what object a user is pointing at so as to select that object for further control actions.

Claims43

Dependent claims

A system for selecting an object within an environment by a user pointing to the object with a pointing device, comprising:

a pointing device comprising a radio frequency (RF) transceiver and orientation sensors, wherein the outputs of the sensors are periodically packaged as orientation messages and transmitted using the RF transceiver;
a base station comprising a RF transceiver which receives orientation messages transmitted by the pointing device;
a pair of imaging devices each of which is located so as to capture images of the environment from different viewpoints;
a computing device which is in communication with the base station and the imaging devices so as to receive orientation messages forwarded to it by the base station and images captured by the imaging devices, and which computes the orientation and location of the pointer from the received orientation message and captured images, and which also selects an object in the environment whenever the pointing device is pointed at it using the orientation and location of the pointing device.

1 more element

A computer-implemented process for selecting an object within an environment by a user pointing to the object with a pointing device, comprising using a computer to perform the following process actions:

inputting orientation sensor readings generated by orientation sensors of the pointing device and provided by the device in an orientation message;
deriving the orientation of the pointing device in relation to a predefined coordinate system of the environment from the inputted orientation sensor readings;
inputting digital video from a pair of video cameras each of which is located so as to capture images of the environment from different viewpoints;
ascertaining the location of the pointing device at a time substantially contemporaneous with the generation of the orientation message and in terms of the predefined coordinate system using the inputted digital video from the pair of video cameras;
using the orientation and location of the pointing device to determine whether the pointing device is being pointed at an object in the environment that is controllable by the computer;
and
whenever the pointing device is being pointed at a controllable object, selecting that object for future control actions.

4 more elements

Dependencies are inferred from the claim text.

System and process for selecting objects in a ubiquitous computing environment

Life of this patent

System and process for selecting objects in a ubiquitous computing environment

Life of this patent

Abstract

Claims43