Loading patent…

Back to results

US 5,704,836 AGranted

Motion-based command generation technology

AssigneePerception Systems, Inc.

InventorsJohn Mark Norton, Bela Laszlo Brozsek

Life of this patent

Estimating the term…

Download PDF Find prior artPrior art Similar

Abstract Drawings Claims Description

Abstract

A command system optically monitors the movement of a subject and provides command signals to a computer system to control a graphic of a graphical user interface displayed on a monitor such as an animated character in a video game. The command system has an optical detector unit which continuously scans a subject frame in which the subject is positioned. The subject frame is divided into sub-regions around the subject in which movement of the subject is detected, for example, an arm moving. A command unit receives and processes frame data from the optical detector unit. The command unit determines the velocity of any movement detected in any of the sub-regions and generates an appropriate command signal based on the velocity of the movement. The command unit may also use the direction of the movement in generating command signals. The command signals are then provided to the computer system to move and control the graphical user interface accordingly.

Claims28

Dependent claims

Claim 1A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface · 395 words1 dependent Claim 3A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface · 461 words6 dependent Claim 10A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface · 537 wordsNo dependents Claim 11A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface · 437 wordsNo dependents Claim 12A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface · 400 words2 dependent Claim 15A method for providing a command signal indicative of movement of a subject · 255 words4 dependent Claim 20A method for providing a command signal indicative of movement of a subject · 350 wordsNo dependents Claim 21A method for providing a command signal indicative of movement of a subject · 240 words1 dependent Claim 22A method for providing a command signal indicative of movement of a subject · 214 words1 dependent Claim 24A method for controlling a character displayed on a video display unit · 73 words1 dependent Claim 27A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface · 316 wordsNo dependents Claim 28A method for providing a command signal indicative of movement of a subject · 229 wordsNo dependents

A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface, the command system comprising:

a) an optical detector unit having an aperture through which light from a field of view is receivable, and an output for frame data indicative of the light received from the field of view the subject being positionable in the field of view, the frame data defining a subject frame, the output providing the frame data on a subject frame-by-subject frame basis;
and
b) a command unit having an input in communication with the output of the optical detector unit for receiving the frame data on the subject frame-by-subject frame basis, and an output for providing a command signal to control the graphic user interface the command unit including processing circuitry in communication with the input and the output of the command unit for:
(1) determining position of the subject in a previous subject frame from the frame data of the previous subject frame, and position of the subject in a present subject frame from the frame data of the present subject frame;
(2) calculating velocity of the subject based upon the position of the subject in the previous subject frame and upon the position of the subject in the present subject frame, the velocity being a vector having a magnitude and a direction;
(3) generating a command signal based upon the velocity of the subject;
(4) determining whether the subject is positioned in a proper position within the field of view, the proper position being defined as a position where the subject is substantially centered within the field of view;
(5) determining which direction the subject needs to move to be in the proper position in the field of view;
and
(6) generating a positioning signal based upon the direction the subject needs to move to be in the proper position, with the output of the command unit providing the positioning signal;
and
c) a subject positioning unit having a input in communication with the output of the command unit for receiving the positioning signal, and a subject interface including an array of light-emitting diodes (LEDs) configured such that when the LEDs illuminate based upon the positioning signal, the LEDs indicate to the subject which direction the subject needs to move to be in the proper position.

9 more elements

A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface, the command system comprising:

an optical detector unit including an aperture through which from a field of view is receivable a charge-coupled device CCD having a plurality of pixels on which light received through the aperture is incident, and an output for frame data indicative of the light received from the field of view, the subject being positionable in the field of view, the frame data defining a subject frame, the output providing the frame data on a subject frame-by-subject frame basis, each of the pixels having a value indicative of the amount of light incident thereon, the value of the pixels changing from one subject frame to the next subject frame when the amount of light incident thereon changes, the frame data for a subject frame being comprised of the values of the pixels for the subject frame;
and
a command unit having an input in communication with the output of the optical detector unit for receiving the frame data on the subject frame-by-subject frame basis, and an output for providing a command signal to control the graphic user interface, the command unit including processing circuitry in communication with the input and the output of the command unit for:
determining position of the subject in a previous subject frame from the frame data of the previous subject frame, and position of the subject in a present subject frame from the frame data of the present subject frame;
calculating velocity of the subject based upon the position of the subject in the previous subject frame and upon the position of the subject in the present subject frame, the velocity being a vector having a magnitude and a direction;
and
generating a command signal based upon the velocity of the subject;
wherein the processing circuitry of the command unit further:
a) compares the value of each of the pixels of the present subject frame with the value of the corresponding pixel of the previous subject frame to obtain a difference value for each of the pixels;
b) assigns an ON difference bit to each of the pixels for which the difference value is greater than a predetermined threshold value;
c) determines an edge of the subject in the subject frame by locating at least two adjacent pixels to which an ON difference bit was assigned;
d) records the position of the adjacent pixels;
e) calculates the velocity of the subject based upon the position of the adjacent pixels in the previous subject frame and upon the position of the adjacent pixels in the present subject frame;
and
f) generates a command signal based upon the velocity of the subject.

12 more elements

A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface, the command system comprising:

an optical detector unit including aperture through which from a field of view is receivable a charge-coupled device CCD having a plurality of pixels on which light received through the aperture is incident, and an output for providing frame data indicative of the light received from the field of view, the subject being positionable in the field of view, the frame data defining a subject frame, the output providing the frame data on a subject frame-by-subject frame basis, each of the pixels having a value indicative of the amount of light incident thereon, the value of the pixels changing from one subject frame to the next subject frame when the amount of light incident thereon changes, the frame data for a subject frame being comprised of the values of the pixels for the subject frame;
and
a command unit having an input in communication with the output of the optical detector unit for receiving the frame data on the subject frame-by-subject frame basis, and an output for providing a command signal to control the graphic user interface, the command unit including processing circuitry in communication with the input and the output of the command unit for:
determining position of the subject in a previous subject frame from the frame data of the previous subject frame, and position of the subject in a present subject frame from the frame data of the present subject frame;
calculating velocity of the subject based upon the position of the subject in the previous subject frame and upon the position of the subject in the present subject frame, the velocity being a vector having a magnitude and a direction;
and
generating a command signal based upon the velocity of the subject;
wherein the processing circuitry of the command unit further:
a) compares the value of each of the pixels of the present subject frame with the value of the corresponding pixel of the previous subject frame to obtain a difference value for each of the pixels;
b) assigns a difference bit to each of the pixels, the difference bit being ON when the difference value is greater than a predetermined threshold value or OFF when the difference value is less than the predetermined threshold value;
c) combines into bytes the difference bits from the pixels, each of the bytes containing the difference bits from eight adjacent pixels;
d) determines whether the number of difference bits being ON in each of the bytes exceeds a predetermined number;
e) assigns a flag to each of the bytes for which the number of difference bits being ON exceeds the predetermined number;
f) determines an edge of the subject in the subject frame by locating at least two bytes to which a flag was assigned;
g) records the position of the adjacent bytes;
h) calculates the velocity of the subject based upon the position of the adjacent bytes in the previous subject frame and upon the position of the adjacent bytes in the present subject frame;
and
i) generates a command signal based upon the velocity of the subject.

15 more elements

A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface, the command system comprising:

an optical detector unit including an aperture through which light from a field of view is receivable, a charge-coupled device (CCD) having a plurality of pixels on which light received through the aperture is incident, and an output for providing frame data indicative of the light received from the field of view, the subject being positionable in the field of view, the frame data defining a subject frame, the output providing the frame data on a subject frame-by-subject frame basis, each of the pixels having a value indicative of the amount of light incident thereon, the value of the pixels changing from one subject frame to the next subject frame when the amount of light incident thereon changes, the frame data for a subject frame being comprised of the values of the pixels for the subject frame, the CCD having a sensitivity to the amount of light incident on the pixels, the sensitivity being adjustable;
and
a command unit having an input in communication with the output of the optical detector unit for receiving the frame data on the subject frame-by-subject frame basis, and an output for providing a command signal to control the graphic user interface, the command unit including processing circuitry in communication with the input and the output of the command unit for:
determining position of the subject in a previous subject frame from the frame data of the previous subject frame, and position of the subject in a present subject frame from the frame data of the present subject frame;
calculating velocity of the subject based upon the position of the subject in the previous subject frame and upon the position of the subject in the present subject frame, the velocity being a vector having a magnitude and a direction;
and
generating a command signal based upon the velocity of the subject;
wherein the processing circuitry of the command unit further:
a) compares the value of each of the pixels of the present subject frame with the value of the corresponding pixel of the previous subject frame to obtain a difference value for each of the pixels;
b) assigns a difference bit to each of the pixels for which the difference value is greater than a predetermined threshold value;
c) counts the number of pixels to which a difference bit was assigned;
and
d) adjusts the sensitivity of the CCD when the number of pixels to which a difference bit was assigned exceeds a predetermined number.

10 more elements

A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface, the command system comprising:

an optical detector unit having an aperture through which light from a field of view is receivable, and an output for providing frame data indicative of the light received from the field of view, the subject being positionable in the field of view, the frame data defining a subject frame, the output providing the frame data on a subject frame-by-subject frame basis;
and
a command unit having an input in communication with the output of the optical detector unit for receiving the frame data on the subject frame-by-subject frame basis, and an output for providing a command signal to control the graphic user interface, the command unit including processing circuitry in communication with the input and the output of the command unit for:
determining position of the subject in a previous subject frame from the frame data of the previous subject frame, and position of the subject in a present subject frame from the frame data of the present subject frame;
calculating velocity of the subject based upon the position of the subject in the previous subject frame and upon the position of the subject in the present subject frame, the velocity being a vector having a magnitude and a direction;
and
generating a command signal based upon the velocity of the subject;
wherein the processing circuitry of the command unit further:
a) defines a plurality of sub-regions within the subject frame;
b) determines the position of the subject in each of the sub-regions of the previous subject frame from the frame data of the previous subject frame, and the position of the subject in each of the sub-regions of the present subject frame from the frame data of the present subject frame;
c) calculates velocity of the subject in each of the sub-regions based upon the position of the subject in each of the sub-regions of the previous subject frame, and upon the position of the subject in the sub-regions of the present subject frame corresponding to each of the sub-regions of the previous subject frame;
and
d) generates a command signal based upon the velocity of the subject in each of the sub-regions, the number of command signals generated being up to the number of sub-regions defined within the subject frame.

10 more elements

A method for providing a command signal indicative of movement of a subject comprising the steps of:

receiving light from a field of view in which a subject is positionable with an array of pixels, each of the pixels having a value indicative of the amount of light incident thereon;
generating sets of data from the values of the pixels, each of the sets of data being indicative of light received from the field of view at an instant of time different from the other sets of data;
calculating velocity of the subject from the sets of data, the velocity being a vector having a magnitude and a direction, with the steps of:
a) comparing the value of each of the pixels from a present set of data with the value of the corresponding pixel from a previous set of data to obtain a difference value for each of the pixels;
b) assigning an ON difference bit to each of the pixels for which the difference value is greater than a predetermined threshold value;
c) determining an edge of the subject in the field of view by locating at least two adjacent pixels to which an ON difference bit was assigned;
d) recording the position of the adjacent pixels;
and
e) calculating velocity of the subject based upon the position of the adjacent pixels from the previous set of data and upon the position of the pixels in the present set of data;
and
generating a command signal based on the velocity of the subject.

8 more elements

A method for providing a command signal indicative of movement of a subject comprising the steps of:

receiving light from a field of view in which a subject is positionable with an array of pixels, each of the pixels having a value indicative of the amount of light incident thereon;
generating sets of data from the values of the pixels, each of the sets of data being indicative of light received from the field of view at an instant of time different from the other sets of data;
calculating velocity of the subject from the sets of data, the velocity being a vector having a magnitude and a direction, with the steps of:
a) comparing the value of each of the pixels from a present set of data with the value of the corresponding pixel from a previous set of data to obtain a difference value for each of the pixels;
b) assigning an ON difference bit to each of the pixels, the difference bit being ON when the difference value is greater than a predetermined threshold value or OFF when the difference value is less than the predetermined threshold value;
c) combining into difference bytes the difference bits of the pixels, each of the difference bytes containing the difference bits from eight adjacent pixels;
d) determining whether the number of difference bits being ON in each of the difference bytes exceeds a predetermined number;
e) assigning a flag to each of the difference bytes for which the number of difference bits being ON exceeds the predetermined number;
f) determining an edge of the subject in the field of view by locating at least two difference bytes to which a flag was assigned;
and
g) recording the position of the adjacent difference bytes;
wherein said step of calculating velocity comprises the step of calculating velocity of the subject based upon the position of the adjacent difference bytes from the previous set of data and upon the position of the adjacent difference bytes from the present set of data;
and
generating a command signal based on the velocity of the subject.

11 more elements

A method for providing a command signal indicative of movement of a subject comprising the steps of:

receiving light from a field of view in which a subject is positionable with a charge-coupled device (CCD) having an array of pixels, each of the pixels having a value indicative of the amount of light incident thereon, the CCD having an adjustable sensitivity;
and
adjusting the sensitivity of the CCD with the steps of:
a) comparing the value of each of the pixels from a present set of data with the value of the corresponding pixels from a previous set of data to obtain a difference value for each of the pixels;
b) assigning a difference bit to each of the pixels for which the difference value is greater than a predetermined threshold value;
c) counting the number of pixels to which a difference bit was assigned;
and
d) adjusting the sensitivity of the CCD when the number of pixels to which a difference bit was assigned exceeds a predetermined number;
generating sets of data from the values of the pixels, each of the sets of data being indicative of light received from the field of view at an instant of time different from the other sets of data;
calculating velocity of the subject from the sets of data, the velocity being a vector having a magnitude and a direction;
and
generating a command signal based on the velocity of the subject.

9 more elements

A method for providing a command signal indicative of movement of a subject comprising the steps of:

receiving light from a field of view in which a subject is positionable;
generating sets of data from the received light, each of the sets of data being indicative of the amount of light received from the field of view at an instant of time different from the other sets of data and defining a subject frame for that instant of time;
defining a plurality of sub-regions within the subject frames;
determining a previous position of the subject in each of the sub-regions of the subject frame from a previous set of data and a present position of the subject in each of the sub-regions of the subject frame from a present set of data;
calculating velocity of the subject in each of the sub-regions based upon the previous position of the subject and the present position of the subject, the velocity being a vector having a magnitude and a direction;
and
generating a command signal based upon the velocity of the subject in each of the sub-regions, the number of command signals being up to the number of sub-regions defined within the subject frame, each of the command signals being different from the other command signals.

4 more elements

A method for controlling a character displayed on a video display unit comprising the steps of:

a) positioning a subject within a field of view of an optical detector unit;
b) defining a plurality of sub-regions around the subject within the field of view;
c) detecting movement of the subject within the sub-regions;
d) generating a command signal based on the detected movement;
and
e) animating the character based upon the command signal.

3 more elements

A command system for monitoring movement of a subject and for controlling in accordance with movement of the subject a graphical user interface, the command system comprising:

a) an optical detector unit having an aperture through which light from a field of view is receivable, and an output for providing frame data indicative of the light received from the field of view, the subject being positionable in the field of view, the frame data defining a subject frame, the output providing the frame data on a subject frame-by-subject frame basis;
and
b) a command unit having an input in communication with the output of the optical detector unit for receiving the frame data on the subject frame-by-subject frame basis, and an output for providing a command signal to control the graphic user interface, the command unit including processing circuitry in communication with the input and the output of the command unit for:
(1) determining position of the subject in a previous subject frame from the frame data of the previous subject frame, and position of the subject in a present subject frame from the frame data of the present subject frame;
(2) calculating velocity of the subject as a vector having a magnitude and a direction, the velocity being determined based upon the position of the subject in the previous subject frame and upon the position of the subject in the present subject frame, the direction of the velocity having a horizontal component and a vertical component;
(3) calculating the magnitude of the horizontal component and the magnitude of the vertical component of the velocity of the subject;
(4) comparing the magnitude of the velocity of the horizontal component with the magnitude of the velocity of the vertical component to determine which of the two magnitudes of velocity is greater;
and
(5) generating a command signal based upon the greater of the two magnitudes of velocity.

6 more elements

A method for providing a command signal indicative of movement of a subject comprising the steps of:

a) receiving light from a field of view in which a subject is positionable;
b) generating sets of data from the received light, each of the sets of data being indicative of light received from the field of view at an instant of time different from the other sets of data;
c) determining a previous position of the subject in the field of view from a previous set of data;
d) determining a present position of the subject in the field of view from a present set of data;
e) determining velocity of the subject from the previous position of the subject and the present position of the subject, the velocity being a vector having a magnitude and a direction with a horizontal component and a vertical component;
f) calculating the magnitude of the velocity of the horizontal component and the magnitude of the velocity of the vertical component of the subject based upon the previous position and the present position of the subject;
g) comparing the magnitude of the velocity of the horizontal component with the magnitude of the velocity of the vertical component to determine which of the two magnitudes of velocity is greater;
and
f) generating a command signal based upon the greater of the two magnitudes of velocity.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application: 08/604,186Filed 21 Feb 1996 · continuation in part
Priority: GB 950591623 Mar 1995
Granted: 6 Jan 1998
Term ends: Not estimatedThe source records don’t support an estimate.

Classification21 codes

G06F 3/011
Input arrangements or combined input and output arrangements for interaction between user and computer: arrangements for interaction with the…
A63F 13/213
Characterised by their sensors, purposes or types: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
A63F 13/215
Characterised by their sensors, purposes or types: comprising means for detecting acoustic signals, e.g. using a microphone
A63F 13/428
By mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual…:…

Family

Serial554,473Parent of this continuation-in-part · filed 7 Nov 1995
US 5,704,836 AThis patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

USPTO full text and PDF
Patent Center file history
USPTO assignment records
Espacenet
Google Patents