Device and method for estimating a mental decision

An apparatus for estimating a mental decision of a viewer for selecting a task, comprising ocular means for generating an ocular line of sight and pupil size of the viewer wherein;

• said ocular means include:
• measuring means for measuring an ocular viewing direction and pupil sizes of the viewer;
• sensor means for measuring a head position and orientation of the viewer in a workspace;
• and
• digital processor means coupled with said measuring means and said sensor means, for computing and generating the ocular line of sight and pupil size data of the viewer in the workspace;
• biosignal processor means for determining electroencephalogramic potential of the viewer;
• said biosignal means include:
• first electrode means for sensing scalp surface electroencephalogramic potentials from the cerebral scalp of the viewer;
• and
• first amplifying means for amplifying the sensed scalp surface electroencaphalogramic potentials;
• second electrode means including a plurality of electrode sensors for generating extracerebral potential from body portions of the view;
• and
• third amplifying means including a plurality of amplifiers for respectively amplifying the generated extracerebral potentials;
• the extracerebral potentials including electrooculogramic potentials generated by ocular processes, electromygramic potentials generated from facial throat, back, shoulder, and arm muscles of the viewer, and electrocardiogramic potentials generated from the heart of the viewer
• said biosignal processor means further include:
• an analog-to-digital (A/D)converter, connected to said first and second amplifying means, for generating digital output signals of the scalp surface electroencephalogramic potentials and the extracerebral potentials;
• and
• a digital signal processor receiving the digital output signal from said A/D converter,
• said digital signal processor having first filtering means for filtering measurement noise from the digital output signals and second filtering means connected to said first filtering means for filtering attenuation from the digital output signals;
• and
• including summing means for summing the filtered digital output signals, so as to remove the electrooculogramic, electromygramic and electrocardiogramic potential from the scalp surface electroencephalogramic potentials;
• computer means coupled to said ocular means and said biosignal processor means, for computing an estimate of the mental decision from the ocular line of sight and pupil size data and the electroencephalogramic potentials
• said computer means include:
• a first computer routine which stores the ocular line of sight and pupil size data from said ocular means;
• and
• a second computer routine which determines if a sight state of the viewer is saccadic or fixation, using the ocular line of sight and pupil size data;
• wherein
• when the fixation is determined, said computer routine isolates the start and end of the fixation, computes s fixation duration, extracts the start and end pupil sizes, and determines a fixation end state and a gaze fixation count;
• wherein
• when the fixation is determined, said second computer routine determines values of fixation parameters and stores the fixation parameter values in memory;
• a third computer means which stores the electroencephalogramic potential determined by said biosignal processor means;
• and
• a fourth computer routine which extracts a single event transient evoked potential for the fixation, from the starting and ending points of the fixation and using the stored electroencephalogramic potentials:
• and
• and computes a parametric representation of the single transient evoked potential using filter modeling techniques;
• a fifth computer routine which generates workspace coordinate data of gaze point of regard of the viewer using the stored fixation parameter values;
• and
• a sixth computer routine which computes an estimate of the mental decision from an expert system employing an artifice neural network;
• said artificial neural network representing a mental decision process and having eye gaze fixation properties and the parametric representation of the transit evoked potential as inputs.