Method of and apparatus for multi-modal information presentation to computer users with dyslexia, reading disabilities or visual impairment

In a computer system, a method of displaying text-based information so to provide text-based information reading and comprehension assistance to a user, said method comprising the computer implemented steps of:

• accessing a source of text-based data;
• analyzing the entire said source of text-based data and deriving context information of said entire source;
• automatically adjusting visual display and audio parameters based on said context information of said entire source;
• displaying text-based data in a first text window displayed on a display screen of said computer system wherein said text-based data is displayed with a pre-selected font size;
• displaying said text-based data in a second text window displayed on said display screen of said computer system, wherein said text-based data is displayed with a magnified font size compared to said pre-selected font size, and wherein said first text window and said second text window are displayed simultaneously;
• sequentially highlighting a word of said text-based data in said second text window one word at a time;
• synchronous with said step of highlighting, generating synthesized speech signals representative of said highlighted word of said text-based data and rendering said synthesized speech signals audible such that text-based information and corresponding audible information can be perceived simultaneously by said user;
• providing a third window of said display screen for displaying images;
• synchronous with said step of highlighting, displaying in said third window an image contextually relevant to said highlighted word such that said image is perceived by said user in conjunction with hearing said synthesized speech signals and viewing said section of said text-based data;
• and
• generating audible sound effects synchronously with said step of highlighting, wherein said audible sound effects are contextually relevant to said section of said text-based data such that said audible sound effects can be perceived in conjunction with hearing said speech signals and seeing said highlighted word of said text-based data.

A computer-usable medium having computer-readable program code embodied therein for causing a computer to perform a method of displaying text-based information to a user with dyslexia, reading disabilities or vision impairment so to provide text-based information reading and comprehension assistance to said user, said method comprising the steps of:

• accessing a source of text-based data;
• analyzing the entire said source of text-based data and deriving context information of said entire source;
• automatically adjusting visual display and audio parameters based on said context information of said entire source;
• displaying text-based data in a first text window displayed on a display screen of said computer system, wherein said text-based data is displayed with a pre-selected font size;
• displaying said text-based data in a second text window displayed on said display screen of said computer system, wherein said text-based data is displayed with a magnified font size compared to said pre-selected font size, and wherein said first text window and said second text window are displayed simultaneously;
• sequentially highlighting a word of said text-based data in said second text window one word at a time;
• synchronous with said step of highlighting, generating synthesized speech signals representative of said highlighted word of said text-based data and rendering said synthesized speech signals audible such that text-based information and corresponding audible information can be perceived simultaneously by said user;
• providing a third window of said display screen for displaying images;
• synchronous with said step of highlighting, displaying in said third window an image contextually relevant to said highlighted word such that said image can be perceived in conjunction with hearing said speech signals and seeing said section of text-based data;
• and
• generating audible sound effects synchronously with said step of highlighting, wherein said audible sound effects are contextually relevant to said section of said text-based data such that said audible sound effects can be perceived in conjunction with hearing said speech signals and seeing said highlighted word of said text-based data.

A computer-usable medium having computer-readable program code embodied therein for causing a computer to perform a method of displaying text-based information to a user with dyslexia, reading disabilities or vision impairment so to provide text-based information reading and comprehension assistance to said user, said method comprising the steps of:

• accessing a source of text-based data;
• analyzing the entire said source of text-based data and deriving context information of said entire source;
• automatically adjusting visual display and audio parameters based on said context information of said entire source;
• displaying text-based data in a text window displayed on a display screen of said computer system, wherein said text-based data is displayed with a pre-selected font size;
• sequentially highlighting a word of said text-based data in said text window one word at a time;
• synchronous with said step of highlighting, generating synthesized speech signals representative of said highlighted word of said text-based data and rendering said synthesized speech signals audible such that text-based information and corresponding audible information can be perceived simultaneously by said user;
• providing a graphics window of said display screen for displaying images;
• synchronous with said step of highlighting, displaying in said graphics window an image contextually relevant to said highlighted word of said text-based data such that said image can be perceived by said user in conjunction with hearing said speech signals and seeing said highlighted word of text-based data;
• and
• generating audible sound effects synchronously with said step of highlighting, wherein said audible sound effects are contextually relevant to said highlighted word of said text-based data such that said audible sound effects can be perceived in conjunction with hearing said speech signals and seeing said highlighted word of said text-based data.