Robot apparatus, method and device for recognition of letters or characters, control program and recording medium

A letter/character recognition device comprising:

• storage means for speech recognition, as a dictionary for speech recognition, having stored therein the relationship of correspondence between a word and the pronunciation information thereof;
• word sound expression storage means, as a table for word sound expressions, having stored therein the relationship of correspondence between the word and the word reading expressing letters thereof;
• imaging means for photographing an object;
• image recognition means for extracting the predetermined patterns of images from the image photographed by said imaging means;
• sound collecting means for acquiring the surrounding sound;
• speech recognition means for recognizing the speech from the sound collected by said sound collecting means;
• reading information generating means for conferring plural word reading expressing letters, inferred from the predetermined patterns of images extracted by said image recognition means, based on said table for word sound expressions, and for generating the pronunciation information for each of the plural word reading expressing letters or characters thus conferred;
• and
• storage controlling means for comparing the pronunciation information generated by said reading information generating means to the speech information of the speech recognized by said speech recognition means and newly storing the closest information of pronunciation in said dictionary for speech recognition as being the pronunciation information corresponding to the pattern recognition result extracted by said image recognition means.

A letter/character recognition method comprising:

• an imaging step of imaging an object;
• an image recognition step of extracting predetermined patterns of images from an image photographed by said imaging step;
• a sound collecting step of collecting the surrounding sound;
• a speech recognition step of recognizing the speech from the sound acquired by said sound collecting step;
• a reading information generating step of conferring plural word reading expressing letters, inferred from the predetermined patterns of images extracted by said image recognition step, based on a table for word sound expressions, having stored therein the relationship of correspondence between a word and a sound expressing letter/character for said word, and for generating the pronunciation information for each of the plural word reading expressing letters or characters thus conferred;
• and
• a storage controlling step of comparing the pronunciation information generated by said reading information generating means to the speech information of the speech recognized by said speech recognition step and newly storing the closest information of pronunciation in said dictionary for speech recognition as being the pronunciation information corresponding to the pattern recognition result extracted by said image recognition step.

A recording medium having stored therein a control program for having a robot apparatus execute

• an imaging step of imaging an object;
• an image recognition step of extracting predetermined patterns of images from an image photographed by said imaging step;
• a sound collecting step of collecting the surrounding sound;
• a speech recognition step of recognizing the speech from the sound acquired by said sound collecting step;
• an pronunciation information generating step of conferring plural word reading expressing letters, inferred from the predetermined patterns of images extracted by said image recognition step, based on a table for word sound expressions, having stored therein the relationship of correspondence between a word and a sound expressing letter/character for said word, and for generating, for each of the plural word reading expressing letters or characters thus conferred, the pronunciation information;
• and
• a storage step of comparing the pronunciation information generated by said reading information generating means to the speech information of the speech recognized by said speech recognition step and newly storing the closest information of pronunciation in said dictionary for speech recognition as being the pronunciation information corresponding to the pattern recognition result extracted by said image recognition step.

A recording medium having recorded therein a control program for having a robot apparatus execute

• an imaging step of imaging an object;
• an image recognition step of extracting predetermined patterns of images from an image photographed by said imaging step;
• a sound collecting step of collecting the surrounding sound;
• a speech recognition step of recognizing the speech from the sound acquired by said sound collecting step;
• an pronunciation information generating step of conferring plural word reading expressing letters, inferred from the predetermined patterns of images extracted by said image recognition step, based on a table for word sound expressions, having stored therein the relationship of correspondence between a word and a sound expressing letter/character for said word, and for generating the pronunciation information for each of the plural word reading expressing letters or characters thus conferred;
• and
• a storage step of comparing the pronunciation information generated by said pronunciation information generating step to the speech information of the speech recognized by said speech recognition step and newly storing the closest information of pronunciation in said dictionary for speech recognition as being the pronunciation information corresponding to the pattern recognition result extracted by said image recognition step.