Method and apparatus for processing voice commands based on optical character recognition (ocr)

An apparatus for processing voice commands of a user based on a display screen displayed to the user, the apparatus comprising:

• at least one memory storing computer-readable instructions;
• and
• at least one processor,
• wherein the at least one processor is configured to execute the computer-readable instructions to:
• perform optical character recognition (OCR) on the display screen to obtain texts displayed on the display screen and respective positions of the texts,
• generate screen keyword information comprising words and sentences included in utterances of the user, based on the texts and the respective positions of the texts,
• determine a screen domain based on the screen keyword information, wherein the screen domain is a command domain corresponding to the display screen, from among a plurality of command domains,
• generate commands corresponding to the utterances of the user based on the screen keyword information and the screen domain, and
• process voice commands of the user based on the generated commands.

A non-transitory computer-readable medium or media storing computer-readable instructions that, when executed by at least one processor, cause the at least one processor to:

• perform OCR on a display screen to obtain texts displayed on the display screen to a user and respective positions of the texts;
• generate screen keyword information comprising words and sentences included in utterances of the user, based on the texts and the respective positions of the texts;
• determine a screen domain based on the screen keyword information, wherein the screen domain is a command domain corresponding to the display screen, from among a plurality of command domains;
• generate commands corresponding to the utterances of the user based on the screen keyword information and the screen domain;
• and
• process voice commands of the user based on the generated commands.