# US 2014/0152622 A1 · Information processing apparatus, information processing method, and computer readable storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0152622 A1 - Document ID: US20140152622A1 - Status: Published application - Kind: A1 - Application number: 14/013,986 - Filed: 2013-08-29 - Published: 2014-06-05 - Earliest claimed: 2012-11-30 - Assignees: Kabushiki Kaisha Toshiba - Applicants: Kabushiki Kaisha Toshiba - Inventors: Kentaro Takeda - Web page: https://patentgrep.com/patent/US20140152622A1 ## CPC - G06F 3/0426: {tracking fingers with respect to a virtual keyboard projected or printed on the surface (virtual keyboards on touch screens G06F3/04886)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An information processing apparatus includes an imaging device, a keyboard detector, a first input detector, and a display. The keyboard detector is configured to detect a virtual keyboard based on an image captured by the imaging device. The first input detector is configured to detect an input to the virtual keyboard based on the captured image. The display is configured to display information corresponding to the input detected by the first input detector. ## Claims ### Claim 1 An information processing apparatus comprising: an imaging module; a keyboard detector configured to detect a virtual keyboard based on an image captured by the imaging module; a first input detector configured to detect an input to the virtual keyboard based on the captured image; and a display configured to display information corresponding to the input detected by the first input detector. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the virtual keyboard includes a keyboard image that is printed on a medium. ### Claim 3 (depends on 2) The apparatus of claim 2, wherein an identification mark for identification of the virtual keyboard is printed on the medium, the apparatus further comprising: a storage configured to store information indicating the identification mark, wherein the keyboard detector is configured to detect the virtual keyboard by comparing the captured image with the stored information indicating the identification mark. ### Claim 4 (depends on 3) The apparatus of claim 3, wherein: the identification mark indicates a type of the virtual keyboard, and the keyboard detector is configured to detect the type of the virtual keyboard by comparing the captured image with the stored information indicating the identification mark. ### Claim 5 (depends on 2) The apparatus of claim 2, further comprising: a storage configured to store a reference image of the virtual keyboard, wherein the keyboard detector is configured to detect the virtual keyboard by comparing the captured image with the reference image. ### Claim 6 (depends on 5) The apparatus of claim 5, wherein the storage is configured to store a plurality of reference images which are different from each other, and the keyboard detector is configured to detect a type of the virtual keyboard by comparing the captured image with the plurality of reference images. ### Claim 7 (depends on 1) The apparatus of claim 1, further comprising: a luminance adjustor configured to increase a luminance of the display when the keyboard detector has not detected the virtual keyboard. ### Claim 8 (depends on 7) The apparatus of claim 7, further comprising: a brightness detector configured to detect brightness around the imaging module, wherein the luminance adjustor increases the luminance of the display according to a detection result of the brightness detector when the keyboard detector has not detected the virtual keyboard. ### Claim 9 (depends on 7) The apparatus of claim 7, wherein the display displays information for prompting a user to print the virtual keyboard when the keyboard detector has not detected the virtual keyboard. ### Claim 10 (depends on 2) The apparatus of claim 2, wherein three or more boundary marks are printed on the medium along a boundary of an inputtable area of the virtual keyboard, the apparatus further comprising: a storage configured to store information indicating the boundary marks; and a non-inputtable state detector configured to detect as to whether or not the virtual keyboard is in a non-inputtable state, based on the captured image and the stored information indicating the boundary marks, wherein when the non-inputtable state detector detects that the virtual keyboard is in the non-inputtable state, the display displays information for prompting a user to correct a position of the virtual keyboard. ### Claim 11 (depends on 10) The apparatus of claim 10, further comprising: a table generator configured to generate a table indicating positions of plural respective keys of the virtual keyboard based on a detection result of the non-inputtable state detector, wherein the storage is configured to stores the generated table. ### Claim 12 (depends on 10) The apparatus of claim 10, further comprising: a mark movement detector configured to detect movements of any of the boundary marks; and a table updater configured to update the stored table based on a detection result of the mark movement detector. ### Claim 13 (depends on 10) The apparatus of claim 10, wherein: the first input detector includes a position detector configured to detect a position of a fingertip of a manipulator based on the captured image, and a fingertip movement detector configured to detect a movement of the fingertip based on the positions detected by the position detector, the first input detector is configured to detect a manipulated key based on the position of the fingertip and positions of plural respective keys of the virtual keyboard at a time when the fingertip movement detector detects the movement of the fingertip. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein the first input detector further includes a start detector configured to detect start of the input to the virtual keyboard, by detecting that the fingertip moves in a first direction toward the virtual keyboard. ### Claim 15 (depends on 13) The apparatus of claim 13, wherein the first input detector further includes an end detector configured to detect end of the input to the virtual keyboard by detecting that the fingertip moves in a second direction away from the virtual keyboard. ### Claim 16 (depends on 13) The apparatus of claim 13, wherein the imaging module includes a plurality of imaging devices, and the position detector detects the position of the fingertip based on a plurality of images captured by the plurality of imaging devices. ### Claim 17 (depends on 13) The apparatus of claim 13, further comprising: a sound detector configured to detect a sound, wherein the first input detector detects a manipulated key based on the position of the fingertip at a time when the fingertip movement detector detects that the fingertip moves and the sound detector detects the sound. ### Claim 18 (depends on 1) The apparatus of claim 1, further comprising: a touch pad detector configured to detect a virtual touch pad based on the captured image; and a second input detector configured to detect input to the virtual touch pad based on the captured image. ### Claim 19 An information processing method comprising: capturing an image; detecting a virtual keyboard based on the captured image; detecting an input to the virtual keyboard based on the captured image; and displaying information corresponding to the detected input. ### Claim 20 A computer readable storage medium storing a program that causes a processor to execute information processing, the information processing comprising: capturing an image; detecting a virtual keyboard based on the captured image; detecting an input to the virtual keyboard based on the captured image; and displaying information corresponding to the detected input.