Text input method and apparatus based on virtual keyboard

A text input apparatus, comprising:

• a processing unit, configured to obtain key points of a first hand based on a first image comprising the first hand, wherein the processing unit is further configured to project a virtual keyboard on the first hand based on the key points of the first hand;
• and
• an image capture unit, configured to obtain a second image comprising the first hand and a finger of a second hand, wherein
• the processing unit is further configured to:
• analyze the second image using a preset neural network model trained to detect a contact state or a pressing state based on texture changes of the first hand, wherein the neural network model was trained with a training set comprising a plurality of first-type training images and a plurality of second-type training images, wherein the plurality of first-type training images comprise the finger of the second hand in a contact state or a pressing state and comprise labels of texture features of the first hand, and the plurality of second-type training images comprise the finger of the second hand separated from the first hand;
• in response to analyzing the second image using the preset neural network model, identify, by the neural network model based on detecting a change in texture on the first hand, the contact state or the pressing state between the finger of the second hand and the first hand;
• and
• when the finger of the second hand and the first hand are in the contact state or the pressing state, determine input text based on a location at which the finger of the second hand touches or presses the first hand.

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

• obtaining key points of a first hand based on a first image comprising the first hand;
• projecting a virtual keyboard on the first hand based on the key points of the first hand;
• obtaining a second image comprising the first hand and a finger of a second hand;
• analyzing the second image using a preset neural network model trained to detect a contact state or a pressing state based on texture changes of the first hand, wherein the neural network model was trained with a training set comprising a plurality of first-type training images and a plurality of second-type training images, wherein the plurality of first-type training images comprise the finger of the second hand in a contact state or a pressing state and comprise labels of texture features of the first hand, and the plurality of second-type training images comprise the finger of the second hand separated from the first hand;
• in response to analyzing the second image using the preset neural network model, identifying, by the neural network model based on detecting a change in texture on the first hand, the contact state or the pressing state between the finger of the second hand and the first hand;
• and
• when the finger of the second hand and the first hand are in the contact state or the pressing state, determining input text based on a location at which the finger of the second hand touches or presses the first hand.