System and method for human computer interaction

A system for human computer interaction, wherein the system comprises:

• a projecting unit, a first image sensing unit, a second image sensing unit, an interface unit, an image processing unit, a projected interface processing unit, and a controlling unit, in which,
• the projecting unit is for receiving human computer interaction interface information data output by the interface unit, and projecting and outputting the data to a projecting plane;
• the first image sensing unit and the second image sensing unit is for collecting projected image information on an area of the projecting plane, and outputting to the image processing unit through the interface unit;
• the interface unit is coupled to the projecting unit, the first image sensing unit, the second image sensing unit, the image processing unit, the projected interface processing unit and the controlling unit, for synchronously controlling the first image sensing unit, the second image sensing unit and the projecting unit;
• the image processing unit is for receiving image data collected by the first image sensing unit and the second image sensing unit;
• receiving a human computer interaction interface image output by the projected interface processing unit, and extracting feature information in the human computer interaction interface image;
• calculating and obtaining system inherent internal parameters which comprise aberration distortion parameters, focus, pixel size and center point position on an optical lens of the first image sensing unit, the second image sensing unit and the projecting unit;
• storing the system inherent internal parameters on a non-volatile memory, and outputting the system inherent internal parameters to the interface unit;
• extracting corresponding feature information for the image data, and matching it with the feature information output by the projected interface processing unit;
• calculating and obtaining system inherent external parameters which comprise position and gesture relationship among the first image sensing unit, the second image sensing unit and the projecting unit;
• and calculating and obtaining system variable external parameters which comprise position and gesture relationship between the human computer interaction system and the projecting plane, and outputting position and gesture relationship parameters between the projecting plane and the projecting unit in the obtained system variable external parameters to the projected interface processing unit;
• the image processing unit is further for processing images of a projection area, which are collected by the first image sensing unit and the second image sensing unit, obtained through the interface unit;
• searching a fingertip or other touch element according to a predetermined fingertip feature, and obtaining a distance between the fingertip or other touch element and the projecting plane based on a principle of binocular disparity, if the distance is in the range of a predetermined first threshold value;
• determining a touch operation has occurred and outputting touch information;
• the projected interface processing unit is for receiving human computer interaction information intended to be projected, extracting feature information from the human computer interface interaction information, and outputting the feature information to the image processing unit;
• the projected interface processing unit is further for receiving the position and gesture parameter between the projecting plane and the projecting unit in the system variable external parameters output by the image processing unit, performing shape predistortion correction processing to interface images in the input human computer interaction interface information based on the parameter and then outputting to the interface unit;
• the controlling unit is for controlling all units of the system, and coordinately controlling work of all units of the system;
• the controlling unit is further for controlling the human computer interaction system in the statuses of obtaining the system inherent internal or external parameters, obtaining the system variable external parameters, and obtaining a fingertip and touch operation.