Projector system and method for projecting images

A method for projecting one or more images on a surface using a projector system, wherein the projector system comprises:

• a projector for projecting light on a surface, wherein the projected light is configured to form one or more images on the surface;
• a first camera for recording an image of the surface, the first camera being sensitive to projected light that has reflected from the surface;
• and
• a second camera for recording images of the surface, the second camera being sensitive to detection light from an object near and/or on the surface and/or between the surface and the second camera, such as a hand on the surface, the second camera being insensitive to the projected light;
• wherein the projector, the first camera, and the second camera are arranged in known positions relative to each other, the second camera being arranged at a certain distance from the first camera;
• the method comprising:
• in a preparatory stage:
• determining or receiving mapping information, the mapping information comprising a first transformation matrix, and the mapping information associating positions within an image as recorded by the second camera to respective positions within an image as recorded by the first camera, based on the relative positions of the projector, the first camera and the second camera and based on intrinsic parameters of the first camera and of the second camera;
• in a calibration stage:
• providing calibration image data to the projector, the calibration image data representing one or more first calibration images having a first image feature at a first position within the calibration image, and causing the projector to project light on the surface to form a projected version of the one or more first calibration images on the surface;
• receiving, from the first camera, image data representing one or more recorded images of the surface as recorded by the first camera while the one or more first calibration images was being projected on the surface, wherein the first image feature is present in the one or more recorded images at a second position within the one or more recorded images;
• determining that the first image feature in the one or more recorded images corresponds to the first image feature in the one or more calibration images thus determining a first correspondence between the first positions within the one or more first calibration images and the second position within the one or more recorded images;
• and
• estimating a second transformation matrix based on the determined first correspondence, the second transformation matrix defining a linear transformation between a projector space and a first camera space;
• and in an operation stage:
• providing image data to the projector, the image data representing one or more images, and causing the projector to project light on the surface to form a projected version of each of the one or more images on the surface;
• receiving, from the second camera, detection image data representing a recorded detection image as recorded by the second camera while the one or more images were being projected on the surface, the recorded detection image having an image of a particular object at a third position within the recorded detection image, and
• determining a position of the particular object within the one or more images represented by the image data, based on a product of the first and second transformation matrices, and the third position.