System and method for selecting a two-dimensional region of interest using a range sensor

A method of obtaining a two-dimensional region of interest in a two-dimensional image, the method comprising:

• acquiring a first image of a scene with a first camera, the first image being represented by a plurality of first points (u1, v1) in a first image plane, the first camera defining a first camera coordinate system (C1);
• acquire a range image of the scene with a range sensor, the range image being represented by a plurality of second points (u2, v2, d) in a second image plane;
• wherein the range sensor is spaced a first distance from the first camera and defines a range sensor coordinate system (R);
• converting the range image to a three-dimensional sample of points (x2, y2, z2) in the range sensor coordinate system (R);
• selecting a three-dimensional region of interest (x2*, y2*, z2*) in the range sensor coordinate system (R) from the three-dimensional sample of points (x2, y2, z2);
• transforming the three-dimensional region of interest from the range sensor coordinate system (R) to the first camera coordinate system (C1) [(x2*, y2*, z2*) to (x1*, y1*, z1*)];
• mapping the three-dimensional region of interest (x1*, y1*, z1*) in the first camera coordinate system (C1) onto the first image plane to obtain the two-dimensional region of interest (u1*, v1*);
• wherein the selected three-dimensional region of interest (x2*, y2*, z2*) in the range sensor coordinate system (R) includes only objects in the scene on a surface of or within a three-dimensional volume, the three-dimensional volume defining selected points;
• a spatial location of each of the selected points is a function of time;
• wherein the selected three-dimensional region of interest (x2*, y2*, z2*) in the range sensor coordinate system (R) includes only objects in the scene that are less than a minimum distance from the range sensor;
• and
• wherein said converting the range image to a three-dimensional sample of points (x2, y2, z2) in the range sensor coordinate system (R) includes:
• obtaining a conversion matrix (P2) for converting the range sensor coordinate system (R) to the second image plane;
• and
• using an inverse of the conversion matrix for converting the range image to the three-dimensional sample of points (x2, y2, z2).