Method and system to segment depth images and to detect shapes in three-dimensionally acquired data

#The invention relates generally to recognizing objects acquired in three-dimensionally acquired data, including data acquired from image sensors, for example depth or range finders, image mapping sensors, three-dimensional image capture sensors including capture of images with color perception not limited by human color perception.

#Electronic camera and range sensor systems that provide a measure of distance from the system to a target object are known in the art. Many such systems approximate the range to the target object based upon luminosity or brightness information obtained from the target object. Some such systems are passive and respond to ambient light reflected from the target object, while other systems emit and then detect emitted light reflected from the target object. However luminosity-based systems may erroneously yield the same measurement information for a distant target object that happens to have a shiny surface and is thus highly reflective, as for a target object that is closer to the system but has a dull surface that is less reflective.

#A more accurate distance measuring system is a so-called time-of-flight (TOF) system. FIG. 1 depicts an exemplary TOF system, as described in U.S. Pat. No. 6,323,942 entitled CMOS-Compatible Three-Dimensional Image Sensor IC (2001), which patent is incorporated herein by reference as further background material. TOF system 100 can be implemented on a single IC 110, without moving parts and with relatively few off-chip components. System 100 includes a two-dimensional array 130 of pixel detectors 140, each of which has dedicated circuitry 150 for processing detection charge output by the associated detector. In a typical application, array 130 might include 100×100 pixels 230, and thus include 100×100 processing circuits 150. IC 110 also includes a microprocessor or microcontroller unit 160, memory 170 (which preferably includes random access memory or RAM and read-only memory or ROM), a high speed distributable clock 180, and various computing and input/output (I/O) circuitry 190. Among other functions, controller unit 160 may perform distance to object and object velocity calculations.

#FIG. 1 is a diagram showing a time-of-flight range finding system, according to the prior art;

#FIG. 2A depicts a phase-shift intensity and range finding system, according to the prior art;

#FIG. 2B depicts a transmitted periodic signal with high frequency components transmitted by the system of FIG. 2A, according to the prior art;

#FIG. 3 depicts a three-dimensional imaging system 200′ comprising a depth image data acquisition system 210, and a segmenter-image recognizer system 220, according to the present invention. Depth image data acquisition system 210 may be almost any system that acquires images with three-dimensional depth data. As such system 210 could include time-of-flight systems such as shown in FIG. 1, phase-shift detection systems such as shown in FIG. 2A, laser range sensors, and stereographic camera systems, among other. As such, energy emanating from system 200 is drawn with phantom lines to denote that system 200 may not actively emit optical energy but instead rely upon passive optical energy from ambient light.

#System 210 may be implemented in many ways and can, more or less, provide a stream of output information (DATA) that includes a measure of distance z to a target object 20. Such DATA may include information as to target objects that might not be readily identifiable from the raw information. However, according to the present invention, segmenter-image recognizer system 220 can process the information stream using an algorithmic procedure 240 stored in memory 230, and executable by a microprocessor 250. If depth image data acquisition system 210 is implemented according to the systems of FIG. 1 or FIG. 2A, functions of some or all elements of segmenter-image recognizer system 220 may be provided elsewhere. For example, processing tasks of microprocessor 250 in FIG. 3 may in fact be carried out by microprocessor 160 in FIG. 1 or FIG. 2A, and/or storage facilities provided by memory 230 in FIG. 3 may be carried out by memory 170 in FIG. 1 or FIG. 2A.

#Referring to FIG. 4A, assume that data processed by generic system 200 in FIG. 3A produced image 300, an image comprising three objects: a person 310, a vehicle 320, and background 330. When software 240 implementing an algorithm according to the present invention is executed, e.g., by microprocessor 250, segmentation and identification of the data acquired by system 200 is carried out, according to an embodiment of the present invention.