US 7,352,880 B2Granted

System and method for detecting and tracking a plurality of faces in real time by integrating visual ques

AssigneeSamsung Electronics Co., Ltd.

InventorsTae-kyun Kim, Jong-ha Lee

Life of this patent

Estimating the term…

Loading patent…

Abstract

A face detection and tracking system for detecting and tracking a plurality of faces in real time from an input image comprises: a background removing unit which extracts an area having a motion by removing a background image from the input image; a candidate area extracting unit which extracts a candidate area in which a face can be located in the area having a motion by using a skin color probability map generated from a face skin color model and a global probability map a face area determination unit which extracts (ICA) features from a candidate area and determines whether the candidate area is a face area by using a trained SVM classifier; and a face area tracking unit which tracks the face area according to a directional kernel indicating a probability that a face is located in a next frame based on the skin color probability map.

Claims19

Dependent claims

A face detection and tracking system for detecting and tracking a plurality of faces in real time from an input image, the system comprising:

a background removing unit which extracts an area having a motion by removing a background image from the input image;
a candidate area extracting unit which extracts a candidate area in which a face is possibly located in the area having a motion by using a skin color probability map (Pskin) generated from a face skin color model and a global probability map (Pglobal), wherein the candidate area extracting unit comprises:
a skin color probability map generation unit which generates the skin color probability map (Pskin) of the area having a motion by using the face skin color model,
a global probability map generation unit which extracts a plurality of highest points of the area having a motion, sets central coordinates at a predetermined distance from the plurality of highest points, and calculates a probability that a face is located within a distance from the central coordinates to generate the global probability map (Pglobal), and
a multiple scale probability map generation unit which generates a multiple scale probability map about the probability that a face is located, by multiplying the skin color probability map (Pskin) and the global probability map (Pglobal), and extracts an area, in which the probability value of the generated multiple scale probability map is equal to or greater than a predetermined threshold value, as the candidate area where a face is possibly located;
a face area determination unit which extracts independent component analysis (ICA) features from the candidate area and determines whether the candidate area is a face area;
and
a face area tracking unit which tracks the face area according to a directional kernel indicating a probability that a face is located in a next frame based on the skin color probability map (Pskin).

5 more elements

A face detection and tracking system for detecting and tracking a plurality of faces in real time from an input image, the system comprising:

a background removing unit which obtains a first area which is not a background by using a brightness difference of the input image and a background image stored in advance, and obtains a second area which is not the background by using a color difference of the input image and the background image stored in advance, and among a plurality of sub-areas included in the second area that is not the background, extracts each sub-area which includes the center of a sub-area included in the first area that is not the background, as areas that are not background, to remove the background image from the input image and extract an area having a motion;
a skin color probability map generation unit which generates a skin color probability map (Pskin) of the area having a motion, by using a face skin color model;
a global probability map generation unit which extracts a plurality of highest points of the area having a motion, sets central coordinates at a predetermined distance from the plurality of highest points, calculates a probability that a face is located within a predetermined distance from the central coordinates, to generate a global probability map (Pglobal);
a multiple scale probability map generation unit which generates a multiple scale probability map about the probability that a face is located by multiplying the skin color probability map (Pskin) and the global probability map (Pglobal), and extracts an area, in which the probability value of the generated multiple scale probability map is equal to or greater than a predetermined threshold value, as a candidate area where a face is possibly located;
a face area determination unit which extracts independent component analysis (ICA) features from the candidate area and determines whether the candidate area is a face area by providing the ICA features of the candidate area to a support vector machine (SVM) which has learned features obtained by performing ICA on learning face images and features obtained by performing ICA on images that are not face images;
and
a face area tracking unit which tracks a face area according to a directional kernel indicating a probability that a face is located in a next frame, based on the skin color probability map (Pskin).

4 more elements

A face detection and tracking method for detecting and tracking a plurality of faces in real time from an input image, the method comprising:

(a) extracting an area having a motion by removing a background image from the input image;
(b) extracting a candidate area in which a face is possibly located in the area having a motion by using a skin color probability map (Pskin) generated from a face skin color model and a global probability map (Pglobal), wherein step (b) comprises:
(b1) generating the skin color probability map (Pskin) of the area having a motion by using the face skin color model,
(b2) extracting a plurality of highest points of the area having a motion, setting central coordinates at a predetermined distance from the plurality of highest points, and calculating a probability that a face is located within a predetermined distance from the central coordinates, to generate the global probability map (Pglobal), and
(b3) generating a multiple scale probability map about the probability that a face is located by multiplying the skin color probability map (Pskin) and the global probability map (Pglobal), and extracting an area, in which the probability value of the generated multiple scale probability map is equal to or greater than a predetermined threshold value, as the candidate area where a face is possibly located;
(c) extracting independent component analysis (ICA) features from a candidate area and determining whether the candidate area is a face area;
and
(d) tracking the face area according to a directional kernel indicating a probability that a face is located in a next frame, based on the skin color probability map (Pskin).

5 more elements

A face detection and tracking method for detecting and tracking a plurality of faces in real time by combining visual information of an input image, the method comprising:

obtaining a first area which is not a background by using a brightness difference of an input image and a background image stored in advance, and obtaining a second area which is not the background by using a color difference of the input image and a background image stored in advance, and among a plurality of sub-areas included in the second area that is not the background, extracting each sub-area which includes the center of a sub-area included in the first area that is not the background, as areas that are not background, to remove the background image from the input image and extract an area having a motion;
generating a skin color probability map (Pskin) of the area having a motion by using a face skin color model;
extracting a plurality of highest points of the area having a motion, setting central coordinates at a predetermined distance from the plurality of highest points, and calculating a probability that a face is located within a predetermined distance from the central coordinates, to generate a global probability map (Pglobal);
generating a multiple scale probability map about the probability that a face is located by multiplying the skin color probability map (Pskin) and the global probability map (Pglobal), and extracting an area, in which the probability value of the generated multiple scale probability map is equal to or greater than a predetermined threshold value, as a candidate area where a face is possibly located;
extracting independent component analysis (ICA) features from the candidate area and determining whether the candidate area is a face area by providing the ICA features of the candidate area to a support vector machine (SVM) which has learned features obtained by performing ICA on learning face images and features obtained by performing ICA on images that are not face images;
and
tracking a face area according to a directional kernel indicating a probability that a face is located in a next frame based on the skin color probability map.

4 more elements

Dependencies are inferred from the claim text.