Back to results
US 9,805,252 B2Granted
Video based facial recognition for customer verification at touchless checkout
AssigneeToshiba Tec Kabushiki Kaisha
InventorsShiyang Lu, Chaminda Weerasinghe
Life of this patent
Estimating the term…
Abstract
Facial recognition method comprises generating an average three-dimensional model from a series of two-dimensional, consecutive, captured video frames and rotating that three-dimensional model to a frontal view. The three-dimensional model may be reduced to a two-dimensional model in order to create a mean reference map. Next, the method includes accessing a facial database to compare the mean reference map to a series of previously-stored reference maps, one for each individual in the database, each of the previously-stored reference maps including a set of previously-stored reference points. Finally, an individual whose previously-stored reference map most closely matches the mean reference map is identified as a match.
Claims10
A facial recognition system comprising:
- a station including at least one video camera for capturing a series of video frames of an individual face;
- and
- a face image processing server for
- generating a series of two-dimensional models of the individual face at various angles, each model based upon one of the series of video frames,
- combining the series of two-dimensional models of the individual face with a projecting matrix to create a three-dimensional model of the individual face, wherein the three-dimensional model is a matrix F3D=[x00 . . . xn, y00 . . . yn, z00 . . . zn], where xn is an x coordinate value of an nth reference point, where n is a number of facial reference points in each video frame and an integer greater than one yn is a y coordinate value of the nth reference point, and zn is a z coordinate value of the nth reference point and wherein the three-dimensional model is generated by multiplying the inverse of a projecting matrix M defined as M=[1 0 0, 0 1 0]*RRotation where RRotation=[cos α cos β cos α sin β sin γ−sin α cos γ cos α sin β cos γ+sin α sin γ, sin α cos β sin α sin β sin γ+cos α cos γ sin α sin β cos γ−cos α sin γ, −sin β cos β sin γ cos β cos γ ] where α is yaw, β is pitch, and γ is roll, times a matrix of two-dimensional coordinates of n facial reference points F2D in the form [x001 . . . xn1 . . . x00t . . . xnt, y001 . . . yn1 . . . y00t . . . ynt] and where M−1 is [R111 R121, R131 . . . R11t R12t R13t, R211 R221 R231 . . . R21t R22t R23t] where t is a number of series of video frames and where Rijt is an entry of the rotation matrix RRotation of the t-th frame of series of video frames;
- rotating the three-dimensional model to a reference frame that corresponds to a frontal view of the individual face,
- generating a two-dimensional mean reference map by projecting from the rotated three-dimensional model, the mean reference map made up of the series of n facial reference points,
- accessing a facial database to compare the mean reference map to a series of previously-stored reference maps, each of the previously-stored reference maps including a set of previously-stored reference points, and
- identifying as a match an individual whose previously-stored reference map most closely matches the mean reference map.
6 more elements
A method of performing facial recognition comprising:
- capturing a series of video frames of an individual face using a camera;
- generating a series of two-dimensional models of the individual face at various angles, each model based upon one of the series of video frames;
- combining the series of two-dimensional models of the individual face with a projecting matrix to create a three-dimensional model of the individual face, wherein the three-dimensional model is a matrix F3D=[x00 . . . xn, y00 . . . yn, z00 . . . zn], where xn is an x coordinate value of an nth reference point, where n is a number of facial reference points in each video frame and an integer greater than one yn is a y coordinate value of the nth reference point, and zn is a z coordinate value of the nth reference point and wherein the three-dimensional model is generated by multiplying the inverse of a projecting matrix M defined as M=[1 0 0, 0 1 0]*RRotation where RRotation=[cos α cos β cos α sin β sin γ−sin α cos γ cos α sin β cos γ+sin α sin γ, sin α cos β sin α sin β sin γ+cos α cos γ sin α sin β cos γ−cos α sin γ, −sin β cos β sin γ cos β cos γ ] where α is yaw, β is pitch, and γ is roll, times a matrix of two-dimensional coordinates of n facial reference points F2D in the form [x001 . . . xn1 . . . x00t . . . xnt, y001 . . . yn1 . . . y00t . . . ynt] and where M−1 is [R111 R121, R131 . . . R11t R12t R13t, R211 R221 R231 . . . R21t R22t R23t] where t is a number of series of video frames and where Rijt is an entry of the rotation matrix RRotation of the t-th frame of series of video frames;
- rotating the three-dimensional model to a reference frame that corresponds to a frontal view of the individual face;
- generating a two-dimensional mean reference map by projecting from the rotated three-dimensional model, the mean reference map made up of the series of n facial reference points;
- accessing a facial database to compare the mean reference map to a series of previously-stored reference maps each of the previously-stored reference maps including a set of previously-stored reference points;
- and
- identifying as a match, an individual whose previously-stored reference map most closely matches the mean reference map.
5 more elements
Dependencies are inferred from the claim text.