Face video based heart rate monitoring using pulse signal modelling and tracking

A system (100) comprising:

• one or more processors (104);
• and
• one or more internal data storage devices (102) operatively coupled to the one or more processors (104) for storing instructions configured for execution by the one or more processors, the instructions being comprised in:
• a window extractor module (102A) configured to:
• extract in real time a plurality of non-overlapping windows from an input video with a detected face, wherein the non-overlapping windows correspond to time intervals of the input video, each of the time intervals containing an equal number of frames;
• a window analyzer module (102B) configured to:
• extract a pulse signal from each of the non-overlapping windows;
• estimate a first quality measure as a quality of the pulse signal in each of the frames based on out-of-plane movement in a z direction, wherein the first quality measure has (f−1) values for f number of the frames;
• estimate a second quality measure based on peak signal to noise ratio (PSNR), wherein the second quality measure is a single value for the pulse signal;
• and
• post process the pulse signal by reconstructing the pulse signal at the frames affected by the out-of-plane movements using the remaining frames to obtain modelled pulse signals;
• and
• a heart rate (HR) tracker module (102C) configured to:
• track heart rate corresponding to the modelled pulse signal from each of the non-overlapping windows based on a Bayesian decision theory.

A computer program product comprising a non-transitory computer readable medium having a computer readable program embodied therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

• extract in real time a plurality of non-overlapping windows from an input video with a detected face, wherein the non-overlapping windows correspond to time intervals of the input video, each of the time intervals containing an equal number of frames;
• analyze each of the non-overlapping windows by:
• extracting a pulse signal from each of the non-overlapping windows;
• estimating a first quality measure as a quality of the pulse signal in each of the frames based on out-of-plane movement in a z direction, wherein the first quality measure has (f−1) values for f number of the frames;
• estimating a second quality measure based on peak signal to noise ratio (PSNR), wherein the second quality measure is a single value for the pulse signal;
• and
• post processing the pulse signal, to obtain a modelled pulse signal from each of the overlapping windows, by reconstructing the pulse signal at the frames affected by the out-of-plane movements using the remaining frames;
• and
• track heart rate corresponding to the modelled pulse signal based on a Bayesian decision theory.