Video retrieval system based on larger pose face frontalization

A computer-implemented method for video retrieval, comprising:

• configuring a requesting server to retrieve a set of video sequences from a remote database of multiple video sequences responsive to an input text specifying a face recognition result in a form of an identity of a subject of an input image,
• wherein the face recognition result is determined by
• estimating, by a processor of the requesting server using a three-dimensional Morphable Model (3DMM) conditioned Generative Adversarial Network (GAN), 3DMM coefficients for the subject of the input image, wherein the subject varies from an ideal front pose;
• producing, by the processor using an image generator, a synthetic frontal face image of the subject of the input image based on the input image and the 3DMM coefficients, wherein an area spanning the frontal face of the subject is made larger in the synthetic frontal face image than in the input image;
• providing, by the processor using a discriminator, a decision indicative of whether the subject of the synthetic frontal face image is an actual person;
• and
• providing, by the processor using a face recognition engine, the identity of the subject in the input image based on the synthetic frontal face image and the input image.

A computer program product for video retrieval, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:

• configuring a requesting server to retrieve a set of video sequences from a remote database of multiple video sequences responsive to an input text specifying a face recognition result in a form of an identity of a subject of an input image,
• wherein the face recognition result is determined by
• estimating, by a processor of the requesting server using a three-dimensional Morphable Model (3DMM) conditioned Generative Adversarial Network (GAN), 3DMM coefficients for the subject of the input image, wherein the subject varies from an ideal front pose;
• producing, by the processor using an image generator, a synthetic frontal face image of the subject of the input image based on the input image and the 3DMM coefficients, wherein an area spanning the frontal face of the subject is made larger in the synthetic frontal face image than in the input image;
• providing, by the processor using a discriminator, a decision indicative of whether the subject of the synthetic frontal face image is an actual person;
• and
• providing, by the processor using a face recognition engine, the identity of the subject in the input image based on the synthetic frontal face image and the input image.