Whole Person Association with Face Screening
AssigneeGoogle Inc.
InventorsRobert McDonald, Shrenik Lad, Andrew Gallagher
Life of this patent
Estimating the term…
Abstract
Example aspects of the present disclosure are directed to computing systems and methods that perform whole person association with face screening and/or face hallucination. In particular, one aspect of the closure is directed to a multi-headed person and face detection model that performs both face and person detection in one model. Each of the face and person detection can find landmarks or other pose information and also a confidence score. The pose information for the face and person detections can be used to select certain face and person detections to associate together as a whole person detection, which can be referred to as an “appearance.”
Claims22
A computer system that screens face detections based on association with person detections, the computer system comprising:
- one or more processors;
- a machine-learned person detection model that is configured to detect human bodies depicted in images;
- a machine-learned face detection model that is configured to detect human faces depicted in images;
- and
- one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the computer system to perform operations, the operations comprising:
- obtaining an input image;
- inputting the input image into the machine-learned person detection model and the machine-learned face detection model;
- receiving one or more person detections as an output of the machine-learned person detection model, wherein each of the one or more person detections indicates a respective detected body location of a respective detected human body in the input image;
- receiving one or more face detections as an output of the machine-learned face detection model, wherein each of the one or more face detections indicates a respective detected face location of a respective detected human face in the input image;
- and
- for at least a first face detection of the one or more face detections:
- determining whether the first face detection is associated with one of the person detections;
- and
- modifying one or both of a first confidence score associated with the first face detection and a first confidence threshold associated with the first face detection based at least in part on whether the first face detection is associated with one of the person detections.
A computer-implemented method to hallucinate face detections in images, the method comprising:
- obtaining, by one or more computing devices, an input image;
- inputting, by the one or more computing devices, the input image into a machine-learned person detection model that is configured to detect human bodies depicted in images;
- receiving, by the one or more computing devices, a person detection as an output of the machine-learned person detection model, wherein the person detection indicates a detected body location of a detected human body in the input image;
- and
- generating, by the one or more computing devices, a hallucinated face detection based at least in part on the detected body location of the detected human body provided by the person detection, wherein the hallucinated face detection indicates a hallucinated face location in the input image of a face associated with the detected human body.
One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising:
- determining one or more face detections and one or more person detections relative to an image, where each face detection indicates a respective region or set of pixels of the image believed to contain a face and each person detection indicates a respective region or set of pixels believed to contain a person;
- computing a respective confidence score associated with each face detection and each putative person detection;
- identifying a respective set of pose points associated with each person detection that indicate a location of a set of body landmarks;
- determining an association score between at least a first face detection and at least a first person detection using the respective set of pose points associated with the first person detection;
- using the association score and the respective confidence score for the first face detection to produce an updated confidence score for the first face detection;
- and
- determining whether to discard the first face detection based on a comparison of the updated confidence scores to a confidence threshold.
Dependencies are inferred from the claim text.