# US 2012/0250950 A1 · Face Recognition Based on Spatial and Temporal Proximity

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0250950 A1
- Document ID: US20120250950A1
- Status: Published application
- Kind: A1
- Application number: 13/074,743
- Filed: 2011-03-29
- Published: 2012-10-04
- Earliest claimed: 2011-03-29
- Assignees: Facebook, Inc.
- Inventors: Phaedra Papakipos; Matthew Nicholas Papakipos
- Web page: https://patentgrep.com/patent/US20120250950A1

## CPC

- G06F 16/583: using metadata automatically derived from the content
- G06F 16/532: Query formulation, e.g. graphical querying
- G06F 16/535: Filtering based on additional data, e.g. user or group profiles
- G06F 16/54: Browsing; Visualisation therefor
- G06F 16/5854: {using shape and object relationship}
- G06F 16/784: {the detected or recognised objects being people}
- G06F 18/24: Classification techniques
- G06V 20/30: in albums, collections or shared content, e.g. social network photos or video
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/172: {Classification, e.g. identification}
- G06V 40/179: {metadata assisted face recognition}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

In one embodiment, a social networking system determines one or more individuals matching one or more faces in an image file of a still image or a video sequence, associated with a first user based on the one or more individuals' spatial and temporal proximity to the image file, and presents the matched individuals to the first user.

## Claims

### Claim 1

A method, comprising:
accessing an image file associated with a first user;
identifying one or more faces in an image corresponding to the image file;
generating a list of candidates comprising one or more individuals in temporal and spatial proximity to spatio-temporal data associated with the image file;
for at least a first face in the one or more identified faces in the image, determining one or more individuals from the list of candidates as a match to the first face; and
presenting the one or more matched individuals to the first user.

### Claim 2 (depends on 1)

The method of claim 1 wherein the list of candidates is further limited to individuals within a threshold social proximity to the first user.

### Claim 3 (depends on 1)

The method of claim 1 wherein the determining one or more individuals from the list of candidates as a match to the first face further comprises:
applying a facial recognition algorithm to the first face and a set of face images associated with the list of candidates to compute a matching score between the first face and each face image in the set of face images; and
determining one or more individuals corresponding to the set of face images as a match to the first face based on the matching scores.

### Claim 4 (depends on 1)

The method of claim 1 wherein the presenting the one or more matched individuals to the first user further comprises:
accessing one or more data stores for information associated with the one or more matched individuals;
for each identified face with one or more matched individuals, generating a frame comprising information associated with the one or more individuals matching the identified face; and
displaying the one or more frames overlaying the image.

### Claim 5 (depends on 4)

The method of claim 4 wherein each frame of the one or more frames further comprises one or more selectable icons.

### Claim 6 (depends on 1)

The method of claim 1 wherein the image file further comprises a photographic picture and associated metadata in digital form.

### Claim 7 (depends on 1)

The method of claim 1 wherein the image file further comprises a moving picture and associated metadata in digital form.

### Claim 8

A system comprising:
a memory;
one or more processors; and
a non-transitory, storage medium storing computer-readable instructions operative, when executed, to cause the one or more processors to:
access an image file associated with a first user;
identify one or more faces in an image corresponding to the image file;
generate a list of candidates comprising one or more individuals in temporal and spatial proximity to spatio-temporal data associated with the image file;
for at least a first face in the one or more identified faces in the image, determine one or more individuals from the list of candidates as a match to the first face; and
present the one or more matched individuals to the first user.

### Claim 9 (depends on 8)

The system of claim 8 wherein the list of candidates is further limited to individuals within a threshold social proximity to the first user.

### Claim 10 (depends on 8)

The system of claim 8 wherein to determine one or more individuals from the list of candidates as a match to the first face further comprises instructions operable to cause the one or more processors to:
apply a facial recognition algorithm to the first face and a set of face images associated with the list of candidates to compute a matching score between the first face and each face image in the set of face images; and
determine one or more individuals corresponding to the set of face images as a match to the first face based on the matching scores.

### Claim 11 (depends on 8)

The system of claim 8 wherein to present the one or more matched individuals to the first user further comprises instructions operable to cause the one or more processors to:
access one or more data stores for information associated with the one or more matched individuals;
for each identified face with one or more matched individuals, generate a frame comprising information associated with the one or more individuals matching the identified face; and
display the one or more frames overlaying the image.

### Claim 12 (depends on 11)

The system of claim 11 wherein each frame of the one or more frames further comprises one or more selectable icons.

### Claim 13 (depends on 8)

The system of claim 8 wherein the image file further comprises a photographic picture and associated metadata in digital form.

### Claim 14 (depends on 8)

The system of claim 8 wherein the image file further comprises a moving picture and associated metadata in digital form.

### Claim 15

One or more computer-readable tangible storage media embodying software operable when executed by one or more computing devices to:
access an image file associated with a first user;
identify one or more faces in an image corresponding to the image file;
generate a list of candidates comprising one or more individuals in temporal and spatial proximity to spatio-temporal data associated with the image file;
for at least a first face in the one or more identified faces in the image, determine one or more individuals from the list of candidates as a match to the first face; and
present the one or more matched individuals to the first user.

### Claim 16 (depends on 15)

The media of claim 15 wherein the list of candidates is further limited to individuals within a threshold social proximity to the first user.

### Claim 17 (depends on 15)

The media of claim 15 wherein to determine one or more individuals from the list of candidates as a match to the first face further comprises software operable when executed by the one or more computing devices to:
apply a facial recognition algorithm to the first face and a set of face images associated with the list of candidates to compute a matching score between the first face and each face image in the set of face images; and
determine one or more individuals corresponding to the set of face images as a match to the first face based on the matching scores.

### Claim 18 (depends on 15)

The media of claim 15 wherein to present the one or more matched individuals to the first user further comprises software operable when executed by the one or more computing devices to:
access one or more data stores for information associated with the one or more matched individuals;
for each identified face with one or more matched individuals, generate a frame comprising information associated with the one or more individuals matching the identified face; and
display the one or more frames overlaying the image.

### Claim 19 (depends on 18)

The media of claim 18 wherein each frame of the one or more frames further comprises one or more selectable icons.

### Claim 20 (depends on 15)

The media of claim 15 wherein the image file further comprises a moving picture and associated metadata in digital form.
