# US 2012/0251078 A1 · Aggregated Facial Tracking in Video

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0251078 A1
- Document ID: US20120251078A1
- Status: Published application
- Kind: A1
- Application number: 13/076,445
- Filed: 2011-03-31
- Published: 2012-10-04
- Earliest claimed: 2011-03-31
- Assignees: Microsoft Corporation
- Inventors: Ido Leichter; Eyal Krupka; Igor Abramovski; Igor Kviatkovsky
- Web page: https://patentgrep.com/patent/US20120251078A1

## CPC

- G06V 40/173: {face re-identification, e.g. recognising unknown faces across different face tracks}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A facial detecting system may analyze a video by traversing the video forwards and backwards to create tracks of a person within the video. After separating the video into shots, the frames of each shot may be analyzed using a face detector algorithm to produce some analyzed information for each frame. A facial track may be generated by grouping the faces detected and by traversing the sequence of frames forwards and backwards. Facial tracks may be joined together within a shot to generate a single track for a person's face within the shot, even when the tracks are discontinuous.

## Claims

### Claim 1

A method performed on at least one computer processor, said method comprising:
receiving a video comprising a sequence of frames;
for at least one shot in said video, analyzing each of said frames to detect faces, said faces being identified with at least a position and a size;
creating a first facial track by:
selecting a first face in a first frame;
analyzing at least one frame subsequent said first frame to identify said first face; and
analyzing at least one frame preceding said first frame to identify said first face to create said first facial track.

### Claim 2 (depends on 1)

The method of claim 1 said creating at least one facial track further comprising:
identifying a second facial track;
determining that said first facial track contains a similar face as said second facial track; and
combining said first facial track and said second facial track into a single facial track.

### Claim 3 (depends on 2)

The method of claim 2, said second facial track not sharing a common frame with said first facial track in said sequence of said frames.

### Claim 4 (depends on 3)

The method of claim 3, said determining that said first facial track contains a similar face as said second facial track being performed using image analysis of at least one face in said first facial track and at least one face said second facial track.

### Claim 5 (depends on 4)

The method of claim 4, said image analysis comprising color histogram analysis.

### Claim 6 (depends on 4)

The method of claim 4, said image analysis comprising facial structure analysis.

### Claim 7 (depends on 1)

The method of claim 1, said first face being identified by comparing said position and said size from said first face in a first frame to said position and said size from said first face in said second frame.

### Claim 8 (depends on 7)

The method of claim 7, said first face being identified by comparing said position and said size from said first face in a first frame to said position and said size from said first face in a group of frames comprising said second frame.

### Claim 9 (depends on 8)

The method of claim 8, said comparing using a clustering algorithm.

### Claim 10

A system comprising:
a face detector that:
analyzes each frame of a first shot in said video to identify faces, said faces being identified with at least a position and a size;
a facial track analyzer that:
selects a first face in a first frame;
analyzes at least one frame after said first frame to identify said first face; and
analyzes at least one frame before said first frame to identify said first face to create said first facial track;
said system being executed on at least one processor.

### Claim 11 (depends on 10)

The system of claim 10, said facial track analyzer that:
combines a second facial track to said first facial track, said first facial track and second facial track not overlapping.

### Claim 12 (depends on 11)

The system of claim 11, said facial track analyzer that:
combines said second facial track to said first facial, a first facial track being in a first shot and said second facial track being in a second shot.

### Claim 13 (depends on 10)

The system of claim 10, said face detector identifying a reliability factor for said first face.

### Claim 14 (depends on 13)

The system of claim 13, said facial track analyzer that further:
analyzes said facial track to determine a second frame comprising said first face and having a high reliability factor; and
selecting at least a portion of said second frame to represent said first face in said facial track.

### Claim 15 (depends on 10)

The system of claim 10, said face detector that further:
generates image content analysis of said faces.

### Claim 16 (depends on 10)

The system of claim 10, further comprising:
a video parser that:
receives a video comprising a sequence of frames; and
analyzing said video to identify at least one shot, said shot being a continuous sequence of said frames;

### Claim 17 (depends on 10)

The system of claim 10, said facial track analyzer that analyzes said first face in said frame and said at least one frame before said first frame using said position and said size only.

### Claim 18

A method performed on at least one computer processor, said method comprising:
receiving a video comprising a sequence of frames;
analyzing said video to identify at least one shot, said shot being a continuous sequence of said frames;
for a first shot, analyzing each of said frames to detect faces, said faces being identified with a position, a size, and a reliability factor;
creating a first facial track by:
selecting a first frame;
selecting a first face in said first frame, said first face having a high reliability factor for a plurality of faces in said first frame;
analyzing a first set of frames subsequent said first frame to identify said first face, said first face being found in at least one of said first set of frames; and
analyzing a second set of frames preceding said first frame to identify said first face to create said first facial track, said first facial track comprising all of said first set of frames and all of said second set of frames.

### Claim 19 (depends on 18)

The method of claim 18, said first face being not found in at least one of said first set of frames.

### Claim 20 (depends on 19)

The method of claim 19, said analyzing being performed only using said position and said size.
