# US 2019/0244052 A1 · Focalized Behavioral Measurements in a Video Stream

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244052 A1
- Document ID: US20190244052A1
- Status: Published application
- Kind: A1
- Application number: 16/386,239
- Filed: 2019-04-16
- Published: 2019-08-08
- Earliest claimed: 2017-03-02
- Assignees: Ricoh Company, Ltd.
- Inventors: Ramya Narasimha; Hector H. Gonzalez-Banos
- Web page: https://patentgrep.com/patent/US20190244052A1

## CPC

- G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
- G06F 16/739: {in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames}
- G06T 7/73: using feature-based methods
- G06V 10/40: Extraction of image or video features
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 20/47: {Detecting features for summarising video content}
- G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06V 40/172: {Classification, e.g. identification}
- G06V 40/176: {Dynamic expression}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06F 16/9566: {URL specific, e.g. using aliases, detecting broken or misspelled links}
- G06V 20/44: {Event detection}
- G06V 40/173: {face re-identification, e.g. recognising unknown faces across different face tracks}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system and method for analyzing behavior in a video is described. The method includes extracting a plurality of salient fragments of a video; building a database of the plurality of salient fragments; generating a focalized visualization, based on a time anchor, from the one or more salient fragments of the video; tagging a human subject in the focalized visualization with a unique identifier; analyzing the focalized visualization, based on the unique identifier, to generate a behavior score; and providing the behavior score via the user device.

## Claims

### Claim 1

A computer-implemented method comprising:
extracting a plurality of salient fragments of a video;
building a database of the plurality of salient fragments;
generating a focalized visualization, based on a time anchor, from the one or more salient fragments of the video;
tagging a human subject in the focalized visualization with a unique identifier;
analyzing the focalized visualization, based the unique identifier, to generate a behavior score; and
providing the behavior score via the user device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
storing the behavior score in a database using the time anchor and the label as a key to the database.

### Claim 3 (depends on 1)

The method of claim 1, wherein tagging the human subject comprises:
detecting a face in a frame of the focalized visualization;
identifying the face using a template of a known human subject, wherein the template is associated with the unique identifier; and
associating the face with the unique identifier.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
tracking the human subject over a plurality of frames of the focalized visualization.

### Claim 5 (depends on 1)

The method of claim 1, wherein analyzing the focalized visualization based on the query and the unique identifier to generate a behavior score comprises:
determining a position of the human subject in a fragment of the focalized visualization at a time corresponding to the time anchor;
assigning a score to the human subject as a function of the position of the human subject.

### Claim 6 (depends on 1)

The method of claim 1, wherein analyzing the focalized visualization based on the query and the unique identifier to generate a behavior score comprises:
determining a mood of the human subject in a fragment of the focalized visualization at a time corresponding to the time anchor;
assigning a score to the human subject as a function of the mood of the human subject.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
generating a baseline score for a behavioral attribute;
determining a comparison score based on the baseline score and the behavior score.

### Claim 8

A computer program product comprising a non-transitory computer readable medium storing a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:
extract a plurality of salient fragments of a video;
build a database of the plurality of salient fragments;
generate a focalized visualization, based on a time anchor, from the one or more salient fragments of the video;
tag a human subject in the focalized visualization with a unique identifier;
analyze the focalized visualization, based on the unique identifier, to generate a behavior score; and
provide the behavior score via the user device.

### Claim 9 (depends on 8)

The computer program product of claim 8, wherein the computer readable program when executed on the computer further causes the computer to:
store the behavior score in a database using the time anchor and the label as a key to the database.

### Claim 10 (depends on 8)

The computer program product of claim 8, wherein to tag the human subject, the computer readable program causes the computer to:
detect a face in a frame of the focalized visualization;
identify the face using a template of a known human subject, wherein the template is associated with the unique identifier; and
associate the face with the unique identifier.

### Claim 11 (depends on 8)

The computer program product of claim 8, wherein the computer readable program when executed on the computer further causes the computer to:
track the human subject over a plurality of frames of the focalized visualization.

### Claim 12 (depends on 8)

The computer program product of claim 8, wherein to analyze the focalized visualization based on the query and the unique identifier to generate a behavior score, the computer readable program causes the computer to:
determine a position of the human subject in a fragment of the focalized visualization at a time corresponding to the time anchor;
assign a score to the human subject as a function of the position of the human subject.

### Claim 13 (depends on 8)

The computer program product of claim 8, wherein to analyze the focalized visualization based on the query and the unique identifier to generate a behavior score, the computer readable program causes the computer to:
determine a mood of the human subject in a fragment of the focalized visualization at a time corresponding to the time anchor;
assign a score to the human subject as a function of the mood of the human subject.

### Claim 14 (depends on 8)

The computer program product of claim 8, wherein the computer readable program when executed on the computer further causes the computer to:
generate a baseline score for a behavioral attribute;
determine a comparison score based on the baseline score and the behavior score.

### Claim 15

A system comprising:
one or more processors; and
a memory, the memory storing instructions which when executed cause the one or more processors to:
extract a plurality of salient fragments of a video;
build a database of the plurality of salient fragments;
generate a focalized visualization, based on a time anchor, from the one or more salient fragments of the video;
tag a human subject in the focalized visualization with a unique identifier;
analyze the focalized visualization, based on the unique identifier, to generate a behavior score; and
provide the behavior score via the user device.

### Claim 16 (depends on 15)

The system of claim 15, wherein the instructions further causes the one or more processors to:
store the behavior score in a database using the time anchor and the label as a key to the database.

### Claim 17 (depends on 15)

The system of claim 15, wherein to tag the human subject, the instructions further causes the one or more processors to:
detect a face in a frame of the focalized visualization;
identify the face using a template of a known human subject, wherein the template is associated with the unique identifier; and
associate the face with the unique identifier.

### Claim 18 (depends on 15)

The system of claim 15, wherein the instructions further causes the one or more processors to:
track the human subject over a plurality of frames of the focalized visualization.

### Claim 19 (depends on 15)

The system of claim 15, wherein to analyze the focalized visualization based on the query and the unique identifier to generate a behavior score, the instructions further causes the one or more processors to:
determine a position of the human subject in a fragment of the focalized visualization at a time corresponding to the time anchor;
assign a score to the human subject as a function of the position of the human subject.

### Claim 20 (depends on 15)

The system of claim 15, wherein to analyze the focalized visualization based on the query and the unique identifier to generate a behavior score, the instructions further causes the one or more processors to:
determine a mood of the human subject in a fragment of the focalized visualization at a time corresponding to the time anchor;
assign a score to the human subject as a function of the mood of the human subject.
