# US 2017/0364741 A1 · Computer-based micro-expression analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0364741 A1
- Document ID: US20170364741A1
- Status: Published application
- Kind: A1
- Application number: 15/183,622
- Filed: 2016-06-15
- Published: 2017-12-21
- Earliest claimed: 2016-06-15
- Applicants: Stockholm University
- Inventors: Stephan Hau
- Web page: https://patentgrep.com/patent/US20170364741A1

## CPC

- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens}
- G06V 10/768: {using context analysis, e.g. recognition aided by known co-occurring patterns}
- G06V 40/174: {Facial expression recognition}
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A computer-implemented method of digital image analysis includes obtaining first digital video of a human subject that indicates facial expressions of the human subject; performing micro-expression analysis on the human subject using the first digital video; comparing results of the performed micro-expression analysis with content of a presentation determined to have been provided to the human subject at the same time that particular portions of the digital video were initially captured; and modifying a manner of performing interaction with the human subject or other human subjects based on the comparing of results.

## Claims

### Claim 1

A computer-implemented method of digital image analysis, the method comprising:
obtaining first digital video of a human subject that indicates facial expressions of the human subject;
performing micro-expression analysis on the human subject using the first digital video;
comparing results of the performed micro-expression analysis with content of a presentation determined to have been provided to the human subject at the same time that particular portions of the digital video were initially captured; and
modifying a manner of performing interaction with the human subject or other human subjects based on the comparing of results.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
obtaining second digital video of a presentation with which the human subject was interacting when the first digital video was captured; and
time-wise aligning data generated from the second digital video with the results of performing the micro-expression analysis, so that particular ones of the results of performing the micro-expression analysis are correlated with particular messages delivered by the presentation to the human subject.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the presentation with which the human subject interacts is a conversation with another human.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, further comprising electronically converting speech by the another human to text for computer analysis.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, further comprising electronically identifying one or more topics to which text from speech of the another human is determined to be directed.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, further comprising electronically correlating each of the identified one or more topics to corresponding emotion tags identified from the micro-expression analysis that indicate emotional reactions of the human subject to the one or more topics.
