# US 2021/0241465 A1 · Expression transfer across telecommunications networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241465 A1
- Document ID: US20210241465A1
- Status: Published application
- Kind: A1
- Application number: 17/235,631
- Filed: 2021-04-20
- Published: 2021-08-05
- Earliest claimed: 2017-10-25
- Assignees: Connectivity Labs Inc.
- Applicants: Connectivity Labs Inc.
- Inventors: Thomas YAMASAKI; Rocky Chau-Hsiung LIN; Koichiro KANDA
- Web page: https://patentgrep.com/patent/US20210241465A1

## CPC

- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 1/0007: {Image acquisition}
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 13/80: Two-dimensional [2D] animation, e.g. using sprites
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- G06T 2207/30201: Face
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Methods, devices, and systems for expression transfer are disclosed. The disclosure includes capturing a first image of a face of a person. The disclosure includes generating an avatar based on the first image of the face of the person, with the avatar approximating the first image of the face of the person. The disclosure includes transmitting the avatar to a destination device. The disclosure includes capturing a second image of the face of the person on a source device. The disclosure includes calculating expression information based on the second image of the face of the person, with the expression information approximating an expression on the face of the person as captured in the second image. The disclosure includes transmitting the expression information from the source device to the destination device. The disclosure includes animating the avatar on a display component of the destination device using the expression information.

## Claims

### Claim 1

A method comprising:
capturing a first image of a face of a person;
generating an avatar based on the first image of the face of the person, wherein the avatar approximates the first image of the face of the person;
transmitting the avatar to a destination device;
capturing a second image of the face of the person on a source device;
calculating expression information based on the second image of the face of the person, wherein the expression information approximates an expression on the face of the person as captured in the second image;
transmitting the expression information from the source device to the destination device; and
animating the avatar on a display component of the destination device using the expression information,
wherein the transmitting the expression information from the source device to the destination device is performed after the transmitting the avatar to the destination device.

### Claim 2 (depends on 1)

The method of claim 1,
wherein the avatar is a photo-realistic avatar used to simulate a presence of the person during a video call with a user of the destination device.

### Claim 3 (depends on 2)

The method of claim 2,
wherein the transmitting the avatar to the destination device is performed at most one time during the video call, and
wherein the transmitting the expression information from the source device to the destination device is performed more than once during the video call.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
capturing a third image of a face of a second person;
generating a second avatar based on the third image of the face of the second person, wherein the second avatar approximates the third image of the face of the second person;
transmitting the second avatar to the source device;
capturing a fourth image of the face of the second person on the destination device;
calculating second expression information based on the fourth image of the face of the second person, wherein the second expression information approximates an expression on the face of the second person as captured in the fourth image;
transmitting the second expression information from the destination device to the source device; and
animating the second avatar on a display component of the source device using the second expression information.

### Claim 5 (depends on 4)

The method of claim 4,
wherein the transmitting the expression information and the transmitting the second expression information are performed substantially simultaneously.

### Claim 6 (depends on 5)

The method of claim 5,
wherein the animating the avatar on the display component of the destination device using the expression information includes animating the avatar in a virtual reality environment.

### Claim 7 (depends on 1)

The method of claim 1,
wherein the calculating the expression information, the transmitting the expression information, and the animating the avatar are performed substantially in real-time with the capturing the second image of the face of the person.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
capturing audio information using an audio input component of the source device;
transmitting the audio information from the source device to the destination device; and
outputting the audio information using the destination device.

### Claim 9 (depends on 8)

The method of claim 8,
wherein the capturing the audio information, the transmitting the audio information, and the outputting the audio information are performed substantially in real-time with the capturing the second image of the face of the person.

### Claim 10 (depends on 1)

The method of claim 1, wherein the expression information comprises facial landmark indicators.

### Claim 11 (depends on 1)

The method of claim 1, wherein the expression information comprises a motion vector of facial landmark indicators.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
generating a second avatar that approximates the face of the person;
receiving a user input indicating a facial avatar to use; and
selecting the avatar based on the user input, wherein the selecting the avatar is performed prior to the transmitting the avatar to the destination device.

### Claim 13 (depends on 12)

The method of claim 12,
wherein the avatar is a photo-realistic avatar, and
wherein the second avatar is a generic avatar.

### Claim 14 (depends on 1)

The method of claim 1, wherein the real-time image of the face of the person is not transmitted from the source device to the destination device.

### Claim 15 (depends on 1)

The method of claim 1, further comprising:
receiving a user input to modify a visual aspect of the avatar; and
modifying a visual aspect of the avatar based on the received user input,
wherein the receiving the user input and the modifying the visual aspect of the avatar are performed prior to the transmitting the avatar to the destination device.

### Claim 16

A system comprising:
one or more source devices configured to:
capture a first image of a face of a person;
generate an avatar based on the first image of the face of the person, wherein the avatar approximates the first image of the face of the person;
transmit the avatar to a destination device;
capture a second image of the face of the person;
calculate expression information based on the second image of the face of the person, wherein the expression information approximates an expression on the face of the person as captured in the second image;
transmit the expression information to the destination device; and
the destination device configured to:
animate the avatar on a display component using the expression information,
wherein the source device is configured to transmit the expression information to the destination device after transmitting the avatar to the destination device.

### Claim 17 (depends on 16)

The system of claim 16,
wherein the one or more source devices are configured to capture the second image of the face of the person, calculate the expression information, and transmit the expression information substantially in real-time with the destination device animating the avatar.

### Claim 18 (depends on 16)

The system of claim 16,
wherein the one or more source devices are further configured to:
capture audio information using an audio input component; and
transmit the audio information to the destination device; and
wherein the destination device is further configured to:
output the audio information.

### Claim 19 (depends on 18)

The system of claim 18,
wherein the one or more source devices is configured to capture the second image of the face of the person, capture the audio information, and transmit the audio information substantially in real-time with the destination device outputting the audio information.

### Claim 20 (depends on 16)

The system of claim 16, wherein the expression information comprises facial landmark indicators.
