# US 2024/0005579 A1 · Representing two dimensional representations as three-dimensional avatars

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005579 A1
- Document ID: US20240005579A1
- Status: Published application
- Kind: A1
- Application number: 17/855,431
- Filed: 2022-06-30
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Mar GONZALEZ FRANCO; Payod PANDA; Andrew D. WILSON; Kori M. INKPEN; Eyal OFEK; William Arthur Stewart BUXTON
- Web page: https://patentgrep.com/patent/US20240005579A1

## CPC

- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- G06T 15/20: Perspective computation
- G06V 20/20: in augmented reality scenes
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Systems and methods for representing two-dimensional representations as three-dimensional avatars are provided herein. In some examples, one or more input video streams are received. A first subject, within the one or more input video streams, is identified. Based on the one or more input video streams, a first view of the first subject is identified. Based on the one or more input video streams, a second view of the first subject is identified. The first subject is segmented into a plurality of planar object. The plurality of planar objects are transformed with respect to each other. The plurality of planar objects are based on the first and second views of the first subject. The plurality of planar objects are output in an output video stream. The plurality of planar objects provide perspective of the first subject to one or more viewers.

## Claims

### Claim 1

A method of creating a virtual avatar, the method comprising:
receiving one or more input video streams;
identifying a first subject, within the one or more input video streams;
identifying, based on the one or more input video streams, a first view of the first subject;
identifying, based on the one or more input video streams, a second view of the first subject;
segmenting the first subject into a plurality of planar objects, the plurality of planar objects being transformed with respect to each other, and the plurality of planar objects being based on the first and second views of the first subject; and
outputting the plurality of planar objects in an output video stream, whereby the plurality of planar objects provide perspective of the first subject to one or more viewers.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of planar objects are billboards.

### Claim 3 (depends on 2)

The method of claim 2, wherein each of the billboards face a camera.

### Claim 4 (depends on 1)

The method of claim 1, wherein the planar objects are connected via virtual joints, and wherein the virtual joints are articular points of rotation.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first subject is identified within a room, and wherein the room is a hybrid meeting space.

### Claim 6 (depends on 5)

The method of claim 5, wherein the hybrid meeting space includes a plurality of participants disposed therein who are two or more from the group of: physically in the room, virtually displayed within the room, and represented within the room via a virtual avatar.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first subject is one of a person, an animal, or an object.

### Claim 8 (depends on 1)

The method of claim 1, wherein the first subject includes one or more pivot points at which two or more portions of the first subject are configured to rotate thereabout, and wherein the plurality of planar objects are segmented at the pivot points.

### Claim 9

A method of creating a virtual avatar, the method comprising:
receiving a plurality of input video streams;
identifying a first subject, within the plurality of input video streams;
identifying, based on the plurality of input video streams, a plurality of views of the first subject;
segmenting the first subject into a plurality of planar objects, the plurality of planar objects being transformed with respect to each other, and the plurality of planar objects being based on the plurality of views of the first subject; and
outputting the plurality of planar objects in an output video stream, whereby the plurality of planar objects provide perspective of the first subject to one or more viewers.

### Claim 10 (depends on 9)

The method of claim 9, wherein the plurality of planar objects are billboards.

### Claim 11 (depends on 10)

The method of claim 10, wherein each of the billboards face a camera.

### Claim 12 (depends on 9)

The method of claim 9, wherein the planar objects are connected via virtual joints, and wherein the virtual joints are articular points of rotation.

### Claim 13 (depends on 9)

The method of claim 9, wherein the first subject is identified within a room, and wherein the room is a hybrid meeting space.

### Claim 14 (depends on 13)

The method of claim 13, wherein the hybrid meeting space includes a plurality of participants disposed therein who are two or more from the group of: physically in the room, virtually displayed within the room, and represented within the room via a virtual avatar.

### Claim 15 (depends on 9)

The method of claim 9, wherein the first subject includes one or more pivot points at which two or more portions of the first subject are configured to rotate thereabout, and wherein the plurality of planar objects are segmented at the pivot points.

### Claim 16

A system, the system comprising:
a head-mounted display (HMD);
a display screen;
at least one processor;
memory storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations, the set of operations comprising:
receiving one or more input video streams;
identifying a first subject, within the one or more input video streams;
identifying a first view of the first subject, based on the one or more input video streams;
identifying a second view of the first subject, based on the one or more input video streams; and
outputting the first view of the first subject to the HMD; and
outputting the second view of the first subject to the display screen.

### Claim 17 (depends on 16)

The system of claim 16, wherein the HMD is located at a first location within a hybrid meeting space, and wherein the display screen is located at a second location within the hybrid meeting space.

### Claim 18 (depends on 17)

The system of claim 17, wherein the first view of the first subject corresponds to the first location, and wherein the second view of the first subject corresponds to the second location.

### Claim 19 (depends on 16)

The system of claim 16, wherein the HMD is one of an augmented-reality headset or a virtual-reality headset.

### Claim 20 (depends on 16)

The system of claim 16, wherein the one or more input video streams are received as part of a hybrid meeting.
