# US 2024/0203075 A1 · Collaborative system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203075 A1
- Document ID: US20240203075A1
- Status: Published application
- Kind: A1
- Application number: 18/065,987
- Filed: 2022-12-14
- Published: 2024-06-20
- Earliest claimed: 2022-12-14
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Edward Sean Lloyd RINTEL; Payod PANDA; Lev TANKELEVITCH; Abigail Jane SELLEN; Kori Marie INKPEN; John C. TANG; Sasa JUNUZOVIC; Andrew D. WILSON; Bo KANG; Andriana BOUDOURAKI; William Arthur Stewart BUXTON; Ozumcan DEMIR CALISKAN; Kunal GUPTA
- Web page: https://patentgrep.com/patent/US20240203075A1

## CPC

- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation}
- H04N 5/265: Mixing
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2219/024: Multi-user, collaborative environment
- G06T 2219/2016: Rotation, translation, scaling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A computer-implemented method is described which comprises generating a representation of a digital space and a representation of the physical space using an audiovisual feed received from a camera proximate to a display located in the physical space. The representation of the digital space is generated using user information identifying a remote user associated with the display and presence information relating to the remote user and the digital representation comprises an avatar of the remote user. The representation of the digital space is output to the display located in the physical space and the representation of the physical space it output to a computing device associated with the remote user. The method further comprises dynamically updating the representation of the digital space and/or physical space in response to changes in the user information and presence information.

## Claims

### Claim 1

A computer-implemented method comprising:
generating a representation of a digital space, wherein the representation is generated using user information identifying a remote user associated with a display located in a physical space and using presence information relating to the remote user and the digital representation comprising an avatar of the remote user;
generating a representation of the physical space using an audiovisual feed received from a camera proximate to the display located in the physical space;
outputting the representation of the digital space to the display located in the physical space;
outputting the representation of the physical space to a computing device associated with the remote user; and
dynamically updating the representation of the digital space and/or physical space in response to changes in the user information and presence information.

### Claim 2 (depends on 1)

The method according to claim 1, wherein dynamically updating the representation of the digital space in response to changes in the user information and presence information comprises dynamically resizing the digital space in response changes in numbers of remote users associated with the display.

### Claim 3 (depends on 1)

The method according to claim 1, wherein dynamically updating the representation of the digital space in response to changes in the user information and presence information comprises dynamically updating the avatar of the remote user in response to a change in presence information relating to the remote user.

### Claim 4 (depends on 1)

The method according to claim 1, wherein generating a representation of the physical space using an audiovisual feed received from a camera proximate to the display located in the physical space comprises processing the audiovisual feed to (i) remove audio data or (ii) filter the audio data to remove a band of frequencies from the audio data.

### Claim 5 (depends on 1)

The method according to claim 1, wherein generating a representation of the physical space using an audiovisual feed received from a camera proximate to the display located in the physical space comprises:
analyzing the audiovisual feed to detect a user action; and
in response to detecting a user action, adding a sound effect corresponding to the user action.

### Claim 6 (depends on 1)

The method according to claim 1, wherein generating a representation of the physical space using an audiovisual feed received from a camera proximate to the display located in the physical space comprises processing the audiovisual feed to replace a user detected in the audiovisual feed with an avatar.

### Claim 7 (depends on 1)

The method according to claim 1, wherein generating a representation of the physical space using an audiovisual feed received from a camera proximate to the display located in the physical space comprises processing the audiovisual feed to obscure a user detected in the audiovisual feed.

### Claim 8 (depends on 1)

The method according to claim 1, further comprising setting a proximity parameter of the computing device associated with the remote user.

### Claim 9 (depends on 8)

The method according to claim 8, further comprising updating the proximity parameter of the computing device in response to detecting user interaction by the remote user with the representation of the physical space.

### Claim 10 (depends on 8)

The method according to claim 8, wherein generating a representation of the physical space using an audiovisual feed received from a camera proximate to the display located in the physical space comprises:
generating a representation of the physical space using the audiovisual feed and the proximity parameter.

### Claim 11 (depends on 8)

The method of claim 8, wherein generating a representation of the physical space using the audiovisual feed and the proximity parameter comprises adjusting a field of view of the representation in response to the proximity parameter.

### Claim 12 (depends on 8)

The method according to claim 8, wherein generating a representation of the physical space using an audiovisual feed received from a camera proximate to the display located in the physical space comprises generating a plurality of representations of the physical space using the audiovisual feed, each of the representations of the physical space corresponding to a different value of the proximity parameter, and
wherein outputting the representation of the physical space to a computing device associated with the remote user comprises selecting one of the plurality of representations using the determined proximity parameter of the computing device and outputting the selected representation of the physical space to the computing device.

### Claim 13 (depends on 8)

The method according to claim 8, further comprising:
in response to detecting an on-site user in proximity to the display located in the physical space and that the proximity parameter of the computing device associated with the remote user satisfies predefined proximity criteria:
stopping outputting both the representation of the digital space to the display and the representation of the space to the computing device associated with the remote user; and
establishing a live video call between the display and the computing device associated with the remote user.

### Claim 14 (depends on 13)

The method according to claim 13, further comprising, in response to detecting an on-site user in proximity to the display located in the physical space and that the proximity parameter of the computing device associated with the remote user satisfies predefined proximity criteria:
outputting the representation of the space to the computing device associated with a second remote user, wherein the user information identifies the second remote user as associated with the display located in the physical space and the proximity parameter of the computing device associated with the second remote user has a proximity parameter that does not satisfy the predefined proximity criteria.

### Claim 15 (depends on 1)

The method according to claim 1, wherein the representation of the physical space generated using an audiovisual feed received from a camera proximate to the display located in the physical space is a first representation of the physical space and the method further comprising:
generating a second representation of the physical space using an audiovisual feed received from a second camera proximate to a second display located in the physical space;
and wherein outputting the representation of the physical space to a computing device associated with the remote user comprises:
outputting a user interface to the computing device associated with the remote user, wherein the user interface comprises the first and second representations displayed in an orientation corresponding to a relative orientation of the display and the second display.

### Claim 16

A system comprising:
a computing device comprising a processor and computer storage memory having computer-executable instructions stored thereon which, when executed by the processor, implement a method comprising:
generating a representation of a digital space, wherein the representation is generated using user information identifying a remote user associated with a display located in a physical space and presence information relating to the remote user and the digital representation comprising an avatar of the remote user; and
generating a representation of the physical space using an audiovisual feed received from a camera proximate to the display located in the physical space;
outputting the representation of the digital space to the display located in the physical space;
outputting the representation of the physical space to a computing device associated with the remote user; and
dynamically updating the representation of the digital space and/or physical space in response to changes in the user information and presence information.

### Claim 17 (depends on 16)

The system according to claim 16, further comprising:
the display located in the physical space; and
a camera, proximate to the display and configured to generate the audiovisual feed.

### Claim 18

A method of operating a system, the method comprising:
in a first mode of operation:
generating a representation of a digital space, wherein the representation is generated using user information identifying a remote user associated with a display located in a physical space and using presence information relating to the remote user and the digital representation comprising an avatar of the remote user;
generating a representation of the physical space using an audiovisual feed received from a camera proximate to the display located in the physical space;
outputting the representation of the digital space to the display located in the physical space;
outputting the representation of the physical space to a computing device associated with the remote user; and
dynamically updating the representation of the digital space and/or physical space in response to changes in the user information and presence information,
and in a second mode of operation:
establishing a live video call between the display and the computing device associated with the remote user,
and wherein the method further comprises:
switching from the first mode of operation to the second mode of operation in response to detecting an on-site user in proximity to the display located in the physical space and that a proximity parameter of the computing device associated with the remote user satisfies predefined proximity criteria.

### Claim 19 (depends on 18)

The method according to claim 18, further comprising setting a proximity parameter of the computing device associated with the remote user.

### Claim 20 (depends on 19)

The method according to claim 19, further comprising updating the proximity parameter of the computing device in response to detecting user interaction by the remote user with the representation of the physical space.
