# US 2022/0021845 A1 · SpaeSee Video Chat System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0021845 A1
- Document ID: US20220021845A1
- Status: Published application
- Kind: A1
- Application number: 17/378,748
- Filed: 2021-07-18
- Published: 2022-01-20
- Earliest claimed: 2020-07-19
- Inventors: Daniel Schneider
- Web page: https://patentgrep.com/patent/US20220021845A1

## CPC

- H04M 3/568: {audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants (echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30)}
- G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
- H04L 12/1827: {Network arrangements for conference optimisation or adaptation}
- H04L 51/04: Real-time or near real-time messaging, e.g. instant messaging [IM]
- H04L 51/10: Multimedia information
- H04N 7/152: {Multipoint control units therefor}
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- H04L 51/222: using geographical location information, e.g. messages transmitted or received in proximity of a certain spot or area
- H04M 3/567: {Multimedia conference systems}
- H04M 2250/20: including a rotatable camera
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system and method for providing remote video communications, and more specifically, a system and method for providing multi-point-of-view video chatting for a group of participants is disclosed. The client device having a microphone, a video capture device having speakers, a display device, a memory having instructions stored thereon, and a processor configured to execute the instructions on the memory to cause the electronic client device to implement the method. The method receives other user location and field of view data for other users within the multi-point-of-view video chatting environment, determines a current location and field of view of a current user the client device within the multi-point-of-view video chatting environment, receives current user streaming video and audio data from the microphone and video capture device of a current user of the client device, transmits the current location and field of view of a current user and the streaming video and audio data from the current user to the network server, receives current location and field of view of each of other users and the streaming video and audio data from each of other user from the network server, generates a visual map of the multi-point-of-view video chatting environment containing an avatar and a puck for the current user and each of the other users, generates a current user audio stream corresponding to a combination of all of the streaming audio data from the other users, and displays the visual map to a the current user and play the combination of all of the streaming audio data from the other users on the display device. The puck associated with each user is rendered at a position within the visual map and an orientation associated with the field of view associated with the corresponding user.

## Claims

### Claim 1

A system for providing multi-point-of-view video chatting environment for a group of participants over a computer network, the computer network being configured to communicatively interconnect a network server and a plurality of client devices over the Internet, the client device comprises:
a microphone;
a video capture device having speakers;
a display device;
a memory having instructions stored thereon; and
a processor configured to execute the instructions on the memory to cause the electronic client device to:
receive other user location and field of view data for other users within the multi-point-of-view video chatting environment;
determine a current location and field of view of a current user the client device within the multi-point-of-view video chatting environment;
receive current user streaming video and audio data from the microphone and video capture device of a current user of the client device;
transmit the current location and field of view of a current user and the streaming video and audio data from the current user to the network server;
receive current location and field of view of each of other users and the streaming video and audio data from each of other user from the network server;
generate a visual map of the multi-point-of-view video chatting environment containing an avatar and a puck for the current user and each of the other users, the puck associated with each user is rendered at a position within the visual map and an orientation associated with the field of view associated with the corresponding user;
generate a current user audio stream corresponding to a combination of all of the streaming audio data from the other users; and
display the visual map to a the current user and play the combination of all of the streaming audio data from the other users on the display device.

### Claim 2 (depends on 1)

The system according to claim 1, wherein the avatar for each of the other users corresponds to an image of the corresponding user.

### Claim 3 (depends on 1)

The system according to claim 1, wherein the avatar for each of the other users corresponds to streaming video data received from each of the other users.

### Claim 4 (depends on 1)

The system according to claim 1, wherein the combination of all of the streaming audio data from the other users utilizes a distance of the current user location to each of the other users locations and an orientation of the field of view for each of the other users to the current user when combining the streaming audio data.

### Claim 5 (depends on 4)

The system according to claim 4, wherein the distance of the current user to each of the other users includes volume of each of the respective streaming audio data based upon a relative difference and orientation in the distances of each of the other users to the current user.

### Claim 6 (depends on 5)

The system according to claim 5, wherein the volume of each of the streaming audio data is reduced when an obstacle is between the current user and each of the other users providing s source of the respective streaming audio data when combined with each of the other users streaming audio data.

### Claim 7 (depends on 1)

The system according to claim 1, wherein the client device receives a move input command to move the current location of the current user within the visual map of the multi-point-of-view video chatting environment.

### Claim 8 (depends on 7)

The system according to claim 7, where the client device further receives a rotate input command to alter the field of view of the current user within the visual map of the multi-point-of-view video chatting environment.

### Claim 9

A method for providing multi-point-of-view video chatting environment for a group of participants over a computer network, the computer network being configured to communicatively interconnect a network server and a plurality of client devices over the Internet, the client device having a microphone, a video capture device having speakers, a display device, a memory having instructions stored thereon, and a processor configured to execute the instructions on the memory to cause the electronic client device to implement the method, the method comprising:
receiving other user location and field of view data for other users within the multi-point-of-view video chatting environment;
determining a current location and field of view of a current user the client device within the multi-point-of-view video chatting environment;
receiving current user streaming video and audio data from the microphone and video capture device of a current user of the client device;
transmitting the current location and field of view of a current user and the streaming video and audio data from the current user to the network server;
receiving current location and field of view of each of other users and the streaming video and audio data from each of other user from the network server;
generating a visual map of the multi-point-of-view video chatting environment containing an avatar and a puck for the current user and each of the other users, the puck associated with each user is rendered at a position within the visual map and an orientation associated with the field of view associated with the corresponding user;
generating a current user audio stream corresponding to a combination of all of the streaming audio data from the other users; and
displaying the visual map to a the current user and play the combination of all of the streaming audio data from the other users on the display device.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the avatar for each of the other users corresponds to an image of the corresponding user.

### Claim 11 (depends on 9)

The method according to claim 9, wherein the avatar for each of the other users corresponds to streaming video data received from each of the other users.

### Claim 12 (depends on 9)

The method according to claim 9, wherein the combination of all of the streaming audio data from the other users utilizes a distance of the current user location to each of the other users locations and an orientation of the field of view for each of the other users to the current user when combining the streaming audio data.

### Claim 13 (depends on 12)

The method according to claim 12, wherein the distance of the current user to each of the other users includes volume of each of the respective streaming audio data based upon a relative difference in the distances of each of the other users to the current user.

### Claim 14 (depends on 13)

The method according to claim 13, wherein the volume of each of the streaming audio data is reduced when an obstacle is between the current user and each of the other users providing s source of the respective streaming audio data when combined with each of the other users streaming audio data.

### Claim 15 (depends on 9)

The method according to claim 9, wherein the client device receives a move input command to move the current location of the current user within the visual map of the multi-point-of-view video chatting environment.

### Claim 16 (depends on 15)

The method according to claim 15, where the client device further receives a rotate input command to alter the field of view of the current user within the visual map of the multi-point-of-view video chatting environment.

### Claim 17

A non-transitory computer-readable recording medium for providing multi-point-of-view video chatting environment for a group of participants over a computer network, the computer network being configured to communicatively interconnect a network server and a plurality of client devices over the Internet, the non-transitory computer-readable recording medium storing one or more programs which when executed by a client device performs steps comprising:
receiving other user location and field of view data for other users within the multi-point-of-view video chatting environment;
determining a current location and field of view of a current user the client device within the multi-point-of-view video chatting environment;
receiving current user streaming video and audio data from the microphone and video capture device of a current user of the client device;
transmitting the current location and field of view of a current user and the streaming video and audio data from the current user to the network server;
receiving current location and field of view of each of other users and the streaming video and audio data from each of other user from the network server;
generating a visual map of the multi-point-of-view video chatting environment containing an avatar and a puck for the current user and each of the other users, the puck associated with each user is rendered at a position within the visual map and an orientation associated with the field of view associated with the corresponding user;
generating a current user audio stream corresponding to a combination of all of the streaming audio data from the other users; and
displaying the visual map to a the current user and play the combination of all of the streaming audio data from the other users on the display device.

### Claim 18 (depends on 16)

The non-transitory computer-readable recording medium according to claim 16, wherein the combination of all of the streaming audio data from the other users utilizes a distance of the current user location to each of the other users locations and an orientation of the field of view for each of the other users to the current user when combining the streaming audio data, the distance of the current user to each of the other users includes volume of each of the respective streaming audio data based upon a relative difference in the distances of each of the other users to the current user.

### Claim 19 (depends on 18)

The non-transitory computer-readable recording medium according to claim 18, wherein the volume of each of the streaming audio data is reduced when an obstacle is between the current user and each of the other users providing s source of the respective streaming audio data when combined with each of the other users streaming audio data.

### Claim 20 (depends on 19)

The non-transitory computer-readable recording medium according to claim 19, the client device receives a move input command to move the current location of the current user within the visual map of the multi-point-of-view video chatting environment and receives a rotate input command to alter the field of view of the current user within the visual map of the multi-point-of-view video chatting environment.
