US 5,736,982 A · Virtual space apparatus with avatars and speech

A virtual space sharing apparatus which has a plurality of terminals connected to a communication network and sharing a predetermined common virtual space and generates a displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each terminal comprising:

control means which generates signals for selectively specifying its position and direction of eyes in said virtual space;
visual field image generating means which generates a visual field image in said direction of eyes in said virtual space from said position as a viewing point;
position information sending and receiving means which sends said position and said direction of eyes as position information to said communication network and receives therefrom position information sent from other terminals;
avatar image forming means which forms avatar images representative of users of said other terminals in said visual field at positions corresponding to said received position information;
display means which displays a combined image containing said visual field image and said avatar images;
and
server means connected to said each terminal via said communication network, said each terminal comprising speech sending and receiving means for sending speech data of its user to said server means via said communication network and for receiving speech data of said users of said other terminals from said server means and speech output means for outputting said received speech data as speech;
said server means comprising:
select means which calculates, from position information received from said terminals, the distances between said avatar of the user of said each terminal and other avatars and selects those of said other avatars which have said distance within a predetermined threshold value;
and
mixer means which, in a group consisting of any one of said avatars and said avatars selected by said select means relative thereto, mixes speech data from the terminals corresponding to said avatars except each particular one and sends said mixed speech data to the terminal corresponding to said each particular avatar.

8 more elements

A virtual space sharing apparatus which has a plurality of terminals connected to a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each terminal comprising:

control means which generates signals for selectively specifying its position and direction in eyes in said virtual space;
visual field image generating means which generates a visual field image in said direction of eyes in said virtual space from said position as a viewing point;
position information sending and receiving means which sends said position and said direction of eyes as position information to said communication network and receives therefrom position information sent from other terminals;
avatar image forming means which forms avatar images representative of users of said other terminals in said visual field at positions corresponding to said received position information;
and
display means which displays a combined image containing said visual field image and said avatar images;
said terminals being interconnected via said communication network, and each of said terminals comprising:
speech sending and receiving means which sends speech data of its user to all the other terminals via said communication network and receives therethrough speech data of users of said other terminals;
select means which calculates, from position information received from said other terminals, the distances between the avatar of the user of said each terminal and said other avatars and selects those of said other avatars which have said distance within a predetermined threshold value;
mixer means which mixes speech data received from the terminals corresponding to said avatars selected by said select means and outputs the mixed speech data;
and
speech output means for outputting said mixed speech data as a sound.

9 more elements

A virtual space sharing apparatus which has a plurality of terminals connected to a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each terminal comprising:

control means which generates signals for selectively specifying its position and direction of eyes in said virtual space;
visual field image generating means which generates a visual field image in said direction of eyes in said virtual space from said position as a viewing point;
position information sending and receiving means which sends said position and said direction of eyes as position information to said communication network and receives therefrom position information sent from other terminals;
avatar image forming means which forms avatar images representative of users of said other terminals in said visual field at positions corresponding to said received position information;
display means which displays a combined image containing said visual field image and said avatar images;
and
server means connected to said each terminal via said communication network, said each terminal comprising speech sending and receiving means for sending speech data of its user to said server means via said communication network and for receiving speech data of said users of said other terminals from said server means and speech output means for outputting said received speech data as speech;
said server means comprising:
first mixer means which mixes speech data received from terminals corresponding to all of said avatars and outputs environment sound data;
conversation monitor means which, on the basis of position information received from each of said terminals, searches for a group of avatars which mutually satisfy a conversation enable condition;
second mixer means which generates, for the terminal of each avatar or said group, mixed sound data by mixing speech data received from the terminal corresponding to the other avatar of said group and said environment sound data;
and
means which sends said mixed sound data to said terminal of said each avatar.

10 more elements

A virtual space sharing apparatus which has a plurality of terminals connected to a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each terminal comprising:

control means which generates signals for selectively specifying its position and direction of eyes in said virtual space;
visual field image generating means which generates a visual field image in said direction of eyes in said virtual space from said position as a viewing point;
position information sending and receiving means which sends said position and said direction of eyes as position information to said communication network and receives therefrom position information sent from other terminals;
avatar image forming means which forms images representative of users of said other terminals in said visual field at positions corresponding to said received position information;
and
display means which displays a combined image containing said visual field image and said avatar images;
said terminals being interconnected via said communication network and each of said terminals comprising:
speech sending and receiving means which sends speech data of its user to all the other terminals via said communication network and receives therethrough speech data of users of said other terminals;
conversation monitor means which, on the basis of position information received from said other terminals, searches for a group of avatars which satisfy a conversation enable condition with respect to the avatar of the user of said each terminal;
first mixer means which mixes speech data received from terminals corresponding to all of said avatars and outputs environment sound data;
second mixer means which mixes speech data received from the terminals corresponding to said avatars of said group and said environment sound data to generate mixed speech data;
and
speech output means for outputting said mixed speech data as a sound.

10 more elements

A display method for a virtual space which has a plurality of terminals connected to a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each terminal performing the steps of:

(a) generating signals for selectively specifying its position and direction of eyes in said virtual space;
(b) sending said position and said direction of eyes as position information to said communication network;
(c) generating a visual field image in said direction of eyes in said virtual space from said position as a viewing point;
(d) receiving, from said communication network, position information sent thereto from other terminals;
(e) forming avatar images representative of users of said other terminals in said visual field at positions corresponding to said received position information;
and
(f) displaying a combined image containing said visual field image and said avatar images;
server means being provided which is connected via said communication network to said terminals, respectively, said step (b) including a step of sending speech data of the user of said each terminal to said server means via said communication network, and said step (d) including a step of receiving speech data of other terminals from said server means and outputting said received speech data as speech;
said server means performing the steps of:
(g) calculating, from position information received from said other terminals, the distances between said avatar of the user of said each terminal and the avatars of said other terminals and selecting those of said avatars of said other terminals which have said distance within a predetermined threshold value;
and
(h) mixing, in a group consisting of any one of said avatars and said avatars selected relative thereto, speech data from the terminals corresponding to said avatars except each particular one and sending said mixed speech data to the terminal corresponding to said each particular avatar.

9 more elements

A display method for a virtual space which has a plurality of terminals connected to a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each terminal performing the steps of:

(a) generating signals for selectively specifying its position and direction of eyes in said virtual space;
(b) sending said position and said direction of eyes as position information to said communication network;
(c) generating a visual field image in said direction of eyes in said virtual space from said position as a viewing point;
(d) receiving, from said communication network, position information sent thereto from other terminals;
(e) forming avatar images representative of users of said other terminals in said visual field at positions corresponding to said received position information;
and
(f) displaying a combined image containing said visual field image and said avatar images;
said terminals being interconnected via said communication network, said step (b) including a step of sending speech data of the user of said each terminal to all the other terminals via said communication network and said step (d) including a step of receiving therethrough speech data of the users of said other terminals via said communication network;
said each terminal performing the steps of:
(g) selecting, from position information received from said other terminals, the distances between the avatar of the user of said each terminal and the avatars of said other terminals and selecting those of the avatars of said other terminals which have said distance within a predetermined threshold value;
and
(h) mixing speech data received from the terminals corresponding to said selected avatars and outputting the mixed speech data as a sound.

9 more elements

A display method for a virtual space which has a plurality of terminals connected to a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each terminal performing the steps of:

(a) generating signals for selectively specifying its position and direction of eyes in said virtual space;
(b) sending said position and said direction of eyes as position information to said communication network;
(c) generating a visual field image in said direction of eyes in said virtual space from said position as a viewing point;
(d) receiving, from said communication network, position information sent thereto from other terminals;
(e) forming avatar images representative of users of said other terminals in said visual field at positions corresponding to said received position information;
and
(f) displaying a combined image containing said visual field image and said avatar images;
wherein server means is provided which is connected via said communication network to said terminals, respectively, said step (b) including a step of sending speech data of the user of said each terminal to said server means via said communication network, and said step (d) including a step of receiving speech data of other terminals from said server means and outputting said received speech data as speech;
said server means performing the steps of:
(g) mixing speech data received from terminals corresponding to said avatars to generate environment sound data;
(h) searching for a group of avatars which mutually satisfy a conversation enable condition on the basis of the position information received from said other terminals;
(i) generating mixed sound data by mixing speech data received from the terminals corresponding to the avatars of said group other than each one of them and said environment sound data;
and
(j) sending said mixed sound data, generated for the terminal of said each avatar, to said terminal.

11 more elements

A display method for a virtual space which has a plurality of terminals connected to a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each terminal performing the steps of:

(a) generating signals for selectively specifying its position and direction of eyes in said virtual space;
(b) sending said position and said direction of eyes as position information to said communication network;
(c) generating a visual field image in said direction of eyes in said virtual space from said position as a viewing point;
(d) receiving, from said communication network, position information sent thereto from other terminals;
(e) forming avatar images representative of users of said other terminals in said visual field at positions corresponding to said received position information;
and
(f) displaying a combined image containing said visual field image and said avatar images;
wherein said terminals are interconnected via said communication network, said step (b) including a step of sending speech data of the user of said each terminal to all the other terminals via said communication network and said step (d) includes a step of receiving speech data of the users of said other terminals via said communication network;
said each terminal performing the steps of:
(g) mixing speech data received from terminals corresponding to said avatars to generate environment sound data;
(h) searching for a group of avatars which mutually satisfy a conversation enable condition on the basis of the position information received from said other terminals;
(i) generating mixed sound data by mixing speech data received from the terminals corresponding to the avatars of said group other than each one of them and said environment sound data and
(j) outputting said mixed sound data as a sound.

10 more elements

A virtual space sharing apparatus which has a plurality of terminals connected to a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each terminal comprising:

sending and receiving means which receives speech data of users of said terminals except said each terminal and position information of their avatars and sends speech data of the user of said each terminal and position information of its avatar to said terminals;
speech quality determining means which determines the levels of quality for speech data of other users in accordance with the relationship of the avatars of said other users to the avatar of said each user through the use of position information of said avatars of said other users received from said other terminals;
speech quality control means which controls the quality of speech data of said other users in accordance with the levels of quality determined therefor relative to the avatar of said each user;
mixer means which mixes said quality-controlled speech data of said other users in correspondence with said each user and outputs mixed sound data;
and
acoustic signal output means which outputs said mixed sound data from said mixer means as an acoustic signal.

3 more elements

A virtual space sharing apparatus which has a plurality of terminals connected to a server and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said server comprising:

sending and receiving means which receives speech data of users of said terminals except said each terminal and position information of their avatars and sends speech data of the user of said each terminal and position information of its avatar to said terminals;
speech and position information distributing means which distributes speech data of the user and position information of its avatar, received from said each terminal, to all the other terminals via said sending and receiving means;
speech quality determining means which determines the speech quality for speech data of other users in accordance with the relationship of the avatars of said other users to the avatar of said each user through the use of position information of said avatars of said other users received from said other terminals;
speech quality control means which controls the speech quality of speech data of said other users in accordance with the speech quality determined therefor relative to the avatar of said each user;
and
mixer means which mixes said quality-controlled speech data of said other users in correspondence with said each user and outputs mixed sound data;
wherein said mixed sound data is sent via said sending and receiving means to corresponding ones of said terminals.

4 more elements

A virtual space sharing apparatus which has a plurality of terminals connected to a server and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal,

wherein said each terminal comprises:
sending and receiving means which receives video image data of each of other users and position information of its avatar from said server means and sends video image data of the user of said each terminal and position information of its avatar to said server;
camera means which picks up the video image of said user of said each terminal and outputs a video signal;
digital processing means which digitally process, said video signal and sends it to said server means via said sending and receiving means;
quality specifying and video requesting means which determines the image quality for the avatar of each of said other users on the basis of the relationship between the position information of the avatar of each of said other users and the position information of the avatar of said each user and sends via said sending and receiving means to said server means a video request signal requesting a video image of said determined quality;
and means which generates an avatar image on the basis of the video image data of each of said other users received from said server means and displays it in a visual field image of said each user at a position specified by the position information of the avatar of each of said other users;
and
wherein said server means comprises:
position information distributing means which sends position information of the avatar of the user, received from said each terminal, to all the other terminals;
video memory means which stores video image data in correspondence with said terminals;
means which writes received video image data into said video memory means;
and quality processing means which analyzes a video image request signal received from each of said other users, then reads out the requested video image data from said video memory means and sends it to a requesting terminal after processing it to have specified quality.

9 more elements

A virtual space sharing apparatus which has a plurality of terminals connected to a server and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal,

said each terminal comprising:
sending and receiving means which receives video image data of the users of other terminals and position information of their avatars and sends to said other terminals the video image of the user of said each terminal and position information of its avatar;
quality specifying and video requesting means which determines the image quality for the avatar of each of said other users on the basis of the relationship between the position information of the avatar received from each of said other terminals and the position information of the avatar of said each terminal and sends via said sending and receiving means to each of said other terminals a video request signal requesting a video image of said determined quality;
video memory means which stores video data;
camera means which picks up the video image of the user of said each terminal;
video processing means which digitally processes said video signal and writes it into said video memory means;
means which analyzes a video image request signal received from each of said other users, then reads out the requested video image data from said video memory means and sends it to a requesting terminal after processing it to have specified quality;
and means which generates an avatar image on the basis of the video image data received from each of said other terminals and displays it in a visual field image of said each user at a position specified by the position information received from each of said other terminals.

5 more elements

A virtual space sharing apparatus which has a plurality of terminals connected to a server and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal,

wherein said each terminal comprises:
sending and receiving means which receives video image data of each of other users and position information of its avatar from said server means and sends video image data of the user of said each terminal and position information of its avatar to said server;
camera means which picks up the video image of said user of said each terminal and outputs a video signal;
digital processing means which digitally processes said video signal and sends it to said server means via said sending and receiving means;
and means which generates an avatar image on the basis of the video image data of each of said other users received from said server means and displays it in a visual field image of said avatar of the user of said each terminal at a position specified by the position information of the avatar of each of said other users;
and
wherein said server means comprises:
position information distributing means which sends position information of the avatar of the user of said each terminal received therefrom to all the other terminals;
image quality determining means which determines the image quality for the avatar of the user of each of said other terminals on the basis of the relationship between the position information of the avatar of the user of said each terminal and the position information of the avatar of the user of each of said other users;
video memory means which stores video image data in correspondence with said terminals;
means which writes received video image data into said video memory means;
and quality processing means which reads out of said video memory means said video image data of each of said terminals received therefrom, and sends it to a requesting terminal after processing it to have said determined quality.

9 more elements

A virtual space sharing apparatus which has a plurality of terminals connected to a server and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal,

said each terminal comprising:
sending and receiving means which receives video image data of the users of other terminals and position information of their avatars and sends to said other terminals the video image of the user of said each terminal and position information of its avatar;
image quality determining means which determines the image quality for the avatar of the user of said each terminal relative to the avatar of each of said other users on the basis of the relationship between the position information of the avatar received from each of said other terminals and the position information of the avatar of the user of said each terminal and sends via said sending and receiving means to each of said other terminals video image data of said determined quality;
camera means which picks up the video image of the user of said each terminal and outputs a video signal;
video memory means which stores said video signal;
video processing means which digitally processes said video signal and writes it into said video memory means;
digital processing means which processes video image data of each of said other users read out of said video memory means to have said determined quality and sends it via said sending and receiving means to each of said other terminals;
and means which generates an avatar image on the basis of the video image data received from each of said other terminals and displays it in a visual field image of the avatar of said each terminal at a position specified by the position information received from each of said other terminals.

5 more elements

A display method for a virtual space which has a plurality of terminals connected to a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each terminal performing the steps of:

(a) receiving speech data of the users and position information of their avatars from the other terminals;
(b) sending speech data of the avatar of said each terminal and position information of its avatar to each of said other terminals;
(c) determining the levels of quality for speech data of the users of said other terminals in accordance with the relationship of the avatars of the users of said other terminals to the avatar of the user of said each terminal through the use of position information of said avatars of said other users received from said other terminals;
(d) controlling the quality of speech data of said other users in accordance with the levels of quality determined therefor relative to the avatar of said each user;
and
(e) mixing said quality-controlled speech data of said other users in correspondence with said each user and outputting mixed sound data as acoustic signal.

3 more elements

A display method for a virtual space which has a plurality of terminals connected to server means and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal, said each server means performing the steps of:

(a) receiving speech data of the users and position information of their avatars from the other terminals;
(b) sending to each terminal speech data and position information of their avatars;
(c) distributing the speech data of the user and position information of its avatar received from said each terminal to all the other terminals;
(d) determining the levels of quality for speech data of the users of said other terminals in accordance with the relationship of the avatars of the users of said other terminals to the avatar of the user of said each terminal through the use of position information of said avatars of said other users received from said other terminals;
(e) controlling the quality of speech data of said other users in accordance with the levels of quality determined therefor relative to the avatar of said each user;
and
(f) mixing said quality-controlled speech data of said other users in correspondence with said each user and outputting and sending mixed sound data to said terminals corresponding thereto.

4 more elements

A display method for a virtual space which has a plurality of terminals connected to a server via a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal,

wherein said each terminal performs the steps of:
(a) picking up the video image of the user of said each terminal, digitally processing the video signal and sending the video image data of said user to said server;
(b) sending position information of the avatar of said each terminal to said server;
(c) receiving position information of the avatar of each user from said server;
(d) determining the image quality for the avatar of each of said other users on the basis of the relationship between the position information of the avatar of each of said other users and the position information of the avatar of said each user;
(e) sending to said server a video request signal requesting a video image of said determined quality;
and
(f) generating an avatar image on the basis of the video image data of each of said other users received from said server and displaying it in a visual field image of said each user at the position specified by the position information of the avatar of each of said other users;
and
wherein said server performs the steps of:
(g) writing video image data received from each terminal in correspondence therewith;
(h) sending position information of the avatar of the user received from each terminal to all the other terminals;
and
(i) analyzing a video image request signal received from each of said other users, then reading out the requested video image data from said video memory means and sending it to the requesting terminal after processing it to have specified quality.

11 more elements

A display method for a virtual space which has a plurality of terminals interconnected via a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal,

said each terminal performing the steps of:
(a) picking up a video signal representing the video image of the user of said each terminal, digitally processing the video signal and writing it in video memory means;
(b) sending position information of the avatar of said each terminal to the other terminals;
(c) receiving position information of the avatar of the user of each of said other terminals;
(d) determining the image quality for the avatar of each of said other users on the basis of the relationship between the position information of the avatar of each of said other users and the position information of the avatar of said each user;
(e) sending to each of said other terminals a video request signal requesting a video image of said determined quality;
(f) receiving the video request signal from each of said other terminals;
(g) sending video data of a user read out of said video image memory to the requesting terminals after processing it to have quality specified by said video request signal;
and
(h) generating an avatar image on the basis of the video image data of each of said other users received therefrom and displaying it in a visual field image of said each user at the position specified by the position information of the avatar of each of said other users.

7 more elements

A display method for a virtual space which has a plurality of terminals connected to a server via a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal,

said each terminal performing the steps of:
(a) picking up video signal representing the video image of the user of said each terminal, digitally processing the video signal and sending the video image data of said user to said server;
(b) sending video image data of said each terminal and position information of its avatar to said server;
(c) receiving video image data of the other users and position information of their avatars from said server;
and
(d) generating an avatar image on the basis of the video image data of each of said other users received from said server and displaying it in a visual field image of said each user at the position specified by the position information of the avatar of each of said other users;
and
said server performing the steps of:
(e) storing video image data received from each terminal in correspondence therewith;
(f) sending position information of the avatar of the user received from each terminal to all the other terminals;
(g) determining the image quality for the avatar of the user of each of said other users on the basis of the relationship between the position information of the avatar of each user and the position information of the avatar of each of said other users;
and
(h) reading out the video image data of each of said other terminals from said video memory means and sending it to each of said other terminals after processing it to have said determined quality.

10 more elements

A display method for a virtual space which has a plurality of terminals interconnected via a communication network and sharing a predetermined common virtual space and generates and displays a visual field image which changes as an avatar representing a user of each terminal moves in said virtual space at said each terminal,

said each terminal performing the steps of:
(a) picking up the video image of the user of said each terminal and writing the video signal in video memory means;
(b) sending position information of the avatar of said each terminal to the other terminals;
(c) receiving position information of the avatar of the user of each of said other terminals;
(d) determining the image quality for the avatar of each of said other users on the basis of the relationship between the position information of the avatar of each of said other users and the position information of the avatar of said each user;
(e) reading out the video image data of the user from said video memory and sending it to each of said other terminals after processing it to have said determined quality;
and
(f) generating an avatar image on the basis of the video image data of each of said other users received therefrom and displaying it in a visual field image of said each user at the position specified by the position information of the avatar of each of said other users.

5 more elements

Virtual space apparatus with avatars and speech

Life of this patent

Virtual space apparatus with avatars and speech

Life of this patent

Abstract

Claims61