Method and Apparatus for Providing Hybrid Reality Environment

A server device configured to generate a hybrid reality environment of real and virtual objects, comprising:

• one or more processors;
• and
• memory storing one or more programs to be executed by the one or more processors, the one or more programs comprising instructions for:
• receiving data of a first group of users located at a first physical location;
• receiving data of a second group of users, each user from the second group of users being located at a physical location different from the first physical location;
• generating, for each user from the first group of users, a scene of an augmented reality (AR) based environment, the AR based environment including the first group of users and one or more virtual objects related to the second group of users that are generated based on the data of the second group of users;
• generating, for each user from the second group of users, a scene of an augmented virtuality (AV) based environment, the AV based environment including the one or more virtual objects related to the second group of users and one or more virtual objects related to the first group of users that are generated based on the data of the first group of users;
• sending signals for displaying the scene of the AR based environment to a user from the first group of users such that the user can interact with the one or more virtual objects related to the second group of users in real time within the AR based environment;
• and
• sending signals for displaying the scene of the AV based environment to a user from the second group of users such that the user can manipulate a virtual object related to that user to interact with the one or more virtual objects related to the first group of users in real time within the AV based environment.

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by one or more processors, cause the processors to perform operations comprising:

• at a server device:
• receiving data of a first group of users located at a first physical location;
• receiving data of a second group of users, each user from the second group of users being located at a physical location different from the first physical location;
• generating, for each user from the first group of users, a scene of an augmented reality (AR) based environment, the AR based environment including the first group of users and one or more virtual objects related to the second group of users that are generated based on the data of the second group of users;
• generating, for each user from the second group of users, a scene of an augmented virtuality (AV) based environment, the augmented virtuality based environment including the one or more virtual objects related to the second group of users and one or more virtual objects related to the first group of users that are generated based on the data of the first group of users;
• sending signals for displaying the scene of the AR based environment to a user from the first group of users such that the user can interact with the one or more virtual objects related to the second group of users in real time within the AR based environment;
• and
• sending signals for displaying the scene of the AV based environment to a user from the second group of users such that the user can manipulate a virtual object related to that user to interact with the one or more virtual objects related to the first group of users in real time within the AV based environment.

A system comprising a first user device, a second user device and a server device, wherein:

• the first user device configured to capture data of a first user located at a first physical location;
• the first user device configured to send data of the first user to the server device;
• the second user device configured to capture data of a second user located at a physical location different from the first physical location;
• the second user device configured to send data of the second user to the server device;
• the server device configured to generate, for the first user, a scene of an augmented reality (AR) based environment, the AR based environment including the first user and a virtual object related to the second user that is generated based on the data of the second group of users;
• the server device configured to generate, for the second user, a scene of an augmented virtuality (AV) based environment, the AV based environment including the virtual object related to the second user and a virtual object related to the first user that is generated based on the data of the first user;
• the server device configured to send signals to the first user device for displaying the scene of the AR based environment on the first user device such that the first user can interact with the virtual object related to the second user in real time within the AR based environment;
• and
• the server device configured to send signals to the second user device for displaying the scene of the AV based environment on the second user device such that the second user can interact with the virtual object related to the first user in real time within the AV based environment.