System and Method for Synthesizing and Preserving Consistent Relative Neighborhood Position in Multi-Perspective Multi-Point Tele-Immersive Environments

An e-learning system comprising:

• a classroom having a fixed seating arrangement for a plurality of students and a plurality of first depth-sensing cameras positioned to capture 3D video images of each student from a variety of viewpoints;
• a computerized session controller coupled to the first depth-sensing cameras, and executing software from a non-transitory medium;
• a presenter station providing a position for a presenter, and a plurality of second depth-sensing cameras positioned to capture video images of a presenter at the presenter station from a variety of viewpoints, the plurality of second depth-sensing cameras coupled to the computerized session controller;
• and
• a plurality of augmented-reality headsets, including video display and audio feedback circuitry, one each associated with each of the seats in the fixed-seating arrangement, and one associated with the presenter station;
• wherein the software executing at the computerized session controller operates in one of an instruction mode, wherein a straight-on view and audio of the presenter station is streamed to each of the augmented-reality headsets at each of the seats in the fixed seating arrangement for the plurality of students, an interaction mode, wherein video and audio of interaction between the presenter station and a particular student at one of the seats in the fixed seating arrangement is streamed, such that the particular student views and hears the presenter station head on, and the presenter station views and hears the particular student head-on, and the remainder of the students view and hear the presenter and the student alternately as each participates in an audio exchange, and a discussion mode, wherein video and audio streaming takes place between individual students, the viewpoints and audio feeds to all of the AR headsets controlled such that the students directly interacting see and hear one another directly, and the remainder of the headsets see and here one or the other of the directly-engaged students according to which student is speaking.