US 11,694,565 B2 · Virtual and augmented reality instruction system

Abstract

A virtual and augmented reality instruction system may include a complete format and a portable format. The complete format may include a board system to capture all movement (including writing and erasing) on the board's surface, and a tracking system to capture all physical movements. The portable format may include a touch-enabled device or digital pen and a microphone, and is designed to capture a subset of the data captured by the complete format. In one embodiment of the complete format, the board system and the tracking system can communicate with each other through a network, and control devices (such as a laptop, desktop, mobile phone and tablet) can be used to control the board system and tracking system through the network. In further embodiments of the complete format, augmented reality can be achieved within the tracking system through the combination of 3D sensors and see through augmented reality glasses.

Claims12

Dependent claims

A virtual and augmented reality instruction system for virtually recreating an environment from captured movement, depth, audio and video, the virtual and augmented reality instruction system in a complete format and comprising:

a board system, comprising of one or more pen-enabled boards, to capture all movements on a surface of each board in the board system;
a tracking system, comprising of one or more tracking units equipped with a microphone and three-dimensional (RGBD) sensor, from which a tracking region is established up to a predetermined distance from the board system, to capture all movement of objects and individuals and individuals' speech;
and,
a control device, the board system and tracking system being configured to communicate with each other over a network and being controlled by the control device through the network, the virtual and augmented reality instruction system being configured to process and sync through various engines sensor output to generate processed data representing writing, erasing, position, orientation, skeleton, speech, background noise, and geometry, allowing to virtually recreate the environment comprised of the boards and the individuals and objects in the tracking region.

1 more element

A virtual and augmented reality instruction system for virtually recreating an environment from captured movement and audio, the virtual and augmented reality instruction system in a portable format and comprising:

a touch-enabled display or digital pen to capture all movements on a surface of the display or of a medium specific to the digital pen;
a microphone to capture audio;
and,
a computing device, the virtual and augmented reality instruction system being configured to process and sync through various engines sensor output to generate processed data representing writing, erasing, speech, and avatar movement, allowing to virtually recreate an environment comprised of the surface and computer generated geometry, avatars, and objects.

1 more element

Dependencies are inferred from the claim text.

Virtual and augmented reality instruction system

Life of this patent

Virtual and augmented reality instruction system

Life of this patent

Abstract

Claims12