System for recognizing user hand gesture and providing virtual reality content based on deep learning using transfer learning
AssigneeElectronics and Telecommunications Research Institute
InventorsYeong Jae CHOI, Seung Joon KWON, Sang Woo SEO
Life of this patent
Estimating the term…
Abstract
Provided is a system for recognizing a user hand gesture and providing virtual reality content based on deep learning using transfer learning. The system includes a head mounted display (HMD) configured to display, to a user, virtual reality content based on a hand gesture, a hand gesture sensor configured to acquire hand data of the user, and a controller device configured to receive the hand data and convert the received hand data into hand gesture data, manage a hand gesture recognizer on the basis of deep learning, and perform content execution management.
Claims17
A system of recognizing a user hand gesture and providing virtual reality content based on deep learning using transfer learning, the system comprising:
- a head mounted display (HMD) configured to display, to a user, virtual reality content based on a hand gesture;
- a hand gesture sensor configured to acquire hand data of the user;
- and
- a controller device configured to receive the hand data and convert the received hand data into hand gesture data, manage a hand gesture recognizer on the basis of deep learning, and perform content execution management.
A method of recognizing a user hand gesture and providing virtual reality content based on deep learning using transfer learning, the method comprising the steps of:
- (a) receiving hand data and converting the received hand data into hand gesture data;
- (b) identifying whether a suitable hand gesture classifier exists using the hand gesture data;
- and
- (c) when it is identified in the step (b) that the suitable hand gesture classifier does not exist, executing content for generating deep learning-based hand gesture recognizer using transfer learning, and when it is identified in the step (b) that the suitable hand gesture classifier exists, executing hand gesture recognition-based user content.
An apparatus for recognizing a user hand gesture and providing virtual reality content based on deep learning using transfer learning, the apparatus comprising:
- an inputter configured to receive hand data;
- a memory in which a program for providing virtual reality content using the hand data is stored;
- and
- a processor configured to execute the program,
- wherein the processor receives the hand data and converts the received hand data into hand gesture data, manages a hand gesture recognizer on the basis of deep learning, and performs content execution management.
Dependencies are inferred from the claim text.