Back to results
US 12,591,306 B2Granted
Nonverbal multi-input and feedback devices for user intended computer control and communication of text, graphics and audio
AssigneeCognixion Corporation
InventorAndreas Forsland
Life of this patent
Estimating the term…
Abstract
There is disclosed devices, systems and methods for nonverbal multi-input and feedback devices for user intended computer control and communication of text, graphics and audio. The system comprises sensory devices comprising sensors to detect a user inputting gestures on sensor interfaces, a cloud system comprising a processor, for retrieving the inputted gestures detected by the sensor on the sensory device, comparing the inputted gestures to gestures stored in databases on the cloud system, identifying at least a text, graphics and/or speech command comprising a word that corresponds to the inputted gesture; showing the command to the user; and transmitting the command to another device.
Claims19
A nonverbal multi-input and feedback system, comprising:
- a. a processing portion configured to receive user inputs from different ones of a first plurality of sensors and process said inputs to context enhanced outputs and actions encoded in at least one of text, graphics, emoji or audio;
- wherein the first plurality of sensors include at least two of a touch sensor, a biosensor, a keyboard, a facial tracking sensor, or an eye and pupil tracking sensor;
- b. a multimodal input and intent detection block configured to receive and process the user inputs to determine user intents, and to feed the user intents to said processing portion;
- c. a context and awareness block configured to:
- i. receive and process metadata inputs from different ones of a second plurality of sensors to provide logic of output and action, the second plurality of sensors including at least two of biometric sensors, environmental sensors, object recognition sensors, facial recognition sensors, voice recognition sensors, date and time sensors, history sensors, location sensors, or proximity sensors, and
- ii. feed the logic of output and action to said processing portion;
- d. an output and action block configured to encode outputs and actions using the output from the processing portion and to send the outputs to at least one display, computing device, controller, speaker or artificial intelligence (AI) assistant, wherein said outputs include non-virtual control commands and communications sent to other computing devices, and wherein said outputs comprise at least one of text, graphics, emoji, or audio;
- and
- e. a vocabulary block configured to provide a lexicon or vocabulary in a selected language to said processing portion to encode user outputs in natural language, wherein said vocabulary block comprises a machine learning component that generates machine language commands.
6 more elements
A method for processing user inputs in a nonverbal multi-input and feedback device system to send control commands and communications to other computing devices, the method comprising:
- a. receiving user inputs from a plurality of sensors which include least two of a touch sensor, a biosensor, a keyboard, a facial tracking sensor, or an eye and pupil tracking sensor;
- b. processing said user inputs to determine a user intent;
- c. enriching the processed user inputs with metadata to determine logic of output and action;
- d. generating outputs based on the logic of output and action, wherein said outputs include non-virtual control commands and communications to be sent to other computing devices, and wherein said outputs include at least two of text, graphics, emoji, or audio;
- and
- e. sending said outputs to at least two other computing devices, each computing device having at least one of a display, a controller, a speaker, or a network communication device, wherein said sending said outputs includes machine learning that generates machine language commands.
3 more elements
A nonverbal multi-input and feedback system, comprising:
- a. a processing portion configured to receive user inputs from different ones of a first plurality of sensors and process said inputs to context enhanced outputs and actions encoded in at least one of text, graphics, emoji or audio;
- wherein the first plurality of sensors include at least two of a touch sensor, a biosensor, a keyboard, a facial tracking sensor, or an eye and pupil tracking sensor;
- and wherein the processing portion includes an augmented intelligence engine that creates the context enhanced outputs that include enhancement of user inputs based on context from logic of output and action provided from a second plurality of sensors;
- b. a multimodal input and intent detection block configured to receive and process the user inputs to determine user intents, and to feed the user intents to said processing portion;
- c. a context and awareness block configured to:
- i. receive and process metadata inputs from different ones of the second plurality of sensors to provide logic of output and action, the second plurality of sensors including at least two of biometric sensors, environmental sensors, object recognition sensors, facial recognition sensors, voice recognition sensors, date and time sensors, history sensors, location sensors, or proximity sensors, and
- ii. feed the logic of output and action to said processing portion;
- d. an output and action block configured to encode outputs and actions using the output from the processing portion and to send the outputs to at least one display, computing device, controller, speaker or artificial intelligence (AI) assistant, wherein said outputs include non-virtual control commands and communications sent to other computing devices, and wherein said outputs comprise at least one of machine language commands, text, graphics, emoji, or audio, wherein said sending the outputs to at least one display includes sending outputs including the machine language commands;
- and
- e. a vocabulary block configured to provide a lexicon or vocabulary in a selected language to said processing portion to encode user outputs in natural language.
7 more elements
Dependencies are inferred from the claim text.