US 2017/0255580 A1Published application

Multi-modal input system for a computer system

AssigneeNorthrop Grumman Systems Corporation

InventorsDEVANG R. PAREKH, HENRY H. FUNG, WAYNE KIM, LOUIS ODDO, FENG CAO, PAUL DOMINGUEZ, BENJAMIN MONTGOMERY, PETER FONG

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

One example includes a computer system. Ports each receive signals corresponding to an interface input associated with user physical interaction provided via an interface device in one of disparate input modes. A multi-modal input system maps an interface input associated with one of the ports provided in a given one of the disparate input modes into a computer input command, maps an interface input associated with another of the ports provided in another one of the disparate input modes into another computer input command, and aggregates the computer input commands into a multi-modal event command. A processor executes a single predetermined function associated with the computer system in response to the multi-modal event command. Thus, the processor is configured to execute the single predetermined function associated with the computer system in response to user physical interaction provided in at least two of the plurality of disparate input modes.

Claims20

Dependent claims

A computer system comprising:

a processor;
a plurality of ports that are each configured to receive signals corresponding to an interface input associated with user physical interaction provided via one of a respective plurality of interface devices in one of a respective plurality of disparate input modes;
and
a multi-modal input system configured to map an interface input associated with one of the plurality of ports that is provided in a given one of the plurality of disparate input modes into a given one of a plurality of computer input commands, to map an interface input associated with another one of the plurality of ports that is provided in another one of the plurality of disparate input modes into another one of the plurality of computer input commands, and to aggregate the given one and the other one of the plurality of computer input commands into a multi-modal event command, the processor being configured to execute a single predetermined function associated with the computer system in response to the multi-modal event command, such that the processor is configured to execute the single predetermined function associated with the computer system in response to user physical interaction provided in at least two of the plurality of disparate input modes.

1 more element

A method for providing input to a computer system, the method comprising:

converting a first physical input action provided by a user in a first input mode via a first interface device into a first interface input based on a first application programming interface (API) associated with the first interface device;
mapping the first interface input to a first computer input command associated with a native schema of the computer system;
converting a second physical input action provided by the user in a second input mode via a second interface device into a second interface input based on a second API associated with the second interface device;
mapping the second interface input to a second computer input command associated with the native schema of the computer system;
aggregating the first computer input command and the second computer input command into a multi-modal event command;
and
executing a single predetermined function associated with the respective computer system in response to the multi-modal event command.

4 more elements

A computer system comprising:

a processor;
a plurality of ports that are each coupled to one of a respective plurality of interface devices configured to receive user physical interaction provided in one of a respective plurality of disparate input modes;
a multi-modal input system comprising:
a deconfliction engine configured to convert the user physical interaction associated with each of the plurality interface devices into a respective plurality of interface inputs via a plurality of application programming interfaces (APIs) associated with the respective plurality of interface devices, and to map the plurality of interface inputs into a respective plurality of computer input commands associated with a native schema of the computer system;
a multi-modal command aggregation controller configured to aggregate the plurality of computer input commands into a multi-modal event command based on comparing a timer value associated with a modality timer with a predetermined threshold timer value;
and
a processor configured to execute a single predetermined function associated with the computer system in response to the multi-modal event command, such that the processor is configured to execute the single predetermined function associated with the computer system in response to user physical interaction provided in at least two of the plurality of disparate input modes.

4 more elements

Dependencies are inferred from the claim text.