# US 2010/0169246 A1 · Multimodal system and input process method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0169246 A1
- Document ID: US20100169246A1
- Status: Published application
- Kind: A1
- Application number: 12/591,832
- Filed: 2009-12-02
- Published: 2010-07-01
- Earliest claimed: 2008-12-30
- Assignees: Samsung Electronics Co., Ltd.
- Inventors: Jun Won Jang; Tae Sin Ha
- Web page: https://patentgrep.com/patent/US20100169246A1

## CPC

- B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G05B 2219/39441: Voice command, camera detects object, grasp, move
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A multimodal system and an input processing method thereof are disclosed. The multimodal system includes a pre-constructed input combination constructing unit and an input combination selection unit for selecting an input combination corresponding to an input signal from a user or a sensor. The system performs learning for selecting an input combination from the pre-constructed input combinations. The system provides available input combinations due to this learning, resulting in high satisfaction with the processed result.

## Claims

### Claim 1

A multimodal system comprising:
a multimodal input unit providing at least one input signal;
an input combination constructing unit in which at least one input combination for executing a certain action corresponding to the at least one input signal is pre-constructed; and
an input combination selection unit for selecting a final input combination according to the at least one input signal.

### Claim 2 (depends on 1)

The multimodal system according to claim 1, wherein the multimodal input unit provides a voice signal or a sensor's input signal.

### Claim 3 (depends on 1)

The multimodal system according to claim 1, wherein the input combination selection unit selects one of the at least one input combination from the input combination constructing unit.

### Claim 4 (depends on 3)

The multimodal system according to claim 3, wherein the input combination selection unit selects the final input combination using only the at least one input signal provided from the multimodal input unit.

### Claim 5 (depends on 1)

The multimodal system according to claim 1, wherein the input combination constructing unit includes an input combination selected by a user.

### Claim 6 (depends on 1)

The multimodal system according to claim 1, wherein the input combination constructing unit performs learning from a first input combination which has been initially selected at random, such that input combinations of the at least one input combination are selected according to the learned result.

### Claim 7 (depends on 1)

The multimodal system according to claim 1, wherein the input combination constructing unit performs learning from a first input combination which has been initially selected by a user, such that input combinations of the at least one input combination are selected according to the learned result.

### Claim 8 (depends on 6)

The multimodal system according to claim 6, wherein the learning is used to select the input combinations of the at least one input combination such that user satisfaction of a result processed by a previous input combination becomes higher.

### Claim 9 (depends on 7)

The multimodal system according to claim 7, wherein the learning is used to select the input combinations of the at least one input combination such that user satisfaction of a result processed by a previous input combination becomes higher.

### Claim 10 (depends on 6)

The multimodal system according to claim 6, wherein the learning corresponds to any one of a process for removing an input combination, a process for generating an input combination, a process for dividing an input combination, a process for merging an input combination, and a process for improving an input combination.

### Claim 11 (depends on 7)

The multimodal system according to claim 7, wherein the learning corresponds to any one of a process for removing an input combination, a process for generating an input combination, a process for dividing an input combination, a process for merging an input combination, and a process for improving an input combination.

### Claim 12

An input processing method for a multimodal system comprising:
pre-constructing at least one input combination for executing a certain action;
selecting an input combination according to an input signal provided from a user or a sensor; and
executing an action corresponding to the selected input combination.

### Claim 13 (depends on 12)

The method according to claim 12, where the selecting selects any one of the at least one input combination of the pre-constructed input combinations.

### Claim 14 (depends on 12)

The method according to claim 12, wherein the selecting selects any one of the at least one input combination using only the input signal provided from the user or the sensor.

### Claim 15 (depends on 12)

The method according to claim 12, wherein the pre-constructing of the at least one input combination includes an input combination selected by the user.

### Claim 16 (depends on 12)

The method according to claim 12, wherein the pre-constructing of the at least one input combination includes an input combination selected by performing learning from a first input combination which has been initially selected at random.

### Claim 17 (depends on 12)

The method according to claim 12, wherein the pre-constructing of the at least one input combination includes an input combination selected by performing learning from a first input combination which has been initially selected by the user.

### Claim 18 (depends on 16)

The method according to claim 16, wherein the learning corresponds to any one of a process for removing an input combination, a process for generating an input combination, a process for dividing an input combination, a process for merging an input combination, and a process for improving an input combination.

### Claim 19 (depends on 17)

The method according to claim 17, wherein the learning corresponds to any one of a process for removing an input combination, a process for generating an input combination, a process for dividing an input combination, a process for merging an input combination, and a process for improving an input combination.
