# US 9,437,206 B2 · Voice control of applications by associating user input with action-context identifier pairs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,437,206 B2
- Document ID: US9437206B2
- Status: Granted
- Kind: B2
- Application number: 13/835,335
- Filed: 2013-03-15
- Published: 2013-09-19
- Granted: 2016-09-06
- Earliest claimed: 2012-03-16
- Assignees: France Telecom
- Applicants: France Telecom
- Inventors: Dachuan Yu; John Benko; Satya Mallya
- Web page: https://patentgrep.com/patent/US9437206B2

## CPC

- G10L 21/00: Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility (G10L19/00 takes precedence)
- G10L 15/1822: {Parsing for meaning understanding}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method is provided for enabling or enhancing a use of voice control in a voice controlled application (VCA) via a development framework. The method includes providing in the framework a plurality of action-context pairs usable in a memory of an application development device, which includes a processor. The action-context pairs serve to direct execution of the VCA, wherein the framework context defines a list of parameters related to the action and their respective value types. At least one of a voice recognition engine (VRE) and a natural language library is provided to match each action-context pair with a semantically related vocabulary. A registration mechanism is provided in the framework, which permits an association to be formed between an action-context pair and a handler in the voice controlled application. An associated development system for developing the VCA and user equipment that executes the VCA are provided as well.

## Claims

### Claim 1

A method for enabling or enhancing a use of voice control in a voice controlled application (VCA) via a development framework, the method comprising:
providing in the development framework a plurality of framework action-context pairs wherein the development framework is stored in a memory of an application development device comprising a processor, the plurality of framework action-context pairs being configured to direct execution of the VCA, wherein each framework action-context pair has an action and a context that defines a list of parameters related to the action and respective value types of the parameters;
providing at least one of a voice recognition engine (VRE) or a natural language library to match each action-context pair with semantically related vocabulary;
generating automatically a grammar to capture knowledge about natural language understanding in the context of said VCA;
providing in the development framework a registration mechanism that forms an association between an action-context pair and a handler in the VCA, the registration mechanism such that when a voice input is received, the voice input is sent to a speech recognition engine based on said grammar to obtain an action-context pair from the plurality of framework action-context pairs and the action-context pair is used to select and trigger the handler in the VCA to execute the action intended by the voice input.

### Claim 2 (depends on 1)

The method of claim 1, further comprising providing in the development framework:
a handler definition mechanism comprising types, classes, and interfaces or utilities that are usable to define the actions and the contexts of the action-context pairs, and associated VCA handlers; and
a dispatching mechanism comprising interfaces or utilities that are usable to look up a registration at runtime;
wherein the registration mechanism comprises interfaces or utilities that are usable to define a registration element to hold registrations at runtime.

### Claim 3 (depends on 2)

The method of claim 2, wherein the registration element comprises a table, a hash, or registration mapping.

### Claim 4 (depends on 2)

The method of claim 2, wherein the development framework is additionally provided with further utility mechanisms selected from the group consisting of:
setup and configuration mechanisms;
start and stop listening to voice command mechanisms used to customize a triggering of voice control;
custom semantic mappings;
special handlers, in the action handler registration, for error and exception handling;
lightweight voice control;
stage setup and transitioning;
a text-to-speech API for voice interaction; and
a mechanism for interrupting a current task with specific voice commands.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
selecting, by an application developer, for the VCA, a plurality of application action-context pairs from the framework action-context pairs, that define voice control of the VCA, wherein each application action-context pair defines an action and a list of parameters related to the action and respective value types of the parameters;
associating an application handler of the VCA with each of the application action-context pairs; and
producing the VCA that uses the action-context pairs and associated application handler.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
defining, by an application developer, a new application action-context pair not present in the framework action-context pairs;
defining an application grammar for the new application action-context pair using the natural language library to match the new application action-context pair with new semantically related vocabulary;
registering the new application action-context pair and associated grammar with the application action-context pairs; and
associating a new application handler with the new application action-context pair.

### Claim 7 (depends on 5)

The method of claim 5, further comprising:
defining a new application grammar for a specific application action-context pair using the natural language library to match the specific application action-context pair with new semantically related vocabulary; and
registering the new application grammar with the specific application action-context pair.

### Claim 8 (depends on 5)

The method of claim 5, further comprising:
executing the VCA on a user equipment (UE);
assigning VCA handler execution addresses to each of the application action-context pairs at run-time;
providing a voice input to the UE comprising verbal commands;
performing voice recognition (VR) and language parsing on the voice input by the device or on a networked cloud;
translating the voice input into a run-time action-context pair, wherein the run-time context contains a list of parameter values related to the action; and
executing the VCA handler at the VCA handler execution address associated with the run-time action-context pair from among the assigned VCA handler addresses.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
dynamically assigning a new VCA handler execution address with a previously assigned application action-context pair.

### Claim 10

A non-transitory computer program product, comprising a computer usable medium having a computer readable program code embodied therein, said computer readable program code being executable by a processor to implement a method for enabling or enhancing a use of voice control in a voice controlled application (VCA) via a development framework, the method comprising:
providing in the development framework a plurality of framework action-context pairs usable in a memory of an application development device comprising a processor, that serve to direct execution of the VCA, wherein each framework action-context pair defines an action and a list of parameters related to the action and respective value types of the parameters;
providing at least one of a voice recognition engine (VRE) or a natural language library to match each action-context pair with semantically related vocabulary; and
generating automatically a grammar to capture knowledge about natural language understanding in the context of said VCA;
providing in the development framework a registration mechanism that forms an association between an action-context pair and a handler in the VCA, the registration mechanism such that when a voice input is received, the voice input is sent to a speech recognition engine based on said grammar to obtain an action-context pair from the plurality of framework action-context pairs and the action-context pair is used to select and trigger the handler in the VCA to execute the action intended by the voice input.

### Claim 11

A voice controlled application (VCA) development system, comprising:
a processor;
a storage device accessible by the processor;
a voice controlled application development framework comprising:
a plurality of framework action-context pairs that serve to direct execution of the VCA, wherein each framework action-context pair defines an action and a list of parameters related to the action and the respective value types of the parameters;
at least one of a voice recognition engine (VRE) and a natural language library to match each framework action-context pair with semantically related vocabulary;
a grammar automatically generated to capture knowledge about natural language understanding in the context of said VCA;
a registration mechanism that permits an association to be formed between an action-context pair and a handler in the voice controlled application;
wherein the processor:
receives a voice input,
applies the voice input to the VRE to transform the voice input into natural language text,
applies the natural language text to the automatically generated grammar to identify at least one framework action-context pair, and
uses the action-context pair to select and trigger the handler in the VCA to execute the action intended by the voice input.

### Claim 12 (depends on 11)

The development system of claim 11, wherein the framework further comprises:
a handler definition mechanism comprising types, classes, and interfaces or utilities that are usable to define actions, contexts, and handlers; and
a dispatching mechanism comprising interfaces or utilities that are usable to look up a registration at runtime;
wherein the registration mechanism comprises interfaces or utilities that are usable to define a registration element to hold registrations at runtime.

### Claim 13 (depends on 12)

The development system of claim 12, wherein the registration element comprises a table, a hash, or registration mapping.

### Claim 14 (depends on 12)

The development system of claim 12, wherein the framework is additionally provided with further utility mechanisms selected from the group consisting of:
setup and configuration mechanisms;
start and stop listening to voice command mechanisms used to customize a triggering of voice control;
custom semantic mappings;
special handlers, in the action handler registration, for error and exception handling;
lightweight voice control;
stage setup and transitioning;
a text-to-speech API for voice interaction; and
a mechanism for interrupting a current task with specific voice commands.

### Claim 15 (depends on 11)

The development system of claim 11, further comprising a data store comprising:
an application action-context pair that holds a plurality of application action-context pairs selected by an application developer for a VCA from the framework action-context pairs, that define voice control of the VCA, wherein each application action-context pair defines an action and a list of parameters related to the action and respective value types of the parameters;
an application handler of the VCA associated with each of the application action-context pairs; and
a VCA executable file that uses the action-context pairs and associated application handler.

### Claim 16 (depends on 15)

The development system of claim 15, wherein the data store further comprises:
a new application action-context pair defined by an application developer that is not present in the framework action-context pairs;
an application grammar for the new application action-context pair that uses the natural language library to match the new application action-context pair with new semantically related vocabulary;
a registration mechanism that registers the new application action-context pair and associated grammar with the application action-context pairs; and
a new application handler association link that links the new application action-context pair.

### Claim 17 (depends on 15)

The development system of claim 15, wherein the data store further comprises:
a new application grammar defined for a specific application action-context pair using the natural language library to match the specific application action-context pair with new semantically related vocabulary, wherein the new application grammar is registered with the specific application action-context pair.

### Claim 18

User equipment (UE), comprising:
a microphone;
a storage area;
a VCA executable file that uses action-context pairs and associated VCA application handlers associated with each action-context pair;
an assignment element that assigns VCA handler execution addresses to each of the action-context pairs at run-time;
an access element for accessing at least one of a voice recognition engine (VRE) and a natural language library on the UE or on a networked cloud and translating voice input into a run-time action-context pair, wherein the run-time context contains a list of parameter values related to the action;
a speech recognition engine for obtaining a run-time action-context pair from a voice input, said speech recognition engine being based on a grammar automatically generated to capture knowledge about natural language understanding in the context of said VCA;
an execution element that executes a specific VCA handler at the VCA handler execution address associated with the run-time action-context pair.

### Claim 19 (depends on 18)

The UE of claim 18, further comprising:
a utility that dynamically assigns a new VCA handler execution address with a previously assigned application action-context pair.