# US 2015/0178044 A1 · System and Method for Controlling Presentations Using a Multimodal Interface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0178044 A1
- Document ID: US20150178044A1
- Status: Published application
- Kind: A1
- Application number: 14/641,762
- Filed: 2015-03-09
- Published: 2015-06-25
- Earliest claimed: 2005-08-19
- Assignees: AT&T Intellectual Property II, L.P.
- Applicants: AT&T Intellectual Property II, L.P.
- Inventors: Patrick EHLEN; David Crawford GIBBON; Mazin GILBERT; Michael JOHNSTON; Zhu LIU; Behzad SHAHRARAY
- Web page: https://patentgrep.com/patent/US20150178044A1

## CPC

- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0483: Interaction with page-structured environments, e.g. book metaphor
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The invention provides for a system, method, and computer readable medium storing instructions related to controlling a presentation in a multimodal system. The method embodiment of the invention is a method for the retrieval of information on the basis of its content for real-time incorporation into an electronic presentation. The method comprises receiving from a presenter a content-based request for at least one segment of a first plurality of segments within a media presentation and while displaying the media presentation to an audience, displaying to the presenter a second plurality of segments in response to the content-based request. The computing device practicing the method receives a selection from the presenter of a segment from the second plurality of segments and displays to the audience the selected segment.

## Claims

### Claim 1

A method comprising:
processing, prior to presentation of a media presentation, the media presentation to enable content searchability as part of a speech dialog;
receiving from a presenter during presentation of the media presentation, at a computing device having a first display screen, a spoken request;
while simultaneously displaying the media presentation to an audience on a second display screen that is physically separate from the first display screen, displaying to the presenter on the first display screen, and not to the audience on the second display screen, graphical views of each of a plurality of segments in response to the spoken request;
receiving a selection of one of the graphical views, to yield a selected segment from the plurality of segments; and
based on the selection, displaying to the audience the selected segment on the second display screen.

### Claim 2 (depends on 1)

The method of claim 1, wherein the second display is not capable of receiving audience interaction input.

### Claim 3 (depends on 1)

The method of claim 1, wherein the processing of the media presentation generates a grammar used to analyze the spoken request prior to the displaying of the graphical views to the presenter.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first plurality of segments and the second plurality of segments are different types of media presentations.

### Claim 5 (depends on 1)

The method of claim 1, wherein the spoken request is coupled with a distinct modality request, wherein the distinct modality request is one of a written request, a gesture, a mouse-click, and a typed command.

### Claim 6 (depends on 5)

The method of claim 5, wherein, the spoken request is made distinguishable from other vocalizations of the presenter by means of a speech control initiation.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first plurality of segments is processed in order to accommodate content-based retrieval of the second plurality of segments using the grammar.

### Claim 8

A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, result in the processor performing operations comprising:
processing, prior to presentation of a media presentation, the media presentation to enable content searchability as part of a speech dialog;
receiving from a presenter during presentation of the media presentation, at a computing device having a first display screen, a spoken request;
while simultaneously displaying the media presentation to an audience on a second display screen that is physically separate from the first display screen, displaying to the presenter on the first display screen, and not to the audience on the second display screen, graphical views of each of a plurality of segments in response to the spoken request;
receiving a selection of one of the graphical views, to yield a selected segment from the plurality of segments; and
based on the selection, displaying to the audience the selected segment on the second display screen.

### Claim 9 (depends on 8)

The system of claim 8, wherein the second display is not capable of receiving audience interaction input.

### Claim 10 (depends on 8)

The system of claim 8, wherein the processing of the media presentation generates a grammar used to analyze the spoken request prior to the displaying of the graphical views to the presenter.

### Claim 11 (depends on 8)

The system of claim 8, wherein the first plurality of segments and the second plurality of segments are different types of media presentations.

### Claim 12 (depends on 8)

The system of claim 8, wherein the spoken request is coupled with a distinct modality request, wherein the distinct modality request is one of a written request, a gesture, a mouse-click, and a typed command.

### Claim 13 (depends on 12)

The system of claim 12, wherein, the spoken request is made distinguishable from other vocalizations of the presenter by means of a speech control initiation.

### Claim 14 (depends on 8)

The system of claim 8, wherein the first plurality of segments is processed in order to accommodate content-based retrieval of the second plurality of segments using the grammar.

### Claim 15

A computer-readable storage device having instructions stored which, when executed by a computing device, result in the computing device performing operations comprising:
processing, prior to presentation of a media presentation, the media presentation to enable content searchability as part of a speech dialog;
receiving from a presenter during presentation of the media presentation, at a computing device having a first display screen, a spoken request;
while simultaneously displaying the media presentation to an audience on a second display screen that is physically separate from the first display screen, displaying to the presenter on the first display screen, and not to the audience on the second display screen, graphical views of each of a plurality of segments in response to the spoken request;
receiving a selection of one of the graphical views, to yield a selected segment from the plurality of segments; and
based on the selection, displaying to the audience the selected segment on the second display screen.

### Claim 16 (depends on 15)

The computer-readable storage device of claim 15, wherein the second display is not capable of receiving audience interaction input.

### Claim 17 (depends on 15)

The computer-readable storage device of claim 15, wherein the processing of the media presentation generates a grammar used to analyze the spoken request prior to the displaying of the graphical views to the presenter.

### Claim 18 (depends on 15)

The computer-readable storage device of claim 15, wherein the first plurality of segments and the second plurality of segments are different types of media presentations.

### Claim 19 (depends on 15)

The computer-readable storage device of claim 15, wherein the spoken request is coupled with a distinct modality request, wherein the distinct modality request is one of a written request, a gesture, a mouse-click, and a typed command.

### Claim 20 (depends on 19)

The computer-readable storage device of claim 19, wherein, the spoken request is made distinguishable from other vocalizations of the presenter by means of a speech control initiation.
