Interactive System Using Speech Recognition and Digital Media
AssigneeCarnegie Mellon University
InventorsAnthony Tomasic, Erik Thiessen, Cassondra Eng, Amy Ogan
Life of this patent
Estimating the term…
Abstract
A system for interactive system using speech recognition and digital media is described. The system uses automated speech recognition and recognizes interactions from users to execute digital media items. The interactions are based on behavior of the user. The user is given a prompt. If the student responds to a prompt correctly, the student is rewarded with an animation. Otherwise the user experience continues without a reward. The system recognizes natural language responses for interactions of the user. The media item is dynamically generated as the user interacts with the digital media item.
Claims21
A server system, comprising:
- a data store configured to store one or more data items each representing at least a portion of a given media item;
- an interface in communication with the data store, the interface configured to enable a user to input one or more instructions specifying a playback configuration for the given media item;
- a controller in communication with the interface and the data store, the controller configured to perform operations comprising:
- receiving at least one instruction specifying a portion of a media item and at least one event associated with the portion of the media item, the at least one event representing a solicitation of a user input to a playback device during playback of the portion of the media item by the playback device, the user input specifying one or more objects, concepts, or both objects and concepts related to the portion of the media item;
- generating, based on the at least one instruction, executable code representing a playback configuration of the portion of the media item, the playback configuration comprising:
- generation of additional media based on the user input, the additional media representing at least one of the objects, concepts, or both objects and concepts;
- incorporation of the additional media into the portion of the media item;
- and
- playback of the portion of the media item of the additional media incorporated into the portion of the media item.
A device configured for presenting an interactive media item, the device comprising:
- a user interface configured to present portions of an interactive media item to a user;
- at least one sensor configured to measure an interaction of the user with the user interface and generate sensor data representing the interaction;
- and
- a controller configured to receive the sensor data from the at least one sensor, the controller configured to perform operations comprising:
- causing, by the user interface, a presentation of a portion of the interactive media item;
- obtaining sensor data indicative of a particular interaction with the portion of the interactive media item;
- determining whether the particular interaction with the portion of the interactive media item satisfies one or more criteria;
- in response to determining that the one or more criteria are satisfied:
- generating a reward media item associated with the portion of the interactive media item, the reward media item related to one or more objects or concepts represented by the particular interaction;
- incorporating the reward media item into the portion of the interactive media item;
- and
- causing presentation of the interactive media item incorporating the reward media item;
- in response to determining that the one or more criteria are not satisfied, causing the presentation of the portion of the interactive media item to repeat without presentation of the reward media item.
Dependencies are inferred from the claim text.