System and method for the creation and playback of soundtrack-enhanced audiobooks

A system for synchronized playback of narration audio with displayed electronic text, the system comprising:

• one or more processors;
• and memory storing instructions that, when executed by the one or more processors, cause the system to:
• display electronic text corresponding to or indicative of narration in narration audio;
• generate or maintain mapping data associating a plurality of text positions in the displayed electronic text with corresponding narration positions in the narration audio;
• receive a narration speed setting;
• output audio content comprising the narration audio corresponding to a text source at a narration speed corresponding to the narration speed setting;
• and
• during output of the audio content, provide a visual indicator in or along the displayed electronic text indicating a current text position corresponding to a current narration position in the narration audio and update the visual indicator based on the mapping data and the narration speed setting such that the visual indicator maintains correspondence with the current narration position when the narration speed setting changes.

A system for generating mapping data for a soundtrack-enhanced audiobook, the system comprising:

• one or more processors;
• and memory storing instructions that, when executed by the one or more processors, cause the system to:
• obtain narration audio of a digital audiobook corresponding to electronic text;
• and
• generate mapping data that maps text positions in the electronic text to corresponding narration positions in an audiobook playback timeline of the narration audio.

A system for generating soundtrack configuration data for a digital audiobook, the system comprising:

• one or more processors;
• and memory storing instructions that, when executed by the one or more processors, cause the system to:
• receive or retrieve electronic text corresponding to narration of the digital audiobook;
• and
• generate soundtrack configuration data defining one or more soundtrack audio regions for synchronized playback with corresponding narration regions of the digital audiobook.

A system for synchronizing playback of a digital soundtrack configured relative to electronic text with playback of a digital audiobook associated with the electronic text, the system comprising:

• one or more processors;
• and memory storing instructions that, when executed by the one or more processors, cause the system to:
• receive a text-position based soundtrack comprising multiple audio regions, each audio region being associated with at least a start text position in the electronic text;
• receive mapping data mapping text positions in the electronic text to narration positions in an audiobook playback timeline;
• and
• convert the text-position based soundtrack into a time-based soundtrack by converting the start text positions of the audio regions into corresponding start times in the audiobook playback timeline for synchronized playback with the digital audiobook.

A content playback system comprising:

• one or more processors;
• and
• memory storing instructions that, when executed by the one or more processors, cause the content playback system to:
• generate or maintain mapping data associating text positions in displayed electronic text with narration positions in an audiobook playback timeline of narration audio corresponding to the displayed electronic text;
• operate in a reading mode in which one or more soundtrack audio regions are played back synchronized to a user reading position in the displayed electronic text, the user reading position being tracked by a reading position variable updated based at least in part on user input associated with the displayed electronic text;
• operate in a listening mode in which the narration audio is played back and the one or more soundtrack audio regions are played back synchronized to a current narration position in the audiobook playback timeline, the current narration position being tracked by a narration position variable updated based at least in part on audiobook playback data;
• receive a transition input initiating a transition from the reading mode to the listening mode;
• store bookmark reference data indicative of the user reading position when the transition input is received;
• determine, based at least in part on the mapping data, a bookmark narration position in the audiobook playback timeline corresponding to the user reading position indicated by the bookmark reference data;
• and
• upon the transition to the listening mode, commence playback of the narration audio from the bookmark narration position and synchronize playback of the one or more soundtrack audio regions based at least in part on the audiobook playback data.

A system for generating soundtrack configuration data, the system comprising:

• one or more processors;
• and
• memory storing instructions that, when executed by the one or more processors, cause the system to:
• receive or retrieve electronic text corresponding to narration audio of a text source;
• perform semantic analysis of the electronic text to identify mood or semantic characteristics of one or more text portions of the electronic text;
• automatically configure one or more soundtrack audio regions for synchronized playback with corresponding narration regions of the narration audio based at least in part on the identified mood or semantic characteristics;
• select one or more audio tracks for the one or more soundtrack audio regions based at least in part on the identified mood or semantic characteristics;
• and
• generate soundtrack configuration data defining the one or more soundtrack audio regions and the selected one or more audio tracks for synchronized playback with the narration audio.

A method for synchronized playback of audio content with displayed electronic text, the method comprising:

• outputting narration audio corresponding to a text source at a narration speed corresponding to a narration speed setting;
• displaying electronic text corresponding to or indicative of narration in the narration audio;
• generating or maintaining mapping data associating a plurality of text positions in the displayed electronic text with corresponding narration positions in the narration audio;
• determining a current text position in the displayed electronic text corresponding to a current narration position in the narration audio based at least in part on the mapping data;
• and
• during output of the narration audio, providing a visual indicator in or along the displayed electronic text indicating the current text position and updating the visual indicator based at least in part on the mapping data and the narration speed setting such that the visual indicator maintains correspondence with the current narration position when the narration speed setting changes.