# US 2014/0156046 A1 · Soundtrack for electronic text

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0156046 A1
- Document ID: US20140156046A1
- Status: Published application
- Kind: A1
- Application number: 14/234,555
- Filed: 2012-07-26
- Published: 2014-06-05
- Earliest claimed: 2011-07-26
- Assignees: Booktrack Holdings Limited
- Inventors: Paul Charles Cameron; Mark Steven Cameron; Rui Zhang; Russell Davenport; Paul Anthony McGrath
- Web page: https://patentgrep.com/patent/US20140156046A1

## CPC

- G11B 20/10527: {Audio or video recording; Data buffering arrangements (G11B20/12 - G11B20/18 take precedence)}
- G06F 3/0483: Interaction with page-structured environments, e.g. book metaphor
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G09B 5/06: with both visual and audible presentation of the material to be studied
- G09B 5/065: {Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems}
- G10L 21/10: Transforming into visible information
- G11B 27/005: {Reproducing at a different information rate from the information rate of recording (for television signals H04N5/783)}
- G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B 27/11: by using information not detectable on the record carrier
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A soundtrack creation method and user playback system for soundtracks synchronized to electronic text. Synchronization is achieved by maintaining a reading speed variable indicative of the user's reading speed. The system provides for multiple channels of audio to enable concurrent playback of two or more partially or entirely overlapping audio regions so as to create an audio output having, for example, sound effects, ambience, music or other audio features that are triggered to playback at specific portions in the electronic text to enhance the reading experience.

## Claims

### Claim at position 1

1-16. (canceled)

### Claim 17

A method for displaying electronic text and synchronizing the playback of a soundtrack for the electronic text, the soundtrack comprising multiple audio regions configured for playback during corresponding text regions of the electronic text, each audio region being defined by audio data comprises: an audio track for playback in the audio region, a start text position in the electronic text corresponding to where the playback of the audio track is to begin; and a stop text position in the electronic text comprising to where playback of the audio track is to cease, comprising:
displaying at least a portion of the electronic text on an electronic display, the display having an associated user input to enable user interaction with the display;
sensing and tracking movement of the user input relative to the displayed electronic text, the movement representing the user's reading position on the display as they trace their reading of the electronic text via the user input;
generating data indicative of the quantity of text traversed and time taken for the movement;
calculating a measurement of reading speed based on the quantity of text read during the movement and the time taken for the movement;
updating a reading speed variable representing the user's reading speed based on the calculated measurement of reading speed;
maintaining a reading position counter corresponding to the estimated text position in the electronic text that the user is currently reading, the counter incrementing at a rate based on the user reading speed variable; and
controlling playback of the audio regions of the soundtrack over an audio output system based on the audio data and reading position counter to synchronize playback of the audio regions with the user's estimated reading position.

### Claim 18 (depends on 17)

A method according to claim 17 wherein sensing and tracking movement of a user input comprises sensing a start text position in the electronic text at which the user initiates the movement and an end text position in the electronic text at which the user ceases the movement, and generating data indicative of the quantity of the text traversed during the movement comprises determining the number of words or characters between the detected start and end text positions associated with the movement.

### Claim 19 (depends on 18)

A method according to claim 18 wherein generating data indicative of the time taken for the movement comprises initiating a timer upon initiation of the movement at the start text position and halting the timer at the end of the movement at the end text position, the output of the timer being indicative of the time taken for the movement.

### Claim 20 (depends on 18)

A method according to claim 18 wherein the electronic display is a touch screen display having a user input in the form of touch input upon the surface of the display which generates representative touch signals, and wherein the start text position in the electronic text is detected at initiation of the movement upon sensing user contact with the surface of the display and the end text position is sensed at the position in the electronic text where the user disengages their contact from the surface of the display.

### Claim 21 (depends on 18)

A method according to claim 18 wherein sensing and tracking movement of a user input comprises tracking the user input as it moves substantially horizontally relative to the displayed electronic text word-by-word.

### Claim 22 (depends on 18)

A method according to claim 18 wherein sensing and tracking movement of the user input comprises tracking the user input as it moves substantially vertically relative to the displayed electronic text line-by-line.

### Claim 23 (depends on 18)

A method according to claim 18 wherein calculating a measurement of reading speed comprises generating a measurement of reading speed at the end of the movement based on the total quantity of text read and total time taken.

### Claim 24 (depends on 18)

A method according to claim 18 wherein calculating a measurement of reading speed comprises continuously or periodically generating a measurement of reading speed in real-time based on quantity of text read since the initiation of the movement and the time elapsed since the initiation of the movement; and continuously or periodically updating the reading speed variable.

### Claim 25

(canceled)

### Claim 26

A method for displaying electronic text and synchronizing the playback of a soundtrack for the electronic text, the soundtrack comprising multiple audio regions configured for playback during corresponding text regions of the electronic text, each audio region being defined by audio data comprises: an audio track for playback in the audio region, a start text position indicative of the word in the electronic text corresponding to where the playback of the audio track is to begin; and a stop text position indicative of the word in the electronic text comprising to where playback of the audio track is to cease, comprising:
displaying at least a portion of the electronic text on a display;
detecting a first user input indicative of a first reading text position corresponding to any arbitrary word in the electronic text the user is reading at the time of the first input, and generating data indicative of the first reading text position;
detecting a second subsequent user input indicative of a second reading text position corresponding to any arbitrary word in the electronic text the user is reading at the time of the second input, and generating data indicative of the second reading text position;
calculating the quantity of text read between the first and second reading text positions based on the generated data;
calculating an estimate of the user's reading speed based on the calculated quantity of text read and the time period between the first and second user inputs;
updating a reading speed variable representing the user's reading speed based on the calculated estimate of the user's reading speed;
maintaining a reading position counter corresponding to the estimated text position indicative of the word in the electronic text that the user is currently reading, the counter incrementing at a rate based on the user reading speed variable; and
controlling playback of the audio regions of the soundtrack over an audio output system based on the audio data and reading position counter to synchronize playback of the audio regions with the user's estimated reading position.

### Claim 27 (depends on 26)

A method according to claim 26 wherein the electronic text is indexed based on the number of words or characters in the electronic text, and the data indicative of the first reading text position comprises a first index value corresponding to the index value at the first reading text position, and the data indicative of the second reading text position comprises a second index value corresponding to the index value at the second reading text position, and wherein calculating the quantity of text read comprises calculating the absolute difference between the first and second index values.

### Claim 28 (depends on 26)

A method according to claim 26 wherein the method further comprises generating a first time stamp and a second time stamp corresponding to the respective times of the first user input and second user input, and calculating the time period between the first and second user inputs based on absolute difference between the first and second time stamps.

### Claim 29 (depends on 26)

A method according to claim 26 wherein the method further comprises starting a timer on detection of the first user input and stopping the timer on detection of the second user input, and the output of the timer representing the time period between the first and second user inputs.

### Claim 30 (depends on 26)

A method according to claim 26 wherein the display is a touch screen display, and detecting the first and second user inputs comprises sensing user selections of the arbitrary text positions in the electronic text by sensing a user touch input on the surface of the touch screen display at locations corresponding to the arbitrary text positions.

### Claim 31 (depends on 26)

A method according to claim 26 wherein detecting the first and second user inputs comprises detecting user selection of the arbitrary text positions in the electronic text by sensing user activation via a user input device of a pointer on the display situated at the location corresponding to the arbitrary text positions.

### Claim 32 (depends on 26)

A method according to claim 26 wherein the method further comprises calculating a new estimate of the user's reading speed for each new subsequent user input indicative of the user's arbitrary reading text position in the electronic text, each new estimate of the user's reading speed being calculated based on the quantity of text and time period between the user input indicative of the newest arbitrary reading text position and the previous indicated arbitrary reading text position.

### Claim 33 (depends on 32)

A method according to claim 32 wherein the method comprises updating the user reading speed variable by applying an update algorithm for each new calculated estimate of the user's reading speed, the update algorithm comprising:
calculating a new moving average of the reading speed variable based on a predetermined number of previous reading speed variables;
calculating a difference value indicative of the percentage change between the new calculated estimate of the user's reading speed and the moving average;
determining a probability weighting for the new calculated estimate of the user's reading speed based on the difference value and a probability density function; and
calculating a new reading speed variable based on the new calculated estimate of the user's reading speed, the moving average, and the probability weighting.

### Claim 34

(canceled)

### Claim 35

A method for displaying electronic text and synchronizing the playback of a soundtrack for the electronic text, the soundtrack comprising multiple audio regions configured for playback during corresponding text regions of the electronic text, each audio region being defined by audio data comprises: an audio track for playback in the audio region, a start text position in the electronic text corresponding to where the playback of the audio track is to begin; and a stop text position in the electronic text comprising to where playback of the audio track is to cease, comprising:
displaying at least a portion of the electronic text on a display;
displaying a moving reading position indicator on the display which identifies the current text position that is estimated as being read based on an initial reading speed variable representing the user's estimated reading speed, the reading position indicator moving relative to the text at a rate based on a reading speed variable representing the user's reading speed;
providing a user adjustment interface that is operable to increase or decrease the reading speed variable; and
updating the reading speed variable according to any user adjustment to thereby modify the rate of movement of the reading position indicator to match the updated reading speed variable,
wherein the user adjustment interface is operable simultaneously as the electronic text and moving reading position indicator are displayed such that the user can view in real-time the effect of any adjustment of the reading speed variable;
maintaining a reading position counter corresponding to the estimated text position in the electronic text that the user is currently reading, the counter incrementing at a rate based on the user reading speed variable; and
controlling playback of the audio regions of the soundtrack over an audio output system based on the audio data and reading position counter to synchronize playback of the audio regions with the user's estimated reading position.

### Claim 36 (depends on 35)

A method according to claim 35 wherein the user adjustment interface is in the form of activatable increase and decrease inputs that are configured to respectively increase or decrease the reading speed variable by a predetermined value upon activation by a user.

### Claim 37 (depends on 35)

A method according to claim 35 wherein the user adjustment interface is in the form of an operable dial or slider that is moveable to increase or decrease the reading speed variable to a user selected value within a scale of reading speed values defined between upper and lower limits.

### Claim 38 (depends on 35)

A method according to claim 35 wherein the reading position indicator is an inline indicator that is configured to traverse the lines of text word-by-word or character-by-character at a rate that is dependent on the reading speed variable.

### Claim 39 (depends on 35)

A method according to claim 35 wherein the reading position indicator is a margin indicator that is configured to move down the margin of the display text line-by-line at a rate that is dependent on the reading speed variable.

### Claim 40

(canceled)