# US 2010/0169776 A1 · Generalized interactive narratives > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2010/0169776 A1 - Document ID: US20100169776A1 - Status: Published application - Kind: A1 - Application number: 12/347,868 - Filed: 2008-12-31 - Published: 2010-07-01 - Earliest claimed: 2008-12-31 - Assignees: Microsoft Corporation - Inventors: Aditya Sankar; Archana Prasad; Narendranath D. Govindachetty; Joseph Joy - Web page: https://patentgrep.com/patent/US20100169776A1 ## CPC - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - H04N 21/2387: Stream processing in response to a playback request from an end-user, e.g. for trick-play - H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection - H04N 21/8541: involving branching, e.g. to different story endings - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A multimedia system specifies a multimedia experience stream by a narrative definition that defines a narrative presentation having sub-narratives. Each sub-narrative may incorporate multiple streams of different types of media with each stream defining a “path” through content of that media type. The multimedia system directs the presentation of the sub-narratives by interfacing with presentation components for each media type through a custom interface component for that media type that implements a common application interface. When a user pauses a presentation, the user can manually navigate around the content of a stream from the current location at the time of the pause to another location. When the user resumes the presentation, the multimedia system automatically transitions from the navigated-to location to the current location at the time of the pause to resume the presentation from where it was paused. ## Claims ### Claim 1 A computing device with a processor and memory for generating a multimedia experience stream, comprising: a plurality of presentation components, each presentation component having a media type, each presentation component for presenting a stream of the media type and providing a type-specific interface for controlling the presenting of the stream, at least one presentation component having an orientation that includes location and provides a user interface for controlling the orientation as the stream is presented; for each of the plurality of presentation components, an interface component that provides a common application interface for controlling the presenting of streams and that maps interactions with the common application interface to interactions with the type-specific application interface of the presentation component, the common application interface providing a pause and a resume control for controlling the pausing and resuming of a stream, the pausing of at least one stream allowing a user to use the user interface of the presentation component to navigate from a current orientation of the paused stream to a resume orientation of the stream, the resuming of at least one stream for automatically transitioning from the resume orientation to the pause orientation and continuing the presenting of the stream from the pause orientation; a narrative definition defining the multimedia experience stream as a narrative with sub-narratives, each sub-narrative defining one or more streams that are part of that sub-narrative; and a narrative experience orchestrator component that, for each sub-narrative of the narrative definition, selects each stream of that sub-narrative and directs the interface component for the type of that stream to present the stream such that the streams of a sub-narrative are presented in parallel; upon receiving from a user an indication to pause, directs each interface component that is currently presenting a stream to pause; and upon receiving from the user an indication to resume, directs each interface component that is currently paused to resume. ### Claim 2 (depends on 1) The computing device of claim 1 wherein the transitioning from a resume orientation to a pause orientation includes generating a trajectory through content of the paused stream so that the pause orientation is attained after a transition period. ### Claim 3 (depends on 1) The computing device of claim 1 wherein the orientation includes a position and a direction associated with a view of the stream. ### Claim 4 (depends on 1) The computing device of claim 1 wherein a stream of a sub-narrative describes foreground and background audio of the sub-narrative such that while the stream is presented the output of the foreground audio is more dominant than the output of the background audio. ### Claim 5 (depends on 4) The computing device of claim 4 wherein when the stream is paused, the background audio is output while the foreground audio is not output. ### Claim 6 (depends on 1) The computing device of claim 1 wherein a stream with a frame-based media type of a sub-narrative defines a sequence of keyframes of the stream along with a view orientation of each keyframe. ### Claim 7 (depends on 6) The computing device of claim 6 wherein the interface component for a frame-based media type generates a sequence of frames of the stream that transitions from the view orientation of one keyframe to the view orientation of the next keyframe and submits the sequence of generated frames to the presentation component for the frame-based media type. ### Claim 8 (depends on 1) The computing device of claim 1 including a zone definition that defines zones within the multimedia experience stream, each zone associated with a region of orientations within the multimedia experience stream, and including an ambient experience orchestrator component that: receives an indication of a current orientation; and upon receiving an indication of a current orientation, provides the current orientation to a zone manager for the multimedia experience stream; receives from the zone manager an indication of active zones within the multimedia experience stream; and provides an indication of the active zones to the interface components of the multimedia experience stream so that the interface components can present ambient artifacts appropriate to the active zones. ### Claim 9 (depends on 8) The computing device of claim 8 wherein an ambient artifact is background audio associated with the current zone. ### Claim 10 A method in a computing device with a processor and memory for generating a multimedia experience stream, the method comprising: providing a plurality of presentation components for presenting a stream of a certain media type using a type-specific application interface for controlling the presenting of the stream; providing, for each of the plurality of presentation components, an interface component that implements a common application interface for controlling the presenting of a stream by mapping interactions with the common application interface to interactions with the type-specific application interface of the presentation component; providing a narrative definition defining the multimedia experience stream as a narrative with sub-narratives, each sub-narrative defining one or more streams that are part of that sub-narrative; and for each sub-narrative of the narrative definition, selecting each stream of that sub-narrative and directing the interface component for the media type of that stream to present the stream such that the streams of a sub-narrative are presented in parallel; upon receiving from a user an indication to pause, directing each interface component that is currently presenting a stream to pause at a pause location; and upon receiving from the user an indication to resume, directing each interface component that is currently paused to resume presenting the stream at the pause location wherein when a user navigated to a different location of the content of the stream during the pause, the interface component for the stream effects a transition from the new location to the pause location before resuming the stream from the pause location. ### Claim 11 (depends on 10) The method of claim 10 wherein the transitioning from the new location to the pause location is attained after a transition period and each interface component resumes presenting of its stream from the pause location after the transition period. ### Claim 12 (depends on 10) The method of claim 10 wherein an interface component pauses with a pause direction and, when resuming, transitions to the pause direction from a new direction resulting from user navigation during the pause. ### Claim 13 (depends on 10) The method of claim 10 wherein a stream of a sub-narrative describes foreground and background audio of the sub-narrative such that while the stream is presented, output of the foreground audio is more dominant than output of the background audio. ### Claim 14 (depends on 10) The method of claim 10 wherein a sub-narrative defines configuration information for a stream with a frame-based media type that is a sequence of keyframes of the stream along with an orientation for each keyframe. ### Claim 15 (depends on 14) The method of claim 14 wherein the interface component for the frame-based media type generates a sequence of frames of the stream that transitions from the orientation of one keyframe to the orientation of the next keyframe and submits the sequence of generated frames to a presentation component for the frame-based media type. ### Claim 16 (depends on 10) The method of claim 10 including a zone definition that defines zones within the multimedia experience stream, each zone associated with a region within the multimedia experience stream, and including: receiving an indication of a current orientation from a presentation component; and upon receiving an indication of a current orientation, providing the current orientation to a zone manager for the multimedia experience stream; receiving from the zone manager an indication of active zones within the multimedia experience stream; and providing an indication of the active zones to the interface components of the multimedia experience stream so that the interface components can present ambient artifacts appropriate to the active zones. ### Claim 17 A computer-readable storage media embedded with computer-readable instructions for controlling a computing device to generate a multimedia experience stream, comprising: instructions implementing a plurality of presentation components for presenting a stream of a media type and providing a type-specific application interface for controlling the presenting of the stream, at least one presentation component having an orientation that includes position and direction for viewing the stream and having a user interface allowing a user to navigate content of the stream; for each of the plurality of presentation components, instructions implementing an interface component that provides a common application interface for controlling the presenting of streams and that maps interactions with the common application interface to interactions with the type-specific application interface, the common application interface providing a pause and a resume function for pausing and resuming of a stream, the pausing of at least one stream allowing a user to use the user interface of a presentation component to navigate from a current orientation to a different orientation of content of the stream, the resuming of at least one stream for automatically transitioning from the different orientation to the current view orientation at the time of the pause and continuing the presenting of the stream from the current orientation; instructions for accessing a narrative definition defining the multimedia experience as a narrative with sub-narratives, each sub-narrative defining one or more streams that are part of that sub-narrative; and instructions that, for each sub-narrative of the narrative definition, select each stream of that sub-narrative and direct the interface component for the media type of that stream to present the stream such that the streams of a sub-narrative are presented in parallel; upon receiving from a user an indication to pause the multimedia experience stream, direct each interface component that is currently presenting a stream to pause; and upon receiving from the user an indication to resume, direct each interface component that is currently paused to resume presenting the paused stream. ### Claim 18 (depends on 17) The computer-readable storage medium of claim 17 wherein the transitioning from a different orientation to a current orientation includes generating a trajectory through the content of the stream so that the current orientation is attained after a transition period. ### Claim 19 (depends on 17) The computer-readable storage medium of claim 17 wherein a stream of a sub-narrative describes foreground and background audio of the sub-narrative such that while the stream is presented, output of the foreground audio is more dominant than output of the background audio. ### Claim 20 (depends on 17) The computer-readable storage medium of claim 17 including instructions to access a zone definition that defines zones within the multimedia experience stream, each zone associated with a region within the multimedia experience stream, and including instructions for: receiving an indication of a current orientation; and upon receiving an indication of a current orientation, providing the current orientation to a zone manager for the multimedia experience stream; receiving from the zone manager an indication of active zones within the multimedia experience stream; and providing an indication of the active zones to the interface components of the multimedia experience stream so that the interface components can present ambient artifacts appropriate to the active zones.