# US 9,438,937 B1 · Video server that provides a user tailored video stream consistent with user input using content of a primary stream and an enhanced stream > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,438,937 B1 - Document ID: US9438937B1 - Status: Granted - Kind: B1 - Application number: 14/721,110 - Filed: 2015-05-26 - Granted: 2016-09-06 - Earliest claimed: 2011-05-31 - Applicants: Brian K. Buchheit - Inventors: Brian K. Buchheit - Web page: https://patentgrep.com/patent/US9438937B1 ## CPC - H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region - H04N 5/04: Synchronising (for television systems using pulse code modulation H04N7/56) - H04N 5/45: Picture in picture {, e.g. displaying simultaneously another television channel in a region of the screen} - H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras} - H04N 21/2187: Live feed - H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement} - H04N 21/242: Synchronisation processes, e.g. processing of PCR [Programme Clock References] {(arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18)} - H04N 21/25891: {being end-user preferences (retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N 21/41265: {having a remote control device for bidirectional communication between the remote control device and client device (unidirectional remote control device H04N21/42204)} - H04N 21/43072: {of multiple content streams on the same device} - H04N 21/43079: {of additional data with content streams on multiple devices} - H04N 21/4312: {involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations} - H04N 21/8547: involving timestamps for synchronizing content - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An enhanced stream associated with a primary stream can be selected. The enhanced stream can be time-synchronized to the primary stream. The enhanced stream can be associated with a secondary viewpoint linked to a primary viewpoint of the primary stream. The primary stream can be a broadcast signal of a digital television broadcast. The enhanced stream can be simultaneously presented on a secondary device and the primary stream on a primary device. The primary device can be a computing device and the secondary device can be a computing device. ## Claims ### Claim 1 A method comprising: conveying, by a video server comprising hardware and software, video over a network to be played on a plurality of user playback devices, said video comprising media content of a geographic area which is concurrently covered by a first camera that captures a first portion of the geographic area in a first field of view and by a second camera that captures a second portion of the geographic area in a second field of view, wherein an overlapping area exists, wherein the first portion includes the overlapping area and a first non-overlapping area, wherein the second portion includes the overlapping area and a second non-overlapping area, wherein a primary stream of video content corresponds content including the overlapping area and the first non-overlapping area, wherein an enhanced stream of video content corresponds to content including the overlapping area and the second non-overlapping area; receiving, at the video server, a user input for user tailored video to be delivered to a user playback device, which is one of the user playback devices, wherein the user input is an input for an adjustment to shift-to-the right, to shift-to-left, to upwards-shift, or to downwards-shift video content shown on a screen of the user playback device, wherein at a time the user input was received the screen showed video content that did not include content from both the first portion and the second portion, wherein the adjustment as specified by the user input would result in playback video that shows at least part of the first portion, the overlapping area, and at least part of the second portion; responsive to receipt of the user input, the video server dynamically creating a user tailored video stream consistent with the user input, wherein the dynamically creating of the user tailored video stream stitches a region of the primary stream to a region from the enhanced stream together to produce the user tailored video stream comprising the at least part of the first portion, the overlapping area, and the at least part of the second portion; and responsive to the dynamically creating, conveying by the video server over the network the user tailored video stream to the user playback device, whereby the user playback device in receipt of the user tailored video stream is able to present the user tailored video stream that shows the at least part of the first portion, the overlapping area, and the at least part of the second portion consistent with the user input. ### Claim 2 (depends on 1) The method of claim 1, wherein the primary stream, the enhanced stream, and the user tailored video stream are time synchronized to each other. ### Claim 3 (depends on 1) The method of claim 1, further comprising: conveying the primary stream from the video server to a first video playback device of a user from which the user input was received; and concurrently conveying the user tailored video stream from the video server to a the user playback device, which is a second device of the user, wherein playback of the primary stream presented on the first video playback device is time synchronized with playback of the user tailored video stream of the user playback device. ### Claim 4 (depends on 1) The method of claim 1, wherein the primary stream is a broadcast stream of an event that is broadcasted by a broadcast source providing video coverage for the event. ### Claim 5 (depends on 1) The method of claim 1, wherein the user input is a portion of an continuous input to “pan” or “zoom” the screen in sequential increments of distance over time, wherein input is a continuous input comprising a first distance input at a first time, a second distance input at a second time, and a third distance input at a third time, wherein the adjustment represents a shifting of the first distance at the first time, wherein the receiving of the user input, the dynamically creating, and the conveying steps are repeated for the second distance input and for the third distance input. ### Claim 6 (depends on 5) The method of claim 5, wherein the user input is an input to pan the screen. ### Claim 7 (depends on 5) The method of claim 5, wherein the user input is an input to zoom the screen. ### Claim 8 (depends on 1) The method of claim 1, wherein the primary stream and the enhanced stream comprise video for a live event being concurrently captured by the first camera and the second camera. ### Claim 9 (depends on 1) The method of claim 1, wherein different user inputs are concurrently submitted to the video server by different users via the plurality of user playback devices, which results in the video server customizing user specific video streams per the different user inputs for the different users for playback on the plurality of user playback device. ### Claim 10 (depends on 1) The method of claim 1, wherein the stitched together region is a continuous video region stitched together along a horizontal or vertical plane. ### Claim 11 (depends on 1) The method of claim 1, wherein throughout the method, an audio output from the user playback device is substantially unaffected and is able to be audibly presented via the user playback device without substantial interruption. ### Claim 12 A video server comprising: at least one processor; at least one non-transitory memory storing program instructions, wherein the video server streams and time-synchronizes a primary stream, an enhanced stream, and user tailored streams of video to a plurality of user playback devices, wherein execution of the program instructions by the processor results in the video server: conveying video over a network to be played on the plurality of user playback devices, said video comprising media content of a geographic area which is concurrently covered by a first camera that captures a first portion of the geographic area in a first field of view and by a second camera that captures a second portion of the geographic area in a second field of view, wherein an overlapping area exists, wherein the first portion includes the overlapping area and a first non-overlapping area, wherein the second portion includes the overlapping area and a second non-overlapping area, wherein the primary stream of video content corresponds content including the overlapping area and the first non-overlapping area, wherein the enhanced stream of video content corresponds to content including the overlapping area and the second non-overlapping area; receiving, at the video server, a user input for user tailored video to be delivered to a user playback device, which is one of the user playback devices, wherein the user input is an input for an adjustment to shift-to-the right, to shift-to-left, to upwards-shift, or to downwards-shift video content shown on a screen of the user playback device, wherein at a time the user input was received the screen showed video content that did not include content from both the first portion and the second portion, wherein the adjustment as specified by the user input would result in playback video that shows at least part of the first portion, the overlapping area, and at least part of the second portion; responsive to receipt of the user input, the video server dynamically creating a user tailored video stream consistent with the user input, wherein the dynamically creating of the user tailored video stream stitches a region of the primary stream to a region from the enhanced stream together to produce the user tailored video stream comprising the at least part of the first portion, the overlapping area, and the at least part of the second portion; and responsive to the dynamically creating, conveying by the video server over the network the user tailored video stream to the user playback device, whereby the user playback device in receipt of the user tailored video stream is able to present the user tailored video stream that shows the at least part of the first portion, the overlapping area, and the at least part of the second portion consistent with the user input. ### Claim 13 (depends on 12) The video server of claim 12, wherein the primary stream, the enhanced stream, and the user tailored video stream are time synchronized to each other. ### Claim 14 (depends on 12) The video server of claim 12, wherein execution of the program instructions by the processor results in the video server: conveying the primary stream from the video server to a first video playback device of a user from which the user input was received; and concurrently conveying the user tailored video stream from the video server to a the user playback device, which is a second device of the user, wherein playback of the primary stream presented on the first video playback device is time synchronized with playback of the user tailored video stream of the user playback device. ### Claim 15 (depends on 12) The video server of claim 12, wherein the primary stream is a broadcast stream of an event that is broadcasted by a broadcast source providing video coverage for the event. ### Claim 16 (depends on 12) The video server of claim 12, wherein the primary stream and the enhanced stream comprise video for a live event being concurrently captured by the first camera and the second camera. ### Claim 17 A method comprising: conveying, by a video server comprising hardware and software, video over a network to be played on a plurality of user playback devices, said video comprising media content of a geographic area which is concurrently covered by a first camera that captures a first portion of the geographic area in a first field of view and by a second camera that captures a second portion of the geographic area in a second field of view, wherein an overlapping area exists, wherein the first portion includes the overlapping area and a first non-overlapping area, wherein the second portion includes the overlapping area and a second non-overlapping area, wherein a primary stream of video content corresponds content including the overlapping area and the first non-overlapping area, wherein an enhanced stream of video content corresponds to content including the overlapping area and the second non-overlapping area; receiving, at the video server, a user input for user tailored video to be delivered to a user playback device, which is one of the user playback devices, wherein the user input is to pan, wherein a pan operation causes a shift-to-the-right, a shift-to-the-left, an upwards-shift, or a downwards-shift to a region shown on the video playback device, wherein the pan operation occurs in sequential increments of distance, wherein at a time the user input was received the screen showed video content that did not include content from both the first portion and the second portion, wherein an adjustment to pan as specified by the user input would result in playback video that shows at least part of the first portion, the overlapping area, and at least part of the second portion; responsive to receipt of the user input, the video server dynamically creating a user tailored video stream consistent with the user input, wherein the dynamically creating of the user tailored video stream upon which a pan operation is represented, wherein during the pan operation in which incremental shifts occur: (i) reaching a regional boundary of the primary stream of content during the pan operation; (ii) responsive to reaching the regional boundary, stitching a region of the primary stream to a region from an enhanced stream together; and (iii) creating a resulting spliced region as video output, responsive to the dynamically creating, conveying by the video server over the network the user tailored video stream to the user playback device, whereby the user playback device in receipt of the user tailored video stream is able to present the user tailored video stream that shows the at least part of the first portion, the overlapping area, and the at least part of the second portion consistent with the user input and that shows results of the pan operation as video output consistent with the user input to pan. ### Claim 18 (depends on 17) The method of claim 17, wherein the primary stream is a broadcast stream of an event that is broadcasted by a broadcast source providing video coverage for the event. ### Claim 19 (depends on 1) The method of claim 1, wherein different user inputs are concurrently submitted to the video server by different users via the plurality of user playback devices, which results in the video server customizing user specific video streams per the different user inputs for the different users for playback on the plurality of user playback device. ### Claim 20 (depends on 1) The method of claim 1, wherein the video server streams and time-synchronizes the primary stream, the enhanced stream, and user tailored streams of video to the plurality of user playback devices.