# US 9,521,470 B2 · Video delivery system configured to seek in a video using different modes > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,521,470 B2 - Document ID: US9521470B2 - Status: Granted - Kind: B2 - Application number: 14/738,425 - Filed: 2015-06-12 - Published: 2015-12-17 - Granted: 2016-12-13 - Earliest claimed: 2014-06-13 - Assignees: Hulu, LLC - Applicants: Hulu, LLC - Inventors: Tao Xiong; Zhibing Wang; Chenyang Cui; Cailiang Liu - Web page: https://patentgrep.com/patent/US9521470B2 ## CPC - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/47202: {for requesting content on demand, e.g. video on demand} - H04N 21/47217: {for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312)} - H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Particular embodiments configure a video delivery system to provide different modes for seeking in a video. The different modes may segment the video on different boundaries based on different characteristics of the video. For example, the different modes may seek by scene, by shot, and by dialogue. The boundaries for scenes, shots, and dialogue may start the video on logical points that do not break up the flow of the video. In another embodiment, the media player may save a seek history for a user and allow the user to scan the previous seek requests to go back to the seek times of previous seek requests. In one embodiment, the previous seek times are adjusted via the boundary information to show thumbnails for a shot, scene, or dialogue that correspond to the boundaries in the video rather than the original seek time. ## Claims ### Claim 1 A method comprising: sending, by a computing device, a request for a video using a media player; receiving, by the computing device, a plurality of boundary information sets for the video, the plurality of boundary information sets segmenting the video differently based on different video characteristics and the plurality of boundary information sets are associated with a plurality of seek modes; sending, by the computing device, requests for segments of the video; receiving, by the computing device, the segments for the video and playing the segments using the media player; receiving, by the computing device, a seek request for a seek time in the video; determining, by the computing device, a seek mode from the plurality of seek modes to use based on the seek time and two or more boundaries from two or more of the plurality of boundary information sets, wherein the two or more boundaries are associated with two or more scene boundaries, shot boundaries, and dialogue boundaries; determining, by the computing device, a segment in the video using the one of the plurality of boundary information sets associated with the determined seek mode, wherein the segment starts at a different time than the seek time; and sending, by the computing device, a request for the segment, wherein the media player plays the segment in response to the seek request. ### Claim 2 (depends on 1) The method of claim 1, wherein determining the segment comprises determining a boundary time closest to the seek time in the one of the plurality of boundary information sets, wherein the segment starts at the boundary time. ### Claim 3 (depends on 1) The method of claim 1, wherein determining the segment comprises determining one of the plurality of boundary information sets that is considered a best context for the seek time. ### Claim 4 (depends on 1) The method of claim 1, wherein determining the seek mode comprises: analyzing the seek time with respect to nearest boundaries in the plurality of boundary information sets. ### Claim 5 (depends on 1) The method of claim 1, wherein determining the seek mode comprises: analyzing a first boundary information set for scenes of the video to determine if the seek time is within a beginning of a first boundary in the first boundary information set to a maximum of the first boundary plus a time or a second boundary in a second boundary information set for shots. ### Claim 6 (depends on 5) The method of claim 5, wherein determining the seek mode comprises: analyzing the second boundary information set for shots of the video to determine if the seek time is within a beginning of the second boundary in the second boundary information set to a maximum of the second boundary plus a time or a third boundary in a third boundary information set for dialogue. ### Claim 7 (depends on 6) The method of claim 6, wherein determining the seek mode comprises: determining the third boundary in the third boundary information set for dialogue to use to determine the segment. ### Claim 8 (depends on 1) The method of claim 1, wherein the plurality of boundary information sets include a first boundary information set for shots in the video, a second boundary information set for scenes in the video, and a third boundary information set for dialogue in the video. ### Claim 9 (depends on 1) The method of claim 1, wherein: the video is segmented at different times than the boundaries in the plurality of boundary information sets. ### Claim 10 (depends on 1) The method of claim 1, further comprising: saving a seek history for a number of previous seek requests at seek times received for the video; and providing thumbnails for the number of previous seek requests to allow a user to seek to one of the number of previous seek requests. ### Claim 11 (depends on 10) The method of claim 10, wherein: the seek history changes the seeks times for the number of previous seek requests based on boundaries in at least one of the plurality of boundary information sets. ### Claim 12 A method comprising: preprocessing, by a computing device, a video to determine a plurality of boundary information sets for the video, the plurality of boundary information sets segmenting the video differently based on different video characteristics; receiving, by the computing device, a request for the video from a client device; and causing, by the computing device, the plurality of boundary information sets to be sent to the client device, wherein the media player plays the video, receives a seek request to a seek time in the video, and uses one of the plurality of boundary information sets to determine a segment of the video to request for playback based on the seek time and two or more boundaries from two or more of the plurality of boundary information sets, wherein the two or more boundaries are associated with two or more of scene boundaries, shot boundaries and dialogue boundaries. ### Claim 13 (depends on 12) The method of claim 12, wherein preprocessing comprises: processing the video to determine shots boundaries for shots in the video. ### Claim 14 (depends on 13) The method of claim 13, wherein preprocessing comprises: processing the video to determine scene boundaries for scenes in the video. ### Claim 15 (depends on 14) The method of claim 14, wherein preprocessing comprises: processing the video to determine dialogue boundaries for dialogue in the video, wherein the plurality of boundary information sets include a first boundary information set for shot boundaries in the video, a second boundary information set for scene boundaries in the video, and a third boundary information set for dialogue boundaries in the video. ### Claim 16 (depends on 15) The method of claim 15, wherein processing the video to determine dialogue boundaries for the dialogue in the video comprises: using speaker recognition or text analysis of captions for the video to determine the dialogue boundaries for the dialogue. ### Claim 17 (depends on 12) The method of claim 12, wherein: the video is segmented at different times than the boundaries in the plurality of boundary information sets. ### Claim 18 An apparatus comprising: one or more computer processors; and a non-transitory computer-readable storage medium comprising instructions, that when executed, control the one or more computer processors to be configured for: sending a request for a video using a media player; receiving a plurality of boundary information sets for the video, the plurality of boundary information sets segmenting the video differently based on different video characteristics and the plurality of boundary information sets are associated with a plurality of seek modes; sending requests for segments of the video; receiving the segments for the video and playing the segments using the media player; receiving a seek request for a seek time in the video; determining a seek mode from the plurality of seek modes to use based on the seek time and two or more boundaries from two or more of the plurality of boundary information sets, wherein the two or more boundaries are associated with two or more of scene boundaries, shot boundaries, and dialogue boundaries; determining a segment in the video using the one of the plurality of boundary information sets associated with the determined seek mode, wherein the segment starts at a different time than the seek time; and sending a request for the segment, wherein the media player plays the segment in response to the seek request. ### Claim 19 (depends on 18) The apparatus of claim 18, wherein determining the segment comprises determining a boundary time closest to the seek time in the one of the plurality of boundary information sets, wherein the segment starts at the boundary time. ### Claim 20 (depends on 18) The apparatus of claim 18, wherein the plurality of boundary information sets include a first boundary information set for shots in the video, a second boundary information set for scenes in the video, and a third boundary information set for dialogue in the video.