# US 12,192,550 B2 · Time marking of media items at a platform using machine learning > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,550 B2 - Document ID: US12192550B2 - Status: Granted - Kind: B2 - Application number: 17/746,818 - Filed: 2022-05-17 - Published: 2023-11-23 - Granted: 2025-01-07 - Earliest claimed: 2022-05-17 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Dongeek Shin - Web page: https://patentgrep.com/patent/US12192550B2 ## CPC - H04N 21/251: {Learning process for intelligent management, e.g. learning user preferences for recommending movies (details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08)} - H04N 21/47217: {for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312)} - H04N 21/8455: {involving pointers to the content, e.g. pointers to the I-frames of the video stream} - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Methods and systems for time marking of media items at a platform using machine learning are provided herein. A media item to be provided to users of a platform is identified. The media item includes two or more content segments. An indication of the identified media item is provided as input to a machine learning model. The machine learning model is trained using to predict, for a given media item, content segments of the given media item depicting an event of interest to the one or more users. One or more outputs of the machine learning model are obtained. The one or more obtained outputs include event data identifying each content segment of the media item and an indication of a level of confidence that each respective content segment depicts an event of interest. In response to determining that at least one content segment is associated with a level of confidence that satisfies a level of confidence criterion, the at least one content segment is associated with a bookmark for a timeline of the media item. The media item and an indication of the bookmark is provided for presentation to the at least one user. ## Claims ### Claim 1 A method comprising: identifying a media item to be provided to users of a platform, wherein the media item comprises a plurality of content segments; providing an indication of the identified media item as input to a machine learning model; obtaining one or more outputs of the machine learning model, wherein the one or more obtained outputs comprise event data identifying each of the plurality of content segments of the media item and, for each of the plurality of content segments, an indication of a level of confidence that a respective content segment depicts an event of interest to a first plurality of users of the platform; responsive to determining that at least one of the plurality of content segments is associated with a level of confidence that satisfies a level of confidence criterion, associating the at least one of the plurality of content segments with a bookmark for a timeline of the media item; and responsive to a request from a client device associated with a second user outside of the first plurality of users of the platform for access to the media item, providing the media item and an indication of the bookmark for presentation via a user interface of the client device. ### Claim 2 (depends on 1) The method of claim 1, further comprising: responsive to providing the media item and an indication of the bookmark for presentation via the user interface of the client device, receiving an indication of an interesting content segment of the media item as selected by a respective user of the platform; determining whether the indicated interesting content segment corresponds to the at least one of the plurality of content segments associated with the bookmark; responsive to determining that the indicated interesting content segment corresponds to the at least one of the plurality of content segments associated with the bookmark, updating the bookmark to correspond to the indicated interesting content segment; and providing an indication of the updated bookmark for presentation to the respective user. ### Claim 3 (depends on 2) The method of claim 2, further comprising: responsive to determining that the indicated interesting content segment does not correspond to the at least one of the plurality of content segments associated with the bookmark, associating the indicated interesting content segment with an additional bookmark for the timeline of the media item; and providing an indication of the additional bookmark for presentation to the respective user. ### Claim 4 (depends on 3) The method of claim 3, wherein the indication of at least one of the updated bookmark or the additional bookmark is also provided for presentation to other users of the platform. ### Claim 5 (depends on 1) The method of claim 1, further comprising: detecting a user engagement with a user interface (UI) element of the UI of the client device, wherein the UI element corresponds to the bookmark; and initiating playback of the at least one of the plurality of content segments associated with the bookmark via the client device. ### Claim 6 (depends on 1) The method of claim 1, wherein the media item is identified responsive to a request from an additional client device associated with a creator of the media item to provide user access to the media item via the platform. ### Claim 7 (depends on 1) The method of claim 1, wherein the media item comprises at least one of a video item or an audio item. ### Claim 8 A system comprising: a memory device; and a processing device coupled to the memory device, the processing device to perform operations comprising: identifying a media item to be provided to users of a platform, wherein the media item comprises a plurality of content segments; providing an indication of the identified media item as input to a machine learning model; obtaining one or more outputs of the machine learning model, wherein the one or more obtained outputs comprise event data identifying each of the plurality of content segments of the media item and, for each of the plurality of content segments, an indication of a level of confidence that a respective content segment depicts an event of interest to a first plurality of users of the platform; responsive to determining that at least one of the plurality of content segments is associated with a level of confidence that satisfies a level of confidence criterion, associating the at least one of the plurality of content segments with a bookmark for a timeline of the media item; and responsive to a request from a client device associated with a second user outside of the first plurality of users of the platform for access to the media item, providing the media item and an indication of the bookmark for presentation via a user interface of the client device. ### Claim 9 (depends on 8) The system of claim 8, wherein the operations further comprise: responsive to providing the media item and an indication of the bookmark for presentation via the user interface of the client device, receiving an indication of an interesting content segment of the media item as selected by a respective user of the platform; determining whether the indicated interesting content segment corresponds to the at least one of the plurality of content segments associated with the bookmark; responsive to determining that the indicated interesting content segment corresponds to the at least one of the plurality of content segments associated with the bookmark, updating the bookmark to correspond to the indicated interesting content segment; and providing an indication of the updated bookmark for presentation to the respective user. ### Claim 10 (depends on 9) The system of claim 9, wherein the operations further comprise: responsive to determining that the indicated interesting content segment does not correspond to the at least one of the plurality of content segments associated with the bookmark, associating the indicated interesting content segment with an additional bookmark for the timeline of the media item; and providing an indication of the additional bookmark for presentation to the respective user. ### Claim 11 (depends on 10) The system of claim 10, wherein the indication of at least one of the updated bookmark or the additional bookmark is also provided for presentation to other users of the platform. ### Claim 12 (depends on 8) The system of claim 8, wherein the operations further include: detecting a user engagement with a user interface (UI) element of the UI of the client device, wherein the UI element corresponds to the bookmark; and initiating playback of the at least one of the plurality of content segments associated with the bookmark via the client device. ### Claim 13 (depends on 8) The system of claim 8, wherein the media item is identified responsive to a request from an additional client device associated with a creator of the media item to provide user access to the media item via the platform. ### Claim 14 A non-transitory computer readable storage medium comprising instructions for a server that, when executed by a processing device, cause the processing device to perform operations comprising: identifying a media item to be provided to users of a platform, wherein the media item comprises a plurality of content segments; providing an indication of the identified media item as input to a machine learning model; obtaining one or more outputs of the machine learning model, wherein the one or more obtained outputs comprise event data identifying each of the plurality of content segments of the media item and, for each of the plurality of content segments, an indication of a level of confidence that a respective content segment depicts an event of interest to a first plurality of users of the platform; responsive to determining that at least one of the plurality of content segments is associated with a level of confidence that satisfies a level of confidence criterion, associating the at least one of the plurality of content segments with a bookmark for a timeline of the media item; and responsive to a request from a client device associated with a second user outside of the first plurality of users of the platform for access to the media item, providing the media item and an indication of the bookmark for presentation via a user interface of the client device. ### Claim 15 (depends on 14) The non-transitory computer readable storage medium of claim 14, wherein the operations further comprise: responsive to providing the media item and an indication of the bookmark for presentation via the user interface of the client device, receiving an indication of an interesting content segment of the media item as selected by a respective user of the platform; determining whether the indicated interesting content segment corresponds to the at least one of the plurality of content segments associated with the bookmark; responsive to determining that the indicated interesting content segment corresponds to the at least one of the plurality of content segments associated with the bookmark, updating the bookmark to correspond to the indicated interesting content segment; and providing an indication of the updated bookmark for presentation to the respective user. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the operations further comprise: responsive to determining that the indicated interesting content segment does not correspond to the at least one of the plurality of content segments associated with the bookmark, associating the indicated interesting content segment with an additional bookmark for the timeline of the media item; and providing an indication of the additional bookmark for presentation to the respective user. ### Claim 17 (depends on 16) The non-transitory computer readable storage medium of claim 16, wherein the indication of at least one of the updated bookmark or the additional bookmark is also provided for presentation to other users of the platform. ### Claim 18 (depends on 14) The non-transitory computer readable storage medium of claim 14, wherein the operations further comprise: detecting a user engagement with a user interface (UI) element of the UI of the client device, wherein the UI element corresponds to the bookmark; and initiating playback of the at least one of the plurality of content segments associated with the bookmark via the client device. ### Claim 19 (depends on 14) The non-transitory computer readable storage medium of claim 14, wherein the media item is identified responsive to a request from an additional client device associated with a creator of the media item to provide user access to the media item via the platform. ### Claim 20 (depends on 14) The non-transitory computer readable storage medium of claim 14, wherein the media item comprises at least one of a video item or an audio item.