# US 12,192,544 B2 · Determining watch time loss regions in media content items > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,544 B2 - Document ID: US12192544B2 - Status: Granted - Kind: B2 - Application number: 18/428,359 - Filed: 2024-01-31 - Published: 2024-05-23 - Granted: 2025-01-07 - Earliest claimed: 2024-01-31 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Wenbo Zhang; Kartikey Gupta - Web page: https://patentgrep.com/patent/US12192544B2 ## CPC - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - G06N 3/08: Learning methods - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 10/806: {of extracted features} - G06V 10/82: using neural networks - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement} - H04N 21/25: Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies {(maintenance or administration in data networks H04L41/00)} - H04N 21/251: {Learning process for intelligent management, e.g. learning user preferences for recommending movies (details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08)} - H04N 21/26241: {involving the time of distribution, e.g. the best time of the day for inserting an advertisement or airing a children programme} - H04N 21/442: Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk {(arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00)} - H04N 21/44222: {Analytics of user selections, e.g. selection of programmes or purchase activity (monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33)} - H04N 21/658: Transmission by the client directed to the server - H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)} - H04N 21/854: Content authoring - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining watch time loss regions in media content items. In one aspect, features for a video are input into a trained model that is trained to output watch time loss regions. The trained model is trained using labels corresponding to known watch time loss regions in training videos and features of training videos that correspond to the known watch time loss regions. A watch time loss region defines a time window of a video during which a likelihood of a user stopping playback of the video is more than a threshold likelihood. In response to inputting the feature for the first video into the trained model, data regarding watch time loss regions for the video is obtained from the model and provided to an entity involved in providing the video to a user. ## Claims ### Claim 1 A computer-implemented method comprising: determining a set of features for a first video; inputting the set of features for the first video into a trained model that is trained to output one or more watch time loss regions in an input video, wherein a watch time loss region specifies a time window of the input video during which input video playback is expected to be to be stopped, wherein the trained model is trained using features and known watch time loss regions of a plurality of training videos; in response to inputting the set of features for the first video into the trained model, obtaining, from the trained model, watch time loss region data for the first video; and providing, to an entity involved in providing the first video to a user, the watch time loss region data for the first video. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the watch time loss region data for the first video is provided to a content provider that uses the watch time loss region data for the first video to determine a time point in the first video at which to play a digital component that includes different content than content in the first video. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein the content provider uses the watch time loss region data for the first video to determine serving decisions for when to provide digital content to a client device during playback of the first video. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the watch time loss region data for the first video is provided to an owner of the first video, the method further comprising: receiving an updated version of the first video that has been modified by the owner of the first video; processing the updated version of the first video using the trained model to generate updated data regarding watch time loss regions for the updated version of the first video; and providing the updated data regarding the watch time loss regions for the updated version of the first video to the owner of the first video. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein training the trained model comprises: for each training video in the plurality of training videos: generating a watch time loss curve for the training video; determining watch time loss regions in the watch time loss curve; and training the trained model using the watch time loss regions in watch time loss curves for the plurality of training videos. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein determining watch time loss regions in the watch time loss curve comprises: determining an average slope of the watch time loss curve; and determining, as the watch time loss regions in the watch time loss curve, regions of the watch time loss curve that have a slope that is greater than the average slope. ### Claim 7 (depends on 5) The computer-implemented method of claim 5, wherein training the trained model using the watch time loss regions in the watch time loss curves for the plurality of training videos comprises training the trained model using watch time loss regions other than a first watch time loss region at a beginning of the watch time loss curve and a last watch time loss region at an end of the watch time loss curve. ### Claim 8 (depends on 5) The computer-implemented method of claim 5, wherein training the trained model using the watch time loss regions in the watch time loss curves comprises: determining, for each watch time loss region determined for a particular training video, features of the particular training video that occur within a predetermined time window of the watch time loss region; and training the trained model using determined features of the particular training video. ### Claim 9 (depends on 5) The computer-implemented method of claim 5, wherein the watch time loss curve for a particular training video is generated from interaction data obtained from a user activity log that includes start and stop times of user viewing sessions for the particular training video. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the trained model comprises a bidirectional recurrent neural network and a fully-connected deep neural network. ### Claim 11 (depends on 1) The computer-implemented method of claim 1, wherein the set of features for the first video and features of the plurality of training videos that correspond to the known watch time loss regions comprise video features, audio features, or semantic features. ### Claim 12 A system, comprising: one or more memory devices storing instructions; and one or more data processing apparatus that are configured to interact with the one or more memory devices, and upon execution of the instructions, perform operations comprising: determining a set of features for a first video; inputting the set of features for the first video into a trained model that is trained to output one or more watch time loss regions in an input video, wherein a watch time loss region specifies a time window of the input video during which input video playback is expected to be to be stopped, wherein the trained model is trained using features and known watch time loss regions of a plurality of training videos; in response to inputting the set of features for the first video into the trained model, obtaining, from the trained model, watch time loss region data for the first video; and providing, to an entity involved in providing the first video to a user, the watch time loss region data for the first video. ### Claim 13 (depends on 12) The system of claim 12, wherein the watch time loss region data for the first video is provided to a content provider that uses the watch time loss region data for the first video to determine a time point in the first video at which to play a digital component that includes different content than content in the first video. ### Claim 14 (depends on 12) The system of claim 12, wherein a content provider uses the watch time loss region data for the first video to determine serving decisions for when to provide digital content to a client device during playback of the first video. ### Claim 15 (depends on 13) The system of claim 13, wherein the watch time loss region data for the first video is provided to an owner of the first video, the operations further comprising: receiving an updated version of the first video that has been modified by the owner of the first video; processing the updated version of the first video using the trained model to generate updated data regarding watch time loss regions for the updated version of the first video; and providing the updated data regarding the watch time loss regions for the updated version of the first video to the owner of the first video. ### Claim 16 (depends on 12) The system of claim 12, wherein training the trained model comprises: for each training video in the plurality of training videos: generating a watch time loss curve for the training video; determining watch time loss regions in the watch time loss curve; and training the trained model using the watch time loss regions in watch time loss curves for the plurality of training videos. ### Claim 17 (depends on 16) The system of claim 16, wherein determining watch time regions in the watch time loss curve comprises: determining an average slope of the watch time loss curve; and determining, as the watch time loss regions in the watch time loss curve, regions of the watch time loss curve that have a slope that is greater than the average slope. ### Claim 18 A non-transitory computer-readable medium storing instructions that, when executed by one or more data processing apparatus, cause the one or more data processing apparatus to perform operations, comprising: determining a set of features for a first video; inputting the set of features for the first video into a trained model that is trained to output one or more watch time loss regions in an input video, wherein a watch time loss region specifies a time window of the input video during which input video playback is expected to be to be stopped, wherein the trained model is trained using features and known watch time loss regions of a plurality of training videos; in response to inputting the set of features for the first video into the trained model, obtaining, from the trained model, watch time loss region data for the first video; and providing, to an entity involved in providing the first video to a user, the watch time loss region data for the first video. ### Claim 19 (depends on 18) The non-transitory computer-readable medium of claim 18, wherein the watch time loss region data for the first video is provided to a content provider that uses the watch time loss region data for the first video to determine a time point in the first video at which to play a digital component that includes different content than content in the first video. ### Claim 20 (depends on 18) The non-transitory computer-readable medium of claim 18, wherein a content provider uses watch time loss region data for the first video to determine serving decisions for when to provide digital content to a client device during playback of the first video.