# US 2026/0230667 A1 · Advertisement break detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0230667 A1
- Document ID: US20260230667A1
- Status: Published application
- Kind: A1
- Application number: 19/635,585
- Filed: 2026-03-31
- Published: 2026-08-06
- Earliest claimed: 2023-04-17
- Assignees: Tubi, Inc.
- Applicants: Tubi, Inc.
- Inventors: Amir Mazaheri; John Trenkle; Jaya Kawale; Kevin Edward Corcoran; Dennis Paul Yost
- Web page: https://patentgrep.com/patent/US20260230667A1

## CPC

- H04N 21/26241: {involving the time of distribution, e.g. the best time of the day for inserting an advertisement or airing a children programme}
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system and method for ad break detection using bipartite matching on video shot analysis is disclosed. A scene break detection service performs audio break detection and video shot detection on a media item, then identifies candidate ad break timestamps where audio and video break timestamps coincide within a predefined proximity. For each candidate, the service identifies contiguous shots preceding and succeeding the candidate timestamp and constructs a graph representation of the two shot groups. Bipartite matching is performed on the graph to generate distance values between shots of the two groups. A scene break score is calculated from the distance values, representing a visual distance across the candidate timestamp. Final ad break timestamps are selected based on the scores, and advertisements are served during media playback at the selected timestamps. Systems, methods, and computer-readable media for implementing the technique are described.

## Claims

### Claim 1

A system for advertising (ad) break detection, comprising:
a computer processor;
a scene break detection service executing on the computer processor and comprising functionality to:
perform audio break detection on an audio component of a media item to obtain a set of audio break timestamps;
identify a set of video break timestamps of a video component of the media item;
identify a set of candidate ad break timestamps corresponding to instances of the set of audio break timestamps and the set of video break timestamps within a predefined proximity;
execute a computer vision scoring model for each candidate ad break timestamp of the set of candidate ad break timestamps by:
identifying a first subset of a set of contiguous shots preceding the candidate ad break timestamp and a second subset of the set of contiguous shots succeeding the candidate ad break timestamp;
constructing a graph representation of the first subset and the second subset;
performing bipartite matching on the graph representation to generate a set of distance values; and
calculating a scene break score for the candidate ad break timestamp based on the set of distance values; and
select, based at least on the scene break score of each of the set of candidate ad break timestamps, a final set of ad break timestamps from the set of candidate ad break timestamps, wherein an advertisement is served for display by an end user application in proximity to a first ad break timestamp of the final set of ad break timestamps during playback of the media item.

### Claim 2 (depends on 1)

The system of claim 1, wherein each of the set of distance values represents a distance between a shot of the first subset and a shot of the second subset, and wherein the scene break score represents a visual distance between the first subset and the second subset.

### Claim 3 (depends on 2)

The system of claim 2, wherein the scene break score is an average of the set of distance values.

### Claim 4 (depends on 1)

The system of claim 1, wherein the first subset and the second subset each comprise a predefined number of contiguous shots, the predefined number defining a scope of analysis preceding and succeeding the candidate ad break timestamp.

### Claim 5 (depends on 1)

The system of claim 1, wherein executing the computer vision scoring model further comprises generating a set of feature vectors by generating a feature vector for each shot of the first subset and the second subset using a convolutional neural network, and wherein each of the set of distance values is calculated based on the set of feature vectors.

### Claim 6 (depends on 5)

The system of claim 5, wherein generating the feature vector for each shot comprises selecting a representative frame of the shot and processing the representative frame through the convolutional neural network to generate the feature vector.

### Claim 7 (depends on 1)

The system of claim 1, wherein performing bipartite matching on the graph representation comprises solving an optimal assignment problem using a Hungarian algorithm to minimize a total of the set of distance values.

### Claim 8 (depends on 1)

The system of claim 1, wherein the scene break detection service further comprises functionality to:
identify a streaming manifest of the media item;
identify a set of streaming segment boundary timestamps defined by the streaming manifest; and
eliminate, from the set of candidate ad break timestamps, candidate ad break timestamps that fall outside a predefined distance of the set of streaming segment boundary timestamps prior to executing the computer vision scoring model.

### Claim 9 (depends on 1)

The system of claim 1, wherein selecting the final set of ad break timestamps comprises executing a dynamic programming process to select an optimal subset of the set of candidate ad break timestamps, the dynamic programming process subject to at least one constraint selected from a group consisting of: (i) a minimum distance between any two consecutive ad break timestamps of the final set of ad break timestamps; (ii) a minimum elapsed time from a start of the media item before a first ad break timestamp of the final set of ad break timestamps; and (iii) a minimum remaining time before an end of the media item after a final ad break timestamp of the final set of ad break timestamps.

### Claim 10 (depends on 1)

The system of claim 1, wherein the computer vision scoring model comprises a machine learning model trained using a labeled dataset comprising media items and tagged advertisement timestamps, and wherein the scene break detection service further comprises functionality to perform online training of the machine learning model using a retention rate of an advertisement displayed at a selected ad break timestamp of the final set of ad break timestamps as a training label.

### Claim 11 (depends on 1)

The system of claim 1, wherein performing audio break detection comprises:
executing a voice activity detection model to identify segments of human speech in the audio component; and
performing natural language processing on a transcription of the audio component to identify a set of contextual boundaries, wherein the set of audio break timestamps are identified based on the set of contextual boundaries.

### Claim 12

A method for advertising (ad) break detection, comprising:
performing audio break detection on an audio component of a media item to obtain a set of audio break timestamps;
identifying a set of video break timestamps of a video component of the media item;
identifying a set of candidate ad break timestamps corresponding to instances of the set of audio break timestamps and the set of video break timestamps within a predefined proximity;
executing, by a computer processor, a computer vision scoring model for each candidate ad break timestamp of the set of candidate ad break timestamps by:
identifying a first subset of a set of contiguous shots preceding the candidate ad break timestamp and a second subset of the set of contiguous shots succeeding the candidate ad break timestamp;
constructing a graph representation of the first subset and the second subset;
performing bipartite matching on the graph representation to generate a set of distance values; and
calculating a scene break score for the candidate ad break timestamp based on the set of distance values; and
selecting, based at least on the scene break score of each of the set of candidate ad break timestamps, a final set of ad break timestamps from the set of candidate ad break timestamps, wherein an advertisement is served for display by an end user application in proximity to a first ad break timestamp of the final set of ad break timestamps during playback of the media item.

### Claim 13 (depends on 12)

The method of claim 12, wherein each of the set of distance values represents a distance between a shot of the first subset and a shot of the second subset, and wherein the scene break score represents a visual distance between the first subset and the second subset.

### Claim 14 (depends on 13)

The method of claim 13, wherein the scene break score is an average of the set of distance values.

### Claim 15 (depends on 12)

The method of claim 12, wherein the first subset and the second subset each comprise a predefined number of contiguous shots, the predefined number defining a scope of analysis preceding and succeeding the candidate ad break timestamp.

### Claim 16 (depends on 12)

The method of claim 12, wherein executing the computer vision scoring model further comprises generating a set of feature vectors by generating a feature vector for each shot of the first subset and the second subset using a convolutional neural network, and wherein each of the set of distance values is calculated based on the set of feature vectors.

### Claim 17 (depends on 12)

The method of claim 12, wherein performing bipartite matching on the graph representation comprises solving an optimal assignment problem using a Hungarian algorithm to minimize a total of the set of distance values.

### Claim 18 (depends on 12)

The method of claim 12, wherein selecting the final set of ad break timestamps comprises executing a dynamic programming process to select an optimal subset of the set of candidate ad break timestamps, the dynamic programming process subject to at least one constraint selected from a group consisting of: (i) a minimum distance between any two consecutive ad break timestamps of the final set of ad break timestamps; (ii) a minimum elapsed time from a start of the media item before a first ad break timestamp of the final set of ad break timestamps; and (iii) a minimum remaining time before an end of the media item after a final ad break timestamp of the final set of ad break timestamps.

### Claim 19 (depends on 12)

The method of claim 12, wherein the computer vision scoring model comprises a machine learning model trained using a labeled dataset comprising media items and tagged advertisement timestamps, and wherein the method further comprises performing online training of the machine learning model using a retention rate of an advertisement displayed at a selected ad break timestamp of the final set of ad break timestamps as a training label.

### Claim 20

A non-transitory computer-readable storage medium comprising a plurality of instructions for advertising (ad) break detection, the plurality of instructions configured to execute on at least one computer processor to enable the at least one computer processor to:
perform audio break detection on an audio component of a media item to obtain a set of audio break timestamps;
identify a set of video break timestamps of a video component of the media item; identify a set of candidate ad break timestamps corresponding to instances of the set of audio break timestamps and the set of video break timestamps within a predefined proximity;
execute a computer vision scoring model for each candidate ad break timestamp of the set of candidate ad break timestamps by:
identify a first subset of a set of contiguous shots preceding the candidate ad break timestamp and a second subset of the set of contiguous shots succeeding the candidate ad break timestamp;
construct a graph representation of the first subset and the second subset;
perform bipartite matching on the graph representation to generate a set of distance values; and
calculate a scene break score for the candidate ad break timestamp based on the set of distance values; and
select, based at least on the scene break score of each of the set of candidate ad break timestamps, a final set of ad break timestamps from the set of candidate ad break timestamps, wherein an advertisement is served for display by an end user application in proximity to a first ad break timestamp of the final set of ad break timestamps during playback of the media item.
