# US 7,613,691 B2 · Dynamic insertion of supplemental video based on metadata

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 7,613,691 B2
- Document ID: US7613691B2
- Status: Granted
- Kind: B2
- Application number: 11/425,570
- Filed: 2006-06-21
- Published: 2007-12-27
- Granted: 2009-11-03
- Earliest claimed: 2006-06-21
- Assignees: Microsoft Corporation
- Inventors: Gregory D. Finch
- Web page: https://patentgrep.com/patent/US7613691B2

## CPC

- H04N 21/44213: {Monitoring of end-user related data (arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33)}
- G06Q 30/02: Marketing; Price estimation or determination; Fundraising
- H04N 7/173: with two-way working, e.g. subscriber sending a programme selection signal
- H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement}
- H04N 21/4126: {The peripheral being portable, e.g. PDAs or mobile phones}
- H04N 21/4331: {Caching operations, e.g. of an advertisement for later insertion during playback}
- H04N 21/43615: {Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals (home Audio Video Interoperability [HAVI] data switching networks H04L12/2805)}
- H04N 21/44016: {involving splicing one content stream with another content stream, e.g. for substituting a video clip}
- H04N 21/472: End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content {(end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612)}
- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A media service application responsive to a media retrieval request received from a user of client computer to retrieve a video designated by the retrieval request. The media service retrieves relevant advertisements as a function of metadata included in the video file and/or stored personal preference data of the user. If the retrieval request is an online viewing request the media service application integrates the retrieved advertisements into the retrieved video and transfers an integrated video/advertisement file to the client If the retrieval request is a download request the media service application transfers the retrieved ad and retrieved video to the client as separate files for integration by the client during playback of the video.

## Claims

### Claim 1

A computerized method for integrating an advertisement clip into a video file being transferred from a server to a client for playback, said video file including one or more scenes, said computerized method comprising computer executable instructions for:
receiving, at the server, input from a user of the client designating the video file to be transferred, said video file including one or more scenes;
extracting metadata from the designated video file for each of the one or more scenes using a computerized metadata detection technique;
determining a value of a predefined metadata property associated with each of the one or more scenes;
determining the topic of each of the one or more scenes included in the designated video file based on the extracted metadata and the predefined metadata property;
querying an advertisement database, said advertisement database storing a plurality of advertisements clips and, for each of the plurality of advertisement clips, identifying a topic of said each advertisement clip;
comparing, for each of the one or more scenes, the determined topic of said each scene to the identified topic of at least one of the plurality of advertisement clips stored in the advertisement database;
determining, for each of the one or more scenes, whether an advertisement clip in the advertisement database is relevant to said each scene based on the comparing; and
transferring the designated video file and the determined relevant advertisement clips to the client for playback.

### Claim 2 (depends on 1)

The method of claim 1 further including:
retrieving personal preference data for the user of the client when no advertisement clip in the advertisement database is determined relevant to said each scene; and
determining a relevant advertisement clip as a function of the retrieved personal preference data.

### Claim 3 (depends on 1)

The method of claim 1 further including editing the video file to include the determined relevant advertisement clips, and wherein the transferring includes transferring the edited video file to the client.

### Claim 4 (depends on 1)

The method of claim 1 extracting metadata from the designated video file for each of the one or more scenes using a computerized metadata detection technique includes extracting metadata from the designated video file for each of the one or more scenes using one or more of the following metadata detection techniques: face detection, dynamic image peak detection, color value detection, dynamic image change detection face recognition, music beats detection, audio fingerprint detection, dynamic peaks detection, speech detection, word and phrase detection.

### Claim 5

A system for selecting an advertisement clip for insertion into a video file being transferred from a server to a client for playback, said system comprising:
a processor configured to execute computer-executable components; and
a memory having stored therein:
a retrieval component responsive to a retrieval request designating a video file for transfer from the server to the client to retrieve the designated video file from a media library linked to said server, said designated video file including one or more video clips;
an extraction component for extracting metadata from each of the video clips included in the designated video file using a computerized metadata detection technique, determining a value of a predefined metadata property associated with each of one or more scenes included in each video clip and determining a topic of the one or more scenes based on the extracted metadata and the predefined metadata property;
and wherein said retrieval component is further responsive to said extracted metadata for:
querying an advertisement library, said advertisement library storing a plurality of advertisements clips and, for each of the plurality of advertisement clips, identifying a topic of said each advertisement clip;
comparing, for each of the one or more scenes, the determined topic of said each scene to the identified topic of at least one of the plurality of advertisement clips stored in the advertisement library;
retrieving one or more advertisement clips relevant to the one or more scenes based on the comparing; and
a transfer component for transferring the designated video file and the relevant one or advertisement clips to the client.

### Claim 6 (depends on 5)

The system of claim 5, wherein the retrieval component further includes retrieving personal preference data for the user of the client, and retrieving one or more advertisement clips as a function of the retrieved personal preference data.

### Claim 7 (depends on 5)

The system of claim 5, wherein a storage component is responsive to user input defining personal preference data to store the defined personal preference data in a preference database.

### Claim 8 (depends on 5)

The system of claim 5 further including:
an authentication component for authenticating the retrieval request; and
a display component responsive to an authenticated retrieval request to transfer the names of the various video files available in the media library to the requesting client for display via a display linked to the client computer.

### Claim 9 (depends on 5)

The system of claim 5, wherein the transfer component transfers the retrieved video file and advertisement clips separately to the client when the media retrieval request is an online viewing request, and wherein the transfer component integrates the retrieved video file and advertisement clips into a single integrated video/advertisement file for transfer to the client computer when the media retrieval request is a download request.

### Claim 10 (depends on 5)

The system of claim 5, wherein the designated video file includes a plurality of video clips, each of said plurality of clips including one or more video frames, and wherein a segmentation component, executed on the client, defines one or more segment boundaries in the video file for segmenting the designated video file into the plurality of video clips based on metadata included in each of the one or more video frames.

### Claim 11 (depends on 10)

The system of claim 10, wherein a content insertion engine, executed on the client, is responsive to the segmented video file to retrieve the one or more advertisement clips related to the video file from a memory of the client computer and to insert the retrieved advertisement between video clips in the designated video file during playback on the client.

### Claim 12 (depends on 5)

The system of claim 5 wherein the extraction component extracts metadata from each of the one or more video clips included in the designated video file using one or more of the following computerized metadata detection techniques: face detection, dynamic image peak detection, color value detection, dynamic image change detection face recognition, music beats detection, audio fingerprint detection, dynamic peaks detection, speech detection, word and phrase detection.

### Claim 13

A computerized method for integrating an advertisement clip into a video data stream being transferred from a server to a client for playback, said video data stream including one or more scenes, said computerized method comprising:
receiving, at the server, input from the user of the client designating the video data stream to be transferred;
extracting at the server metadata from the video data stream, said metadata defining a topic for each of one or more scenes in the video data stream, wherein said extracting includes determining a value of a predefined metadata property associated with each of the one or more scenes, said predefined metadata property defining a default topic of the scene associated therewith;
assigning a higher priority to one of the personal preference data for the user of the client and the metadata extracted from the video data stream;
retrieving a relevant advertisement clip for each of the one or more scenes based on the personal preference data for the user when the personal preference data is assigned the higher priority and retrieving a relevant advertisement clip for each of the one or more scenes based on the defined topic of the metadata of the one or more scenes when the metadata is assigned the higher priority; and
transferring the video data stream and relevant advertisement clips to the client in an integrated data stream.

### Claim 14 (depends on 13)

The method of claim 13, wherein the personal preference data defines a topic of interest for the user, and wherein retrieving the relevant video clip includes querying a database storing a plurality of video clips and a corresponding relevant topic for each of the plurality of video clips, and wherein the method further includes comparing the defined topic of interest to the corresponding relevant topic for each of the plurality of video clips to identify the relevant video clip.

### Claim 15 (depends on 13)

The method of claim 13 further comprising editing the video data stream to include the determined relevant advertisement clips, and wherein the transferring includes transferring the edited video file to the client.

### Claim 16 (depends on 13)

The method of claim 13 further including:
storing the received personal preference data in a preference database; and
retrieving the personal preference data from the preference database.