# US 2017/0374430 A1 · Caption replacement service system and method for interactive service in video on demand

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0374430 A1
- Document ID: US20170374430A1
- Status: Published application
- Kind: A1
- Application number: 15/699,085
- Filed: 2017-09-08
- Published: 2017-12-28
- Earliest claimed: 2013-12-27
- Assignees: Inha University Research and Business Foundation
- Inventors: Geun Sik Jo
- Web page: https://patentgrep.com/patent/US20170374430A1

## CPC

- H04N 21/4884: {for displaying subtitles}
- H04N 21/4725: using interactive regions of the image, e.g. hot spots {(details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94)}
- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- H04N 21/8545: for generating interactive applications
- H04N 21/8586: {by using a URL (processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02)}
- H04N 5/2723: {Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement (data processing systems or methods specially adapted for marketing G06Q30/02)}
- H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement}
- H04N 21/8126: {involving additional data, e.g. news, sports, stocks, weather forecasts}
- H04N 21/858: Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Provided is a caption replacement service system and method for servicing an interactive advertisement on a video on demand (VOD). A system for making an annotation may include an annotation creator configured to create annotation information on a moving object included in a video and an annotation player configured to play a preview on the annotation information based on a frame unit or a shot unit of the video. The video and the annotation information may be used to provide an interactive advertisement by replacing a caption system of the video with a meaningful annotation.

## Claims

### Claim 1

A caption replacement service method performed by a computer system, the method comprising:
playing a video present on a web;
searching for information on a virtual object area corresponding to the video from metadata associated with the video, the searched for virtual object area being configured to cover, on a frame of the video when the video is played, an image of text that comprises textual information that is original to the video;
dynamically selecting an interactive advertisement based on a profile of a user;
automatically augmenting the interactive advertisement to the virtual object area over the image of text while the video is playing using the searched for information on the virtual object area in response to a determination that content of the textual information contains non-current information based on a current context making a meaning of the content outdated; and
providing an interactive service about an event occurring in the virtual object area to the user.

### Claim 2 (depends on 1)

The method of claim 1, wherein the playing of the video comprises playing a video selected by the viewer on the web.

### Claim 3 (depends on 1)

The method of claim 1, wherein the searching for information on the virtual object area comprises:
searching for information on the virtual object area corresponding to a video selected by the viewer at: a first time when the selected video is played or a predetermined time interval; and
determining whether a time at which the selected video is to be played and a start time for displaying the virtual object area match.

### Claim 4 (depends on 1)

The method of claim 1, wherein the augmenting of the interactive advertisement information comprises displaying the interactive advertisement information within the video based on information on a temporal location and a spatial location of the virtual object area in the information on the virtual object area.

### Claim 5 (depends on 1)

The method of claim 1, wherein the event occurs in response to at least one of a click and a mouse over on the interactive advertisement information.

### Claim 6 (depends on 1)

The method of claim 1, wherein the providing of the interactive service comprises providing additional information associated with the interactive advertisement information as the interactive service, in response to the occurrence of the event.

### Claim 7 (depends on 1)

The method of claim 1, wherein the interactive advertisement information comprises an advertisement image extracted from a predetermined advertisement pool.

### Claim 8 (depends on 7)

The method of claim 7, wherein the image extracted from the advertisement pool is extracted based on a propensity and a matter of interest of the viewer.

### Claim 9 (depends on 1)

The method of claim 1, wherein the information on the virtual object area comprises information on at least one of a start time and an end time for displaying the virtual object area, a size of the virtual object area, a display location of the virtual object area, content of an interactive advertisement to be added to the virtual object area, a URL address of the interactive advertisement, and an event address generated in response to a click on the virtual object area.
