# US 8,279,343 B2 · Summary content generation device and computer program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,279,343 B2
- Document ID: US8279343B2
- Status: Granted
- Kind: B2
- Application number: 12/211,488
- Filed: 2008-09-16
- Published: 2009-03-19
- Granted: 2012-10-02
- Earliest claimed: 2007-09-18
- Assignees: Kddi Corporation
- Inventors: Toshiaki Uemukai; Kazunori Matsumoto; Fumiaki Sugaya
- Web page: https://patentgrep.com/patent/US8279343B2

## CPC

- H04N 7/0885: {for the transmission of subtitles}
- G11B 27/034: on discs (G11B27/036, G11B27/038 take precedence)
- G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)}
- H04N 21/426: Internal components of the client {; Characteristics thereof} (H04N5/44 takes precedence)
- H04N 21/4348: {Demultiplexing of additional data and video streams}
- H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/4532: {involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)}
- H04N 21/47: End-user applications
- H04N 21/4884: {for displaying subtitles}
- H04N 21/8549: Creating video summaries, e.g. movie trailer {(retrieval in video databases by using presentations in form of a video summary G06F16/739)}
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/6112: {involving terrestrial transmission, e.g. DVB-T}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)

## Abstract

When generating animation content as summary content for a digital broadcast program, the timing for switching of animation images for display is controlled appropriately. Subtitle character string extraction means for extracting a subtitle character string from subtitle data contained in digital broadcast signals, still image extraction means for extracting one still image corresponding to the subtitle character string, and summary content generation means for generating summary content to display the extracted subtitle character strings together with the corresponding extracted still images, are provided; the summary content generation means decides the timing for switching display of the plurality of subtitle character strings and still images comprised by the summary content, based on the subtitle character strings.

## Claims

### Claim 1

A summary content generation device, which uses digital broadcast signals having video data and subtitle data to generate summary content for a broadcast program, comprising:
a subtitle character string extraction unit, for extracting a subtitle character string from subtitle data contained in said digital broadcast signals;
a still image extraction unit, for extracting one still image corresponding to said subtitle character string from video data contained in said digital broadcast signals; and
a summary content generation unit, for generating summary content, which displays, on a screen, said extracted subtitle character string together with said corresponding extracted still image, wherein
said summary content generation unit decides the timing for switching display of the plurality of subtitle character strings and still images comprised by said summary content, based on the subtitle character strings.

### Claim 2 (depends on 1)

The summary content generation device according to claim 1, wherein said summary content generation unit decides the time during which a subtitle character string is displayed on the screen (the display time duration) according to the number of characters in the subtitle character string.

### Claim 3 (depends on 2)

The summary content generation device according to claim 2, wherein a threshold for the number of characters in a subtitle character string is provided, and the display time duration is lengthened in stages with the threshold as a boundary.

### Claim 4 (depends on 1)

The summary content generation device according to claim 1, wherein said summary content generation unit decides the time during which a subtitle character string is displayed on the screen according to the difficulty of words in the subtitle character string.

### Claim 5 (depends on 1)

The summary content generation device according to claim 1, wherein said summary content generation unit decides the time during which a subtitle character string is displayed on the screen according to a profile of the viewer.

### Claim 6 (depends on 1)

The summary content generation device according to claim 1, wherein said summary content generation unit creates schedule information for displaying subtitles and still images, specifying reproduction times for a plurality of subtitle character strings and still images comprised by said summary content.

### Claim 7 (depends on 1)

The summary content generation device according to claim 1, wherein said summary content generation unit judges the possibility of combining a plurality of contiguous subtitle character strings, combines a plurality of subtitle character strings judged to be combinable, and associates combined subtitle character strings with one still image.

### Claim 8 (depends on 7)

The summary content generation device according to claim 7, wherein judgment criteria for combinability of a plurality of contiguous subtitle character strings comprises a judgment criterion 1: that for two subtitle character strings A, B, continuous in the order of subtitle character string A and then subtitle character string B, subtitle character string A does not end with a period.

### Claim 9 (depends on 7)

The summary content generation device according to claim 7, wherein judgment criteria for combinability of a plurality of contiguous subtitle character strings comprises a judgment criterion 2: that for two subtitle character strings A, B, continuous in the order of subtitle character string A and then subtitle character string B, the total number of characters of the subtitle character strings A and B is smaller than a first stipulated number of characters.

### Claim 10 (depends on 7)

The summary content generation device according to claim 7, wherein judgment criteria for combinability of a plurality of contiguous subtitle character strings comprises a judgment criterion 3: that for two subtitle character strings A, B, continuous in the order of subtitle character string A and then subtitle character string B, the time after display of the subtitle character string A until the display of subtitle character string B is shorter than a stipulated time.

### Claim 11 (depends on 7)

The summary content generation device according to claim 7, wherein judgment criteria for combinability of three or more contiguous subtitle character strings comprises a judgment criterion 4: that for three subtitle character strings A, B, C, continuous in the order of subtitle character string A, then subtitle character string B, and then subtitle character string C, the total number of characters of the subtitle character strings A, Band C is greater than a second stipulated number of characters, and moreover that by dividing the combination “A+B+C” of the subtitle character strings A, B, C into two character strings, both the character strings after division have a number of characters smaller than the first stipulated number of characters.

### Claim 12 (depends on 11)

The summary content generation device according to claim 11, wherein the division position when dividing a combination of three subtitle character strings satisfying said judgment criterion 4 is decided so as to satisfy anyone among the conditions of ending on a punctuation mark, not ending in the middle of a word, or being close to the middle position of the entire character string.

### Claim 13 (depends on 1)

The summary content generation device according to claim 1, wherein said summary content generation unit judges whether a plurality of continuous still images are similar, and adopts a representative still image from among a plurality of contiguous similar still images.

### Claim 14 (depends on 1)

The summary content generation device according to claim 1, wherein said still image extraction unit extracts the first image, the image in the center, or one image selected at random, as a still image corresponding to a subtitle character string extracted by said subtitle character string extraction unit, from among a candidate image group corresponding to the subtitle character string.

### Claim 15 (depends on 1)

The summary content generation device according to claim 1, comprising human detection unit which analyzes video data comprised by said digital broadcast signals and detects humans appearing in images, and wherein said still image extraction unit extracts images in which humans appear as said still images.

### Claim 16 (depends on 1)

The summary content generation device according to claim 1, comprising human detection unit which analyzes video data comprised by said digital broadcast signals, detects humans appearing in images, and judges whether the humans are facing forward, and wherein said still image extraction unit extracts images in which humans are facing forward as said still images.

### Claim 17 (depends on 1)

The summary content generation device according to claim 1, comprising human detection unit which analyzes video data comprised by said digital broadcast signals, detects humans appearing in images, and judges whether the humans are facing forward, and facial expression detection unit which analyzes the image data and judges the facial expression of a human detected by said human detection unit, and wherein said still image extraction unit extracts images in which specific facial expressions of humans appear as said still images.

### Claim 18 (depends on 17)

The summary content generation device according to claim 17, wherein said specific facial expressions are expressions in which the eyes are open, or are laughing expressions, or are crying expressions.

### Claim 19 (depends on 1)

The summary content generation device according to claim 1, comprising title detection unit, which analyzes video data comprised by said digital broadcast signals and detects whether title data, which is data of character strings inserted on the screen on which video data is displayed, is inserted into images, and wherein said still image extraction unit extracts images into which title data has been inserted as said still images.

### Claim 20 (depends on 1)

The summary content generation device according to claim 1, comprising anchor shot detection unit, which analyzes video data comprised by said digital broadcast signals and judges whether anchor shots, which are video intervals in which a main newscaster appears in a news program in video data comprised by said digital broadcast signals, appear in images, and wherein said still image extraction unit extracts images in which anchor shots appear as said still images.

### Claim 21

A computer readable non-transitory medium storing a computer program, to generate summary content of a broadcast program using digital broadcast signals having video data and subtitle data, causing a computer to realize:
a subtitle character string extraction function of extracting a subtitle character string from subtitle data contained in said digital broadcast signals;
a still image extraction function of extracting one still image corresponding to said subtitle character string from the video data contained in said digital broadcast signals; and
a summary content generation function of generating summary content, for the display on a screen of said extracted subtitle character string together with said corresponding extracted still image,
wherein
said summary content generation function decides the timing for switching between a plurality of subtitle character strings and still images comprised by said summary content, based on the subtitle character strings.
