# US 2008/0304808 A1 · Automatic story creation using semantic classifiers for digital assets and associated metadata

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0304808 A1
- Document ID: US20080304808A1
- Status: Published application
- Kind: A1
- Application number: 11/935,737
- Filed: 2007-11-06
- Published: 2008-12-11
- Earliest claimed: 2007-06-05
- Assignees: Intellectual Ventures Fund 83 LLC
- Inventors: Catherine D. Newell; Mark D. Wood; Kathleen M. Costello; Robert B. Poetker
- Web page: https://patentgrep.com/patent/US20080304808A1

## CPC

- G11B 27/034: on discs (G11B27/036, G11B27/038 take precedence)
- G06F 16/435: Filtering based on additional data, e.g. user or group profiles
- G06F 16/4393: {Multimedia presentations, e.g. slide shows, multimedia albums}
- G06F 16/48: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G11B 27/322: {used signal is digitally coded}
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method and system for automatically creating an image product based on media assets stored in a database. A number of stored digital media files are analyzed to determine their semantic relationship to an event and are classified according to requirements and semantic rules for generating an image product. Rule sets are applied to assets for finding one or more assets that can be include in a story product. The assets, which best meet the requirements and rules of the image product are included.

## Claims

### Claim 1

A computer implemented method for automatically producing a story product comprising the steps of:
automatically applying a rule set to a plurality of digital image assets;
identifying ones of the digital image assets that satisfy the rule set;
automatically producing a story product containing identified ones of the digital image assets; and
notifying a user that the story product has been generated.

### Claim 2 (depends on 1)

The method of claim 1, further comprising the step of indexing the digital image assets before the step of applying a rule set.

### Claim 3 (depends on 1)

The method of claim 1, further comprising the steps of:
producing a story product representation of the story product before the step of producing the story product;
notifying the user that the representation has been produced before the step of producing the story product; and
receiving an approval of the representation from the user before the step of producing the story product.

### Claim 4 (depends on 2)

The method of claim 2, wherein the step of indexing the digital image assets includes a step selected from the steps consisting of:
scene classifying the digital image assets including storing scene metadata with a corresponding asset;
event classifying the digital image assets including storing event metadata with a corresponding asset; and
person classifying the digital image assets including storing person identification metadata with a corresponding asset.

### Claim 5 (depends on 4)

The method of claim 4, wherein the step of scene classifying the digital image assets further includes a step selected from the steps consisting of:
classifying as indoor;
classifying as outdoor;
classifying as natural;
classifying as manmade;
classifying as sunset;
classifying as beach;
classifying as fall foliage;
classifying as field;
classifying as mountain;
classifying as sky;
classifying as grass;
classifying as snow;
classifying as water;
classifying as sand; and
classifying as clouds.

### Claim 6 (depends on 4)

The method of claim 4, wherein the step of event classifying the digital image assets further includes a step selected from the steps consisting of:
classifying as vacation;
classifying as sports;
classifying as party; and
classifying as family moments.

### Claim 7 (depends on 4)

The method of claim 4, wherein the step of person classifying the digital image assets further includes the step of storing with a corresponding digital image asset a name of a person who appears in the corresponding digital image asset.

### Claim 8 (depends on 1)

The method of claim 1 wherein the step of applying the rule set is preceded by the step of monitoring a storage device for an upload of the digital image assets and beginning the applying step in response thereto.

### Claim 9 (depends on 1)

The method of claim 1 wherein the step of applying the rule set is preceded by the step of detecting an upcoming holiday and beginning the applying step in response thereto, and wherein the step of applying the rule set includes the step of selecting a rule set for application that corresponds to the detected upcoming holiday.

### Claim 10 (depends on 1)

The method of claim 1 wherein the step of applying the rule set is preceded by the step of detecting an upcoming calendar date and beginning the applying step in response thereto, and wherein the step of applying the rule set includes the step of selecting a rule set for application that corresponds to the detected calendar date.

### Claim 11 (depends on 1)

The method of claim 1 wherein the step of notifying the user includes the step of transmitting the story product to a user's device.

### Claim 12 (depends on 11)

The method of claim 11 wherein the user's device is selected from the group consisting of:
a set top box;
a cell phone;
a PDA;
a home media server;
a digital picture frame;
a printer; and
a PC.

### Claim 13 (depends on 1)

The method of claim 1 wherein the rule set includes a step selected from the group consisting of:
requiring a depiction of a particular person in a digital image asset;
requiring a depiction of a particular person and a relative of the particular person in a digital image asset;
requiring a depiction of a particular person and a friend of the particular person in a digital image asset;
requiring a particular date range of a digital image asset;
requiring a minimum value index of a digital image asset;
requiring a particular range of playback time of a digital image asset;
requiring a particular owner of a digital image asset;
selecting a best one of a group of digital image assets;
requiring an event classification of a digital image asset;
requiring a scene classification of a digital image asset;
requiring a depiction of a particular logo in a digital image asset;
requiring a depiction of a particular number of people in a digital image asset;
requiring an absence of persons in a digital image asset;
requiring that a digital image asset is a member of a particular class of assets; and
requiring that a recipient enjoys an event class in which a digital image asset belongs.

### Claim 14

A computer system comprising:
an asset uploader for receiving a plurality of digital image assets to be stored, the digital image assets including metadata;
an asset store for storing a plurality of digital assets;
a database for storing the metadata;
a semantic indexer for indexing the digital assets according to their content and for storing indexing information as metadata in the database associated with corresponding digital assets;
a rule base comprising a plurality of story rule sets each of which is capable of being satisfied by metadata stored in the database; and
a product generator for generating a story product that includes at least one digital asset whose metadata satisfies one of the story rule sets.

### Claim 15 (depends on 14)

The computer system of claim 14 wherein the semantic indexer comprises:
a scene classifier for categorizing a scene depicted in a digital image asset into at least one scene type and for storing the scene type in the database in association with a corresponding digital image asset.

### Claim 16 (depends on 14)

The computer system of claim 14 wherein the semantic indexer comprises:
an event classifier for categorizing an event depicted in a digital image asset into at least one event type and for storing the event type in the database in association with a corresponding digital image asset.

### Claim 17 (depends on 14)

The computer system of claim 14 wherein the semantic indexer comprises:
a people recognition component for identifying a person depicted in a digital image asset and for storing a name of the person in the database in association with a corresponding digital image asset.

### Claim 18 (depends on 14)

The computer system according to claim 14 further comprising:
an inferencing engine for evaluating the rule sets to identify one or more of the digital image assets having metadata associated therewith that satisfy a story rule set.

### Claim 19 (depends on 18)

The computer system according to claim 18 wherein one of the rule sets comprises a plurality of pre-selected story rules for identifying one or more of the digital image assets that satisfy a pre-selected theme.

### Claim 20 (depends on 18)

The computer system according to claim 18 further comprising an asset selector for identifying one or more of the digital image assets that satisfy a story rule set that also fail an image value rule.

### Claim 21 (depends on 19)

The computer system according to claim 19 wherein the pre-selected theme comprises a theme selected from the group consisting of:
celebratory;
religious;
interest group;
pets;
holiday;
anniversary;
solemn;
birthday;
sports;
vacation;
hobby;
leisure; and
family.

### Claim 22 (depends on 14)

The computer system according to claim 14 further comprising a story notifier service for notifying a user that the story product has been generated.

### Claim 23 (depends on 22)

The computer system according to claim 22 wherein the story notifier service includes a device for transmitting a viewable representation of the story product to a user device having a display for a user's approval.

### Claim 24 (depends on 14)

The computer system according to claim 14 further comprising:
a musical asset uploader for receiving a plurality of digital music assets to be stored, the digital music assets including metadata, and wherein the asset store includes the plurality of digital music assets, the database includes the digital music asset metadata, the rule base includes a story rule set capable of being satisfied by digital music asset metadata stored in the database, and wherein the product generator includes a music product generator for including in the story product at least one digital music asset whose metadata satisfies the story rule set capable of being satisfied by digital music asset metadata.

### Claim 25 (depends on 1)

A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform the method steps of claim 1.
