# US 2016/0253356 A1 · Composing media stories method and system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0253356 A1
- Document ID: US20160253356A1
- Status: Published application
- Kind: A1
- Application number: 15/055,168
- Filed: 2016-02-26
- Published: 2016-09-01
- Earliest claimed: 2015-02-27
- Assignees: RealNetworks, Inc.
- Applicants: RealNetworks, Inc.
- Inventors: Milko BOIC
- Web page: https://patentgrep.com/patent/US20160253356A1

## CPC

- G06F 16/4393: {Multimedia presentations, e.g. slide shows, multimedia albums}
- G06F 16/285: {Clustering or classification}
- G06F 16/583: using metadata automatically derived from the content
- G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Disclosed is a method and apparatus to programmatically determine a home location for a user, to cluster photographs and videos into events and trips relative to the home location, to determine names for the events and trips, to composite stories created from the events and trips, to update the composited stories over time, and to output the composited stories for display.

## Claims

### Claim 1

A method of organizing images for display, which method is performed by a computer comprising a memory and a processor, wherein the memory comprises instructions, which, when executed by the processor, perform the following:
for a set of items associated with at least one user account, wherein items in the set of items comprise at least one photograph and at least one video,
obtaining by the computer a first item and a second item for inclusion in the set of items, wherein, in the memory of the computer, the items may be associated with item metadata comprising an item location and an item date-time, wherein the item location records a location where the item was recorded and the item date-time records a date-time when the item was recorded;
determining a set of location cells, each location cell in the set containing at least one item from the set of items;
recording the set of location cells in a location cell map;
selecting a home location within the set of location cells;
clustering at least a subset of the items in an event cluster relative to the home location;
determining an event name for the event cluster;
clustering entities in a trip cluster relative to the home location, wherein entities comprise an item and an event;
determining a trip name for the trip cluster;
selecting items from the event cluster and the trip cluster for display;
displaying the selected items in conjunction with the event name or the trip name.

### Claim 2 (depends on 1)

The method of claim 1, wherein each location cell comprises all items with item locations within a maximum location cell span and wherein determining the set of location cells comprises:
determining whether a first item location associated with a first item will cause an existing location cell to exceed the maximum location cell span and when the first item location will not, setting the existing location cell to be the first location cell of the first item and when the first item location will cause all existing location cells to exceed the maximum location cell span, creating a new location cell for the first item.

### Claim 3 (depends on 2)

The method of claim 2, further comprising excluding a subset of the existing location cells that are too remote relative to the first item location.

### Claim 4 (depends on 1)

The method of claim 1, wherein selecting a home location within the set of location cells comprises determining a location cell associated with a most number of visits and selecting the center of such location cell as the home location.

### Claim 5 (depends on 4)

The method of claim 4, wherein a visit is defined as creation of the item in the location cell after a chronologically preceding item creation outside of the location cell.

### Claim 6 (depends on 4)

The method of claim 4, wherein when a group comprising more than one location cell have the same most number of visits, selecting within this group a location cell with the most number of items and selecting the center of such location cell as the home location.

### Claim 7 (depends on 1)

The method of claim 1, further comprising determining that a first subset of entities are at a same summarized location and that a second subset of entities are possibly at the same summarized location, wherein the event may be associated with an event location.

### Claim 8 (depends on 7)

The method of claim 7, further comprising determining that the first subset of entities are at the same summarized location when an item location or an event location or a place name is available for each of the entities and i) when the item location or event location for each of the entities is within a distance threshold or ii) when the place names are the same.

### Claim 9 (depends on 7)

The method of claim 7, further comprising determining that the second subset of entities are possibly at the same summarized location when at least one of the entities does not have a place name and i) if at least one of the entities does not have an item location or an event location or ii) if each entity has an item location or an event location and the distance between the item or event locations is less than a distance threshold.

### Claim 10 (depends on 7)

The method of claim 7, further comprising determining either i) that the first subset of entities are more than a trip-distance threshold away from the home location and forming a cluster of items recorded within a 48 hour period or ii) that the first subset of entities are less than the trip-distance threshold away from the home location and forming a cluster of items recorded within a 24 hour period.

### Claim 11 (depends on 7)

The method of claim 7, further comprising forming a cluster of items in the second subset of entities recorded within a 24 hour period.

### Claim 12 (depends on 7)

The method of claim 7, further comprising not displaying an event cluster if the event cluster comprises fewer than a required minimum event cluster number.

### Claim 13 (depends on 7)

The method of claim 7, further comprising not displaying an event cluster if the event cluster is within a threshold distance of the home location and if the event cluster comprises fewer than a required minimum home location event cluster number.

### Claim 14 (depends on 1)

The method of claim 1, wherein clustering entities in the trip cluster relative to the home location comprises determining for a new item i) that an immediately previous item was not clustered as part of a trip cluster and that an item location for the new item is greater than a trip distance threshold, starting the trip cluster, and adding such new item to the trip cluster.

### Claim 15 (depends on 14)

The method of claim 14, further comprising receiving a subsequent item, relative to the new item, and ending the trip cluster if at least one of the following determinations is affirmative: i) that an item location of the subsequent item is within a threshold distance of the home location, ii) that there is no item location of the subsequent item and a trip separation time threshold has been exceeded, or iii) that the item location of the subsequent item exceeds the trip distance threshold relative to a preceding item in the trip.

### Claim 16 (depends on 15)

The method of claim 15, further comprising starting a new trip cluster if the item location of the subsequent item exceeds the trip distance threshold relative to a trip location of the trip.

### Claim 17 (depends on 1)

The method of claim 1, wherein determining an event name for the event cluster comprises identifying at least one item in the event cluster with a place name and setting the event name to comprise the place name.

### Claim 18 (depends on 17)

The method of claim 17, further comprising identifying more than one place name for more than one item in the event cluster and setting the event name to comprise a range comprising the more than one place name, ordered from first visited to most recently visited that is different from the first visited.

### Claim 19 (depends on 1)

The method of claim 1, wherein determining an event name for the event cluster comprises identifying that no items in the event cluster have a place name and setting the event name to comprise a date or a date range determined from the item date-time for items in the event cluster.

### Claim 20 (depends on 1)

The method of claim 1, wherein determining a trip name for the trip cluster comprises determining that entities in the trip comprise a place name and setting the trip name to comprise the place name.

### Claim 21 (depends on 20)

The method of claim 20, further comprising identifying more than one place name for more than one item in the trip cluster and setting the trip name to comprise a range comprising the more than one place name, ordered from first visited to most recently visited that is different from the first visited.

### Claim 22 (depends on 21)

The method of claim 21, and when more than one place name is available, selecting the place name from the more than one place names according to which place name corresponds to the smallest geographic unit.

### Claim 23 (depends on 1)

The method of claim 1, wherein determining a trip name for the trip cluster comprises identifying that no items in the trip cluster have a place name and setting the trip name to comprise a date or a date range determined from the item date-time for items in the trip cluster.

### Claim 24 (depends on 1)

The method of claim 1, wherein selecting items in the set of items for display comprises, for each item which is a photograph, ranking such photographs according to how blurry each photograph is, whether each photograph is a duplicate, and whether each photograph comprises a face image, and for each item which is a video, ranking each video according to focused scenes and user encoder rate control, and selecting roughly equal proportions of the top ranked photographs and videos for display.

### Claim 25

A non-transitory computer-readable storage medium for displaying photographs and videos, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
for a set of items associated with at least one user account, wherein items in the set of items comprise at least one photograph and at least one video,
obtain by the computer a first item and a second item for inclusion in the set of items, wherein, in the memory of the computer, the items may be associated with item metadata comprising an item location and an item date-time, wherein the item location records a location where the item was recorded and the item date-time records a date-time when the item was recorded or a date-time when the item was first obtained by the computer for inclusion in the set of items;
determine a set of location cells, each location cell in the set containing at least one item in the set of items;
record the set of location cells in a location cell map;
select a home location within the set of location cells;
cluster the items in an event cluster relative to the home location;
cluster entities in a trip cluster relative to the home location, wherein entities comprise an item and an event;
determine an event name for the event cluster;
determine a trip name for the trip cluster;
select items from an event cluster and a trip cluster for display;
display the selected items in conjunction with the event name or the trip name corresponding to the event cluster or to the trip cluster.
