# US 2021/0343316 A1 · Scene and activity identification in video summary generation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0343316 A1
- Document ID: US20210343316A1
- Status: Published application
- Kind: A1
- Application number: 17/378,324
- Filed: 2021-07-16
- Published: 2021-11-04
- Earliest claimed: 2014-07-23
- Assignees: GoPro, Inc.
- Applicants: Gopro, Inc.
- Inventors: David A. Newman; Adam Silver
- Web page: https://patentgrep.com/patent/US20210343316A1

## CPC

- G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/47: {Detecting features for summarising video content}
- G10L 15/063: {Training}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 25/57: for processing of video signals
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B 27/13: the information being derived from movement of the record carrier, e.g. using tachometer
- G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)}
- G11B 27/3081: {used signal is a video-frame or a video-field (P.I.P)}
- G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)}
- H04N 5/77: between a recording apparatus and a television camera
- H04N 5/772: {the recording apparatus and the television camera being placed in the same enclosure}
- H04N 5/91: Television signal processing therefor
- H04N 9/8205: {involving the multiplexing of an additional signal and the colour video signal}
- H04N 19/513: Processing of motion vectors
- G06V 20/44: {Event detection}
- G10L 25/54: for retrieval
- G10L 2015/0631: {Creating reference templates; Clustering}
- G10L 2015/223: {Execution procedure of a spoken command}
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Video and corresponding metadata is accessed. Events of interest within the video are identified based on the corresponding metadata, and best scenes are identified based on the identified events of interest. A video summary can be generated including one or more of the identified best scenes. The video summary can be generated using a video summary template with slots corresponding to video clips selected from among sets of candidate video clips. Best scenes can also be identified by receiving an indication of an event of interest within video from a user during the capture of the video. Metadata patterns representing activities identified within video clips can be identified within other videos, which can subsequently be associated with the identified activities.

## Claims

### Claim 1

A system configured to generate a video summary, the system comprising:
one or more physical processors configured by machine readable instructions to:
obtain metadata for one or more videos;
analyze the metadata for the one or more videos to identify events of interest in the one or more videos;
select portions of the one or more videos for inclusion in the video summary based on the events of interest in the one or more videos; and
generate the video summary from the selected portions of the one or more videos.

### Claim 2 (depends on 1)

The system of claim 1, wherein the one or more physical processors are further configured to rank the events of interest within the one or more videos.

### Claim 3 (depends on 2)

The system of claim 2, wherein the one or more physical processors are further configured such that selection of the portions of the one or more videos for inclusion in the video summary is further based on the rankings of the individual events of interest.

### Claim 4 (depends on 1)

The system of claim 1, wherein:
the metadata for the one or more videos includes location information identifying capture locations for the one or more videos; and
a given event of interest is identified based on a given capture location for a given video being within a threshold distance of a pre-determined location.

### Claim 5 (depends on 4)

The system of claim 4, wherein:
the metadata for the one or more videos includes time stamps representing capture times for the one or more videos; and
the given event of interest is identified further based on a given capture time of the given video being within a threshold time of a pre-determined time.

### Claim 6 (depends on 1)

The system of claim 1, wherein:
the metadata for the one or more videos includes location information identifying capture locations for the one or more videos; and
a given event of interest is identified based on a given capture location for a given video being within a threshold distance of a user.

### Claim 7 (depends on 1)

The system of claim 1, wherein:
the metadata for the one or more videos includes weather information associated with capture of the one or more videos; and
a given event of interest is identified based on the weather information associated with the capture of the one or more videos.

### Claim 8 (depends on 1)

The system of claim 1, wherein the one or more physical processors are further configured to identify a given video as not including any of the events of interest based on the metadata for the given video.

### Claim 9 (depends on 8)

The system of claim 8 wherein responsive to the given video being identified as not including any of the events of interest, the given video is disqualified from consideration for inclusion in the video summary or is hidden from user view.

### Claim 10 (depends on 1)

The system of claim 1, wherein the one or more physical processors are further configured to provide individual selected portions of the one or more videos.

### Claim 11

A method for generating a video summary, the method performed by a computing system including one or more processors, the method comprising:
obtaining, by the computing system, metadata for one or more videos;
analyzing, by the computing system, the metadata for the one or more videos to identify events of interest in the one or more videos;
selecting, by the computing system, portions of the one or more videos for inclusion in the video summary based on the events of interest in the one or more videos; and
generating, by the computing system, the video summary from the selected portions of the one or more videos.

### Claim 12 (depends on 11)

The method of claim 11, further comprising ranking, by the computing system, the events of interest within the one or more videos.

### Claim 13 (depends on 12)

The method of claim 12, wherein the selecting of the portions of the one or more videos for inclusion in the video summary is further based on the rankings of the individual events of interest.

### Claim 14 (depends on 11)

The method of claim 11, wherein:
the metadata for the one or more videos includes location information identifying capture locations for the one or more videos; and
a given event of interest is identified based on a given capture location for a given video being within a threshold distance of a pre-determined location.

### Claim 15 (depends on 14)

The method of claim 14, wherein:
the metadata for the one or more videos includes time stamps representing capture times for the one or more videos; and
the given event of interest is identified further based on a given capture time of the given video being within a threshold time of a pre-determined time.

### Claim 16 (depends on 11)

The method of claim 11, wherein:
the metadata for the one or more videos includes location information identifying capture locations for the one or more videos; and
a given event of interest is identified based on a given capture location for a given video being within a threshold distance of a user.

### Claim 17 (depends on 11)

The method of claim 11, wherein:
the metadata for the one or more videos includes weather information associated with capture of the one or more videos; and
a given event of interest is identified based on the weather information associated with the capture of the one or more videos.

### Claim 18 (depends on 11)

The method of claim 11, further comprising identifying, by the computing system, a given video as not including any of the events of interest based on the metadata for the given video.

### Claim 19 (depends on 18)

The method of claim 18 wherein responsive to the given video being identified as not including any of the events of interest, the given video is disqualified from consideration for inclusion in the video summary or is hidden from user view.

### Claim 20 (depends on 11)

The method of claim 11, further comprising providing, by the computing system, individual selected portions of the one or more videos.
