# US 2018/0027282 A1 · Method and apparatus for referencing, filtering, and combining content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0027282 A1
- Document ID: US20180027282A1
- Status: Published application
- Kind: A1
- Application number: 15/479,774
- Filed: 2017-04-05
- Published: 2018-01-25
- Earliest claimed: 2016-07-20
- Assignees: Art Research and Technology, L.L.C.
- Inventors: David Hirschfeld; Mark Phelps; Theodore V. Haig; Barry Fernando
- Web page: https://patentgrep.com/patent/US20180027282A1

## CPC

- H04L 51/52: for supporting social networking services
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 16/90335: {Query processing}
- G06Q 10/40: Business processes related to social networking or social networking services
- G11B 27/327: {Table of contents}
- G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)}
- H04L 12/1818: {Conference organisation arrangements, e.g. handling schedules, setting up parameters needed by nodes to attend a conference, booking network resources, notifying involved parties}
- H04L 12/4625: {Single bridge functionality, e.g. connection of two networks over a single bridge}
- H04L 51/216: Handling conversation history, e.g. grouping of messages in sessions or threads
- H04L 65/762: {at the source (reformatting of additional data in video distribution servers H04N21/2355)}
- H04N 21/2187: Live feed
- H04N 21/4318: {by altering the content in the rendering process, e.g. blanking, blurring or masking an image region (image enhancement or restoration in general G06T5/00)}
- H04N 21/4334: {Recording operations (recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27)}
- H04N 21/47205: {for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally}
- H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments}
- H04W 4/02: Services making use of location information
- H04W 4/21: for social networking applications
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L 67/306: User profiles
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

A method and system for annotating playable media files in a social network having a plurality of members, for displaying information associated with the playable media tiles, and for marking a portion of interest in the playable media files is disclosed herein.

## Claims

### Claim 1

A method for displaying information associated with a Playable Media File, comprising:
obtaining stored data describing the information, the stored data comprising a storage location of the playable media file and a plurality of virtual clips each associated with the playable media file and including a first data element identifying a first time within the playable media file at which the corresponding virtual clip begins, and a second data element identifying a first user profile associated with creating the corresponding virtual clip;
accessing the playable media file at the storage location;
causing a graphical user interface (GUI) to be displayed on a computing device of a user, wherein said GUI enables the user to generate user inputs by interacting with the GUI, and the GUI comprises:
a display window for displaying content encoded by the playable media file;
a first interactable graphical object, wherein the first interactable graphical object overlays a first portion of the display window and displays a timeline representing a duration of the playable media file and a plurality of clip indicators each associated with a corresponding virtual clip of the plurality of virtual clips, each clip indicator appearing on the timeline at a display position corresponding to the first time identified by the first data element of the corresponding virtual clip; and
a second interactable graphical object, wherein the second interactable graphical object overlays a second portion of the display window, is configured to display up to a first number of graphic elements each associated with a corresponding virtual clip of the plurality of virtual clips, and is initially not displayed in the GUI;
receiving a first user input indicating a first interaction of the user with a first display position on the timeline;
determining a selected time within the playable media file that corresponds to the first display position;
identifying, as a plurality of displayable virtual clips:
a first virtual clip of the plurality of virtual clips, the corresponding first time of the first virtual clip being the closest, of the plurality of virtual clips, to the selected time; and
one or more of the virtual clips wherein the corresponding first time precedes and is approximate to the first time of the first virtual clip, and one or more of the virtual clips wherein the corresponding first time is subsequent and approximate to the first time of the first virtual clip, such that at most the first number of the plurality of virtual clips are selected as the one or more displayable virtual clips; and
updating the user interface on the computing device to display a list of the one or more displayable virtual clips in the second interactable graphical object.

### Claim 2 (depends on 1)

The method of claim 1, wherein each of the plurality of virtual clips further comprises a first access type indicating whether the virtual clip is a public clip available to any requesting user, a shared clip made accessible to the user by one of a plurality of network users of a web service, or a private clip accessible only by the user, and wherein causing the user interface to be displayed on the computing device comprises determining, for each of the plurality of clip indicators, a display color based on the first access type of the corresponding virtual clip.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
receiving a second user input;
creating, from the second user input, an annotation associated with a first displayable virtual clip of the one or more displayable virtual clips;
providing said annotation to a network server;
providing a data profile to said network server, wherein said data profile comprises a location in said Playable Media File where said annotation should be made visible;
determining by said network server if said annotation is a first annotation submitted for said Playable media File;
if said annotation is not a first annotation submitted for said Playable Media File, encoding said data profile in a previously-created table of contents for said Playable Media File;
if said annotation is a first annotation submitted for said Playable Media File:
creating a table of contents by said network server for said Playable Media File; and
encoding by said network server said data profile in said table of contents.

### Claim 4 (depends on 3)

The method of claim 3, wherein the data profile further comprises: a second access type indicating whether the corresponding annotation is a public annotation to all network users of a social network, a shared annotation made accessible to the user by one of the network users, or a private annotation accessible only by the user; and
an identifier of a creating user of the corresponding annotation.

### Claim 5 (depends on 1)

The method of claim 1, wherein the graphical user interface further comprises a third interactable graphical object overlaying a third portion of the display window, having a visible state controlling whether the third interactable graphical object is visible or hidden, and when visible displaying a discussion thread comprising annotations associated with a selected virtual clip of the one or more displayable virtual clips.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
receiving a second user input describing an interaction with the selected virtual clip within the second interactable graphical object; and
responsive to the second user input, displaying the discussion thread within the third interactable graphical object.

### Claim 7 (depends on 1)

The method of claim 1, wherein obtaining the stored data comprises:
causing a recording device to begin capturing, as the playable media file, a recording of live content;
while the recording device is capturing the live content, receiving a second user input at a first time, the recording device continuing to capture the live content subsequent to the second user input;
while the recording device is capturing the live content, receiving on the user interface a third user input at a second time; and
creating a first virtual clip of the plurality of virtual clips, the first virtual clip having the first time as the corresponding first position and the second time as the corresponding second position.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
receiving a second user input associated with a first virtual clip of the plurality of virtual clips;
determining that the second user input comprises a character string in a predetermined format comprising:
a tag identifier indicating that the user input includes one or more taxonomy tags;
a first tag following the tag identifier; and
one or more subtags sequentially following the first tag and delimited by a delimiter character;
identifying one or more taxonomy tags from the second user input; and
associating, in the stored data, the first virtual clip with the one or more taxonomy tags to produce a searchable virtual clip that is delivered to a requesting device in response to a query from the requesting device for any of the plurality of virtual clips that are associated with the user profile and the one or more taxonomy tags.

### Claim 9 (depends on 8)

The method of claim 8, wherein associating the first virtual clip with the one or more taxonomy tags comprises creating, based on an order in the second user input of the one or more taxonomy tags, a directed relationship between a first taxonomy tag and a second taxonomy tag sequentially following the first taxonomy tag in the character string, the directed relationship enabling a user to retrieve the first virtual clip from the stored data using an ordered combination of the first and second taxonomy tags as the query.

### Claim 10

A method for marking a portion of interest in a Playable Media File, comprising:
causing a recording device to begin capturing a recording of a live event as the Playable Media File;
while the recording device is capturing the recording, receiving a first user input, the recording device continuing to capture the live content subsequent to the first user input;
determining from the first user input, a first temporal point of interest during said recording of the Playable Media File;
generating a first temporal place marker that indexes said first temporal point of interest; and
electronically storing the first temporal place marker.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
receiving a second user input while the recording device is capturing the recording;
determining from the second user input, a second temporal point of interest during said recording of the Playable Media File; and
generating a second temporal place marker that indexes said second temporal point of interest;
wherein electronically storing the first temporal place marker comprises electronically storing a first virtual clip associated with the playable media file, the first virtual clip comprising the first temporal place marker and the second temporal place marker.

### Claim 12 (depends on 10)

The method of claim 10, wherein the generating step further comprises using a control device to create the first temporal place marker.

### Claim 13 (depends on 12)

The method of claim 12, wherein the control device communicates to the recording device via a connected data link.

### Claim 14

The method of 12, wherein the control device communicates to the recording device remotely.

### Claim 15

The method of 10, wherein said recording device is selected from the group consisting of a mainframe computer, a mobile telephone, a smart telephone, a personal digital assistant, a personal computer, a laptop, a set-top box, an MP3 player, an email enabled device, a tablet computer, a web enabled device, and other special purpose computer each having one or more processors.

### Claim 16

A method of annotating a playable media file, the method comprising:
obtaining a virtual clip comprising a first location within the playable media file and a second location within the playable media file, the first and second locations together defining a clip of the playable media file occurring between the first and second locations;
causing, using the virtual clip, the clip to be displayed on a computing device of a user;
receiving a first user input associated with the virtual clip;
determining that the first user input conforms to a predetermined format defining taxonomy tags;
identifying one or more taxonomy tags from the user input; and
associating, in an account of the user, the virtual clip with each of the one or more taxonomy tags identified from the user input.

### Claim 17 (depends on 16)

The method of claim 16, wherein identifying the one or more taxonomy tags comprises determining the one or more taxonomy tags using the predetermined format comprising:
a tag identifier indicating that the user input includes taxonomy tags; and
one or more tags following the tag identifier and separated from each other by a delimiter, the one or more tags including a first tag and zero or more subtags arranged in sequence.

### Claim 18 (depends on 17)

The method of claim 17, wherein associating the virtual clip with the one or more taxonomy tags comprises arranging the one or more taxonomy tags according to the sequence in the user input of the one or more taxonomy tags.

### Claim 19 (depends on 18)

The method of claim 18, wherein the virtual clip is associated with each of the one or more taxonomy tags in a global data store tracking use of the taxonomy tags by all network users of a social network, and wherein the first user input includes a first taxonomy tag following the tag identifier, a second taxonomy tag following the first taxonomy tag, and a third taxonomy tag following the second taxonomy tag, the method further comprising:
causing the first user input to be displayed in association with an interactable graphical object on the computing device;
receiving a second user input indicating a selection of the second taxonomy tag in the first user input;
determining, as a tag filter, a portion of the sequence including the first taxonomy tag and the second taxonomy tag;
querying the global data store to obtain a plurality of filtered virtual clips each associated with the first taxonomy tag and the second taxonomy tag according to the tag filter; and
causing the plurality of filtered virtual clips to be displayed on the computing device.

### Claim 20 (depends on 19)

The method of claim 19, wherein associating the virtual clip with the one or more taxonomy tags further comprises associating the virtual clip in the global data store with a user identifier of a creating user of the virtual clip, and wherein the tag filter further includes the user identifier, such that the plurality of filtered virtual clips are each associated with the user identifier.