Organization, retrieval, annotation and presentation of media data files using signals captured from a viewing environment

An article of manufacture, comprising:

• a computer storage medium;
• computer program instructions stored on the computer storage medium, that when executed by a host computer configure the host computer to provide a computer system comprising:
• a. computer-accessible storage configured to store:
• entity data comprising data identifying a plurality of entities and data identifying relationships among the entities;
• media data files and media file metadata, wherein each media data file has associated media file metadata, the metadata comprising:
• an indication of one or more entities associated with the media data file, and text data;
• and
• story data comprising a plurality of stories, each story comprising at least text data, an indication of a plurality of the entities associated with the story and an indication of a plurality of media data files associated with the story;
• and
• b. a processing system configured to:
• receive the signal from a sensor in an environment, the sensor generating a signal representing at least verbal communication by an individual in the environment;
• process the signal to derive context, the context including at least data extracted from the received signal representing the verbal communication by the individual in the environment and data indicating an entity;
• present media data from a selection of media data files on a presentation device in the environment while receiving the signal from the sensor;
• annotate a media data file among the selection of media data files based at least on the context derived from the signal from the sensor.

A computer-implemented process using a computer system comprising computer-accessible storage configured to store:

• entity data comprising data identifying a plurality of entities and data identifying collections of entities from the plurality of entities;
• media data files and media file metadata, wherein each media data file has associated media file metadata, the metadata comprising an indication of one or more entities of the plurality of entities associated with the media data file, and text data;
• and story data comprising a plurality of stories, each story comprising at least text data, an indication of a plurality of the entities associated with the story and an indication of a plurality of media data files associated with the story;
• and a sensor in an environment, the sensor generating a signal representing at least verbal communication by an individual in the environment;
• and a presentation device in the environment configured to receive media data and process the media data to present the media data in the environment, the computer-implemented process, performed by the computer system, comprising:
• receiving the signal from the sensor;
• processing the signal to derive context, the context including at least data extracted from the received signal representing the verbal communication by the individual in the environment and data indicating an entity;
• presenting media data from a selection of media data files on the presentation device in the environment while receiving the signal from the sensor;
• annotating a media data file among the selection of media data files based at least on the context derived from the signal from the sensor.