Authorized syndicated descriptions of linked web content displayed with links in user-generated content

A computer program product comprising a non-transitory computer-readable storage medium containing computer program code for obtaining a corpus of authorized syndicated descriptions of web content, the code for:

• obtaining items from a plurality of feeds from one or more sources, each item comprising a link to a content page and a description of content included in the content page;
• for each of one or more items of one or more of the feeds,
• accessing the content page associated with a link included in an item;
• analyzing metadata associated with the accessed content page to identify an indication of a source from which the content page was obtained;
• determining, using a processor, whether the analyzed metadata associated with the accessed content page includes a pointer to the source from which a feed including the item was obtained;
• responsive to determining that the accessed content page includes a pointer to the source from which the feed including the item was obtained, recording a description of content from the item for use as an authorized syndicated description of the item;
• and
• responsive to determining that the analyzed metadata associated with the accessed content page does not include the pointer to the source feed from which a feed including the item was obtained, checking a root of the domain for the content page for the pointer to the source feed from which the feed including the item was obtained;
• and
• generating a page including a selected item from the obtained items, the page including a pointer to the feed from which the selected item was obtained responsive to the description of content from the selected item being recorded as an authorized syndicated description of the selected item.

A system for obtaining a corpus of authorized syndicated descriptions of web content, the system comprising:

• a processor;
• a memory, communicatively coupled to the processor, storing instructions that, when executed by the processor, perform a method comprising:
• obtaining items from a plurality of feeds from one or more sources, each item comprising a link to a content page and a description of content included in the content page;
• for each of one or more items of one or more of the feeds,
• accessing the content page associated with a link included in an item;
• analyzing metadata associated with the accessed content page to identify an indication of a source from which the content page was obtained;
• determining whether the analyzed metadata associated with the accessed content page includes a pointer to the source from which a feed including the item was obtained;
• responsive to determining that the accessed content page includes a pointer to the source from which the feed including the item was obtained, recording a description of content from the item for use as an authorized syndicated description of the item;
• and
• responsive to determining that the analyzed metadata associated with the accessed content page does not include the pointer to the source feed from which a feed including the item was obtained, checking a root of the domain for the content page for the pointer to the source feed from which the feed including the item was obtained;
• and
• generating a page including a selected item from the obtained items, the page including a pointer to the feed from which the selected item was obtained responsive to the description of content from the selected item being recorded as an authorized syndicated description of the selected item.