Enhancement of massive data ingestion by similarity linkage of documents

A computer system for ingesting a plurality of content according to a statistical similarity of at least one portion of the ingested plurality of content into an online information handling system running on a first computer and capable of answering questions, wherein the ingested plurality of content is based on a received topic entered on a second computer and ingesting the plurality of content comprises ingesting a plurality of documents associated with the received topic, wherein the ingested plurality of documents are acquired from searching a plurality of online sources including a plurality of repositories and internet, the computer system comprising:

• one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:
• determining at least one similarity between each document within the ingested plurality of documents based on a similarity criteria;
• applying a statistical model to characterize the determined at least one similarity between each document within the ingested plurality of documents;
• creating at least one pair-wise link between a word, an image, or a plurality of other media for each document within the ingested plurality of documents based on the applied statistical model;
• mapping the created at least one pair-wise link for a first document within the ingested plurality of documents to at least one other document within the ingested plurality of documents based on a context of the word, the image, or the plurality of other media;
• generating a plurality of rules for ingesting a plurality of additional content based on the mapping of the created at least one pair-wise link;
• storing the generated plurality of rules in an online repository;
• retrieving and utilizing the stored plurality of rules when a plurality of future documents are loaded into the online information handling system;
• creating a plurality of hard and soft links between a word, an image, or a plurality of other media for each document within the ingested plurality of documents based on the applied statistical model, wherein each hard link within the plurality of hard and soft links meets or exceeds a predetermined threshold and each soft link within the plurality of hard and soft links falls below a predetermined threshold;
• and
• creating a single annotation between each document within the ingested plurality of documents based on each hard link and each soft link within the plurality of hard and soft links.

A computer program product for ingesting a plurality of content according to a statistical similarity of at least one portion of the ingested plurality of content into an online information handling system running on a first computer and capable of answering questions, wherein the ingested plurality of content is based on a received topic entered on a second computer and ingesting the plurality of content comprises ingesting a plurality of documents associated with the received topic, wherein the ingested plurality of documents are acquired from searching a plurality of online sources including a plurality of repositories and internet, the computer program product comprising:

• one or more computer-readable non-transitory storage devices and program instructions stored on at least one of the one or more computer readable non-transitory storage devices, the program instructions executable by a processor, the program instructions comprising:
• program instructions to determine at least one similarity between each document within the ingested plurality of documents based on a similarity criteria;
• program instructions to apply a statistical model to characterize the determined at least one similarity between each document within the ingested plurality of documents;
• program instructions to create at least one pair-wise link between a word, an image, or a plurality of other media for each document within the ingested plurality of documents based on the applied statistical model;
• program instructions to map the created at least one pair-wise link for a first document within the ingested plurality of documents to at least one other document within the ingested plurality of documents based on a context of the word, the image, or the plurality of other media;
• program instructions to generate a plurality of rules for ingesting a plurality of additional content based on the mapping of the created at least one pair-wise link;
• program instructions to store the generated plurality of rules in an online repository;
• program instructions to retrieve and utilize the stored plurality of rules when a plurality of future documents are loaded into the online information handling system;
• program instructions to create a plurality of hard and soft links between a word, an image, or a plurality of other media for each document within the ingested plurality of documents based on the applied statistical model, wherein each hard link within the plurality of hard and soft links meets or exceeds a predetermined threshold and each soft link within the plurality of hard and soft links falls below a predetermined threshold;
• and
• program instructions to create a single annotation between each document within the ingested plurality of documents based on each hard link and each soft link within the plurality of hard and soft links.