Table indexing and retrieval using intrinsic and extrinsic table similarity measures

A system comprising:

• (a) at least one hardware processor;
• and
• (b) a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by said at least one hardware processor to automatically:
• represent each of a plurality of tables as a multi-field text document in which:
• different modalities of the table are represented as separate fields, and
• a concatenation of all the modalities is represented as a separate, auxiliary field,
• receive a query,
• execute the query on the multi-field text documents, to retrieve a list of candidate tables out of the plurality of tables, wherein the candidate tables have preliminary rankings which result from the execution of the query,
• calculate an intrinsic table similarity score for each of the candidate tables, based on the query and the auxiliary field, wherein said calculation of the intrinsic table similarity score comprises, for each of the candidate tables:
• extracting candidate passages from the auxiliary field using a partially-overlapping sliding window applied to the auxiliary field,
• calculating a relevancy of the query to each of the candidate passages,
• biasing the calculated relevancies in favor of the candidate passages appearing earlier in the auxiliary field, and
• using a highest one of the biased calculated relevancies as the intrinsic table similarity score of the respective candidate table;
• calculate an extrinsic table similarity score for each of the candidate tables, based on a cluster hypothesis of the candidate tables,
• combine:
• the preliminary rankings, the intrinsic table similarity scores, and the extrinsic table similarity scores, to re-rank the candidate tables, and
• provide an ordered list of the re-ranked candidate tables.

A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by at least one hardware processor to automatically:

• represent each of a plurality of tables as a multi-field text document in which:
• different modalities of the table are represented as separate fields, and
• a concatenation of all the modalities is represented as a separate, auxiliary field;
• receive a query;
• execute the query on the multi-field text documents, to retrieve a list of candidate tables out of the plurality of tables, wherein the candidate tables have preliminary rankings which result from the execution of the query;
• calculate an intrinsic table similarity score for each of the candidate tables, based on the query and the auxiliary field, wherein said calculation of the intrinsic table similarity score comprises, for each of the candidates tables:
• extracting candidate passages from the auxiliary field using a partially-overlapping sliding window applied to the auxiliary field,
• calculating a relevancy of the query to each of the candidate passages,
• biasing the calculated relevancies in favor of candidate passages appearing earlier in the auxiliary field, and
• using a highest one of the biased calculated relevancies as the intrinsic table similarity score of the respective candidate table;
• calculate an extrinsic table similarity score for each of the candidate tables, based on a cluster hypothesis of the candidate tables;
• combine:
• the preliminary rankings, the intrinsic table similarity scores, and the extrinsic table similarity scores, to re-rank the candidate tables;
• and
• provide an ordered list of the re-ranked candidate tables.