A computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a computer processing circuit to cause the circuit to perform the method comprising:
- generating, for a question, a first set of question answer data using a first answering pipeline, wherein the first set of question answer data comprises a first set of answers and a set of first pipeline confidence values for each answer in the first set of answers;
- generating, for the question, a second set of question answer data using a second answering pipeline, wherein the second set of question answer data comprises a second set of answers and a set of second pipeline confidence values for each answer in the second set of answers;
- determining, using a weighting formula and a first blending profile associated with the first answering pipeline, a first vote weight for an answer in the first set of question answer data that was assigned a first pipeline highest confidence value by the first answering pipeline;
- determining, using the weighting formula and a second blending profile associated with the second answering pipeline, a second vote weight for an answer in the second set of question answer data that was assigned a second pipeline highest confidence value by the second answering pipeline;
- and
- selecting, as a first answer to the question, the answer with an overall highest vote weight from among the answers in the first set of question answer data and in the second set of question answer data, wherein the first blending profile comprises metadata including:
- a first answer accuracy value (AAV), the first AAV reflecting a first probability that the first answering pipeline will produce a first correct answer to the question;
- a first answer confidence table (ACT), the first ACT reflecting a first set of confidence threshold value and probability association pairs, each first pair indicating a probability that the first answering pipeline will produce an answer with a confidence value exceeding an associated confidence threshold value;
- and
- a first correct answer confidence table (CACT), the first CACT reflecting a second set of confidence threshold value and probability association pairs, each second pair indicating a correct answer probability that the first answering pipeline will produce a correct answer with a second confidence value equal to or greater than the associated second confidence threshold value;
- and
- wherein the weighting formula is used to determine the first vote weight of the answer in the first set of question answer data that was assigned the highest confidence value by the first answering pipeline by:
- obtaining a first CACT probability by searching the first CACT for a CACT probability, the CACT probability associated with a greatest confidence value that does not exceed the confidence value of the answer, the confidence value of the answer assigned by the first answering pipeline;
- obtaining a first ACT probability by searching the first ACT for a ACT probability, the first ACT probability associated with a greatest confidence value that does not exceed the confidence value of the answer;
- calculating a first ratio of the first CACT probability divided by the first ACT probability;
- calculating a first product of the first ratio multiplied by the first AAV;
- and
- assigning the first product as the first vote weight.