Latent token representations for passage and answer scoring in question answering systems

A computer system for determining a precise answer to a question using a latent token representation technique, the computer system comprising:

• one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:
• receiving the question in a natural language;
• receiving a plurality of candidate answers and a plurality of documents associated with the plurality of candidate answers in the natural language;
• tokenizing the question, the plurality of the candidate answers, and the plurality of the documents into a corresponding n-gram sequence;
• mapping n-gram elements from the tokenized question to the n-gram elements of the plurality of the tokenized candidate answers and the plurality of the tokenized documents using the latent token representation technique;
• scoring the plurality of tokenized candidate answers based on the latent token representation technique;
• and
• determining the precise answer based on the plurality of scored candidate answers.

A computer program product for determining a precise answer to a question using a latent token representation technique, the computer program product comprising:

• one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor, the program instructions comprising:
• program instructions to receive the question in a natural language;
• program instruction to receive a plurality of candidate answers and a plurality of documents associated with the plurality of candidate answers in the natural language;
• program instructions to tokenize the question, the plurality of the candidate answers, and the plurality of the documents into a corresponding n-gram sequence;
• program instructions to map n-gram elements from the tokenized question to the n-gram elements of the plurality of the tokenized candidate answers and the plurality of the tokenized documents using the latent token representation technique;
• program instructions to score the plurality of tokenized candidate answers based on the latent token representation technique;
• and
• program instructions to determine the precise answer based on the plurality of scored candidate answers.