Method and apparatus for searching multi-vector using span-level sequence compression technique

A computer-implemented method comprising:

• generating a span comprising a span index for any document token;
• encoding an input query token to generate a query token encoding vector;
• encoding the document token to generate a document token encoding vector;
• compressing the document token encoding vector into a span vector according to the span index to output a compressed document token encoding vector;
• and
• calculating a score between the query token encoding vector and the compressed document token encoding vector and retrieving a document token encoding vector having the highest similarity to the query token encoding vector.

A non-transitory computer-readable recording medium having instructions stored thereon, wherein the instructions, when executed by a computer, cause the computer to:

• generate a span comprising a span index for any document token;
• encode an input query token to generate a query token encoding vector;
• encode the document token to generate a document token encoding vector;
• compress the document token encoding vector into a span vector according to the span index to output a compressed document token encoding vector;
• and
• calculate a score between the query token encoding vector and the compressed document token encoding vector and retrieve a document token encoding vector having the highest similarity to the query token encoding vector.