Detecting and processing sections spanning processed document partitions

A system comprising:

• a memory having computer readable instructions;
• and
• one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:
• caching a document partition, the document partition including metadata indicating that the document partition is a portion of a whole document;
• pairing a candidate paragraph from the document partition with a cached paragraph segment;
• determining, using a coherence model, a probability that the candidate paragraph and the cached paragraph segment constitute a semantically coherent paragraph;
• discarding the cached paragraph segment and processing the candidate paragraph and the cached paragraph segment separately based on a determination that the probability is less than a threshold level;
• and
• processing the candidate paragraph and the cached paragraph segment together as a cross-partition paragraph based on a determination that the probability is greater than the threshold level.

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:

• caching a document partition, the document partition including metadata indicating that the document partition is a portion of a whole document;
• pairing a candidate paragraph from the document partition with a cached paragraph segment;
• determining, using a coherence model, a probability that the candidate paragraph and the cached paragraph segment constitute a semantically coherent paragraph;
• discarding the cached paragraph segment and processing the candidate paragraph and the cached paragraph segment separately based on a determination that the probability is less than a threshold level;
• and
• processing the candidate paragraph and the cached paragraph segment together as a cross-partition paragraph based on a determination that the probability is greater than the threshold level.