Delimiter determination in input data

A system comprising:

• a hardware processor;
• and
• a non-transitory storage medium storing instructions executable on the hardware processor to:
• identify candidate record delimiters and candidate field delimiters in input data;
• provide different pairs of candidate record delimiters and candidate field delimiters, wherein each respective pair of the different pairs comprises a corresponding candidate record delimiter and a corresponding field delimiter,
• for each respective pair of the different pairs:
• identify records using the corresponding candidate record delimiter of the respective pair, and
• compute a measure indicating a consistency in a quantity of fields observed in the records identified using the corresponding field delimiter of the respective pair;
• and
• select, based on values of the measure computed for corresponding pairs of the different pairs, a record delimiter and a field delimiter in a pair of the different pairs.

A method comprising:

• applying, by a system comprising a hardware processor, a heuristic-based delimiter identification process that comprises:
• identifying candidate record delimiters and candidate field delimiters in input data;
• providing different pairs of candidate record delimiters and candidate field delimiters, wherein each respective pair of the different pairs comprises a corresponding candidate record delimiter and a corresponding field delimiter,
• for each respective pair of the different pairs:
• identifying records using the corresponding candidate record delimiter of the respective pair, and
• computing a measure indicating a consistency in a quantity of fields observed in the records identified using the corresponding field delimiter of the respective pair;
• and
• selecting, based on values of the measure computed for corresponding pairs of the different pairs, a record delimiter and a field delimiter in a pair of the different pairs.