File-based error handling during ingestion with transformation

A machine comprising:

• at least one processor;
• and
• at least one memory storing instructions that, when executed by the at least one processor, cause the machine to perform operations comprising:
• parsing one or more input files to generate one or more input data columns during a copy operation to transfer data from the one or more input files to a database;
• performing one or more operations on the one or more input data columns to produce one or more result data columns;
• identifying one or more errors occurring during the performing of the one or more operations;
• recording the one or more errors in one or more error indicator arrays corresponding to the one or more result data columns;
• filtering the one or more result data columns based on the one or more error indicator arrays to generate one or more filtered result data columns;
• and
• storing the one or more filtered result data columns into the database.

A machine-storage medium storing instructions that, when executed by a machine, cause the machine to perform operations comprising:

• parsing one or more input files to generate one or more input data columns during a copy operation to transfer data from the one or more input files to a database;
• performing one or more operations on the one or more input data columns to produce one or more result data columns;
• identifying one or more errors occurring during the performing of the one or more operations;
• recording the one or more errors in one or more error indicator arrays corresponding to the one or more result data columns;
• filtering the one or more result data columns based on the one or more error indicator arrays to generate one or more filtered result data columns;
• and
• storing the one or more filtered result data columns into the database.