Flexible imputation of missing data

A computing system comprising:

• one or more processors;
• memory storing a dataset;
• wherein the memory is configured to cause the one or more processors to perform operations comprising:
• receiving a data missing record with a plurality of fields out of the dataset, wherein the data missing record is missing a value for a first field out of the plurality of fields;
• identifying, in the dataset, a closest record to the data missing record, wherein the closest record is a single record of a closest hierarchical cluster out of a plurality of hierarchical clusters, wherein the closest hierarchical cluster comprises two or more records;
• determining a mean estimate for the first field based on records in the closest hierarchical cluster comprising two or more records;
• determining a record similarity between the data missing record and the closest record;
• and
• based on the record similarity, the mean estimate, and an observed value for the first field of the closest record, imputing an imputed value for the first field of the data missing record, wherein the imputing comprises adjusting the observed value for the first field of the closest record based on the record similarity.

One or more non-transitory computer-readable media comprising computer-executable instructions that, when executed, cause a computing system to perform a method comprising:

• receiving a dataset with at least one data missing record;
• partitioning the dataset into data missing records and data complete records;
• selecting fields of the dataset for clustering;
• partitioning the data complete records into a plurality of hierarchical clusters via hierarchical clustering with the selected fields;
• receiving a data missing record out of the data missing records, wherein the data missing record comprises a plurality of fields and the data missing record is missing a value for a missing field out of the plurality of fields;
• identifying, out of the data complete records, a closest record to the data missing record, wherein the closest record is a single record of a particular hierarchical cluster of the plurality of hierarchical clusters, wherein the particular hierarchical cluster comprises two or more records;
• determining a mean estimate for the missing field based on records in the particular hierarchical cluster comprising two or more records;
• determining a record similarity between the data missing record and the closest record;
• based on the record similarity, the mean estimate, and an observed value for the missing field of the closest record, imputing an imputed value for the missing field of the data missing record, wherein the imputing comprises adjusting the observed value for the missing field of the closest record based on the record similarity;
• adding the data missing record with the imputed value to a training dataset;
• training a neural network with the training dataset;
• and
• making a prediction with the trained neural network.