Error prediction with partial feedback

A system for performing data processing through a pipeline of components, comprising:

• a learning component which generates an error prediction model which models a probability of at least one of a plurality of pipeline components forming a pipeline committing an error for input data, parameters of the error prediction model being learned using a set of training observations, each observation comprising partial user feedback relating to error in data output by the pipeline for respective input data, wherein each of the plurality of pipeline components commits errors for at least some of the input data which contributes to an error in the respective output data;
• an error prediction component which for a new observation for the pipeline, determines which of the plurality of pipeline components in the pipeline contributed to the error in the data output by the pipeline, based on the input data for the new observation and the error prediction model, the new observation comprising input data and optionally further including partial user feedback relating to error in data output by the pipeline for the input data, the partial user feedback for the new observation indicating that an error has occurred without specifying which of the pipeline components contributed to the error;
• and
• a processor which implements the learning component and the error prediction component.

A method for performing data processing through a pipeline of components, comprising:

• providing an error prediction model in computer memory which, for a plurality of pipeline components of a pipeline, models a probability of at least one of the plurality of pipeline components committing an error for input data, parameters of the error prediction model having been learned using a set of training observations, each training observation comprising user feedback relating to error in data output by the pipeline for respective input data, wherein each of the plurality of pipeline components commits errors for at least some of the input data which contributes to an error in the respective output data;
• wrapping each of the plurality of pipeline components in an error module which modifies the output of the respective pipeline component when the error prediction model predicts that the pipeline component has contributed to an observed error in output data;
• receiving a new observation for the pipeline, the new observation comprising input data, the new observation optionally further including user feedback relating to error in data output by the pipeline for the input data, the user feedback for the new observation indicating that an error has occurred without specifying which of the plurality of pipeline components contributed to the error;
• and
• determining which of the plurality of pipeline components in the pipeline contributed to an error in the data output by the pipeline, based on the input data for the new observation and the error prediction model, wherein the determining which of the plurality of pipeline components contributed is performed with a computer processor.