Line searching techniques

A method of line-searching to find a minimum value of a loss function projected onto a line, wherein the loss function incorporates a multi-class L1-regularized hinge loss function that is continuous and piece-wise linear, the method comprising the steps of:

• identifying points along the line with respect to an unregularized loss output and a regularization penalty output, the outputs each derived from the loss function , the points identified based on changes in slope of the outputs at the points;
• determining at least one derivative value for an initial point of the outputs and at least one change in derivative value for at least some of the identified points of the outputs using a processor;
• determining a cumulative reduction in loss for at least some of the points of the outputs using, for the initial point, the determined derivative values of the outputs at the initial point, and for points other than the initial point, the change in derivative values at the identified point other than the initial point;
• and
• selecting the point corresponding to the minimum value of the loss function based on a largest cumulative reduction in loss.