System and method for estimating perturbation norm for the spectrum of robustness
AssigneeRobert Bosch GmbH; CARNEGIE MELLON UNIVERSITY
InventorsLeslie Rice, Jeremy Kolter, Wan-Yi Lin
Life of this patent
Estimating the term…
Abstract
A computer-program product storing instructions which, when executed by a computer, cause the computer to, for one or more iterations, update parameters associated with a machine-learning network utilizing perturbations for input data, wherein the perturbations are sampled utilizing Markov chain Monte Carlo, identify a loss value associated with each perturbation in each iteration, and evaluate the machine learning network by identifying an average loss value across each iteration and outputting the average loss value.
Claims19
A computer-implemented method for training a neural network, comprising:
- receiving a set of input data from one or more sensors, wherein the data includes time-series data, image data, video data, or sound data;
- initializing a random perturbation sample associated with the set of input data;
- for one or more iterations, executing one or more steps including:
- computing a loss value associated with the random perturbation sample associated with the set of input data;
- determining a gradient of the loss value associated with one or more parameters of the neural network;
- updating, utilizing one or more machine learning optimizers, the one or more parameters in response to the gradient of the loss value associated with an intermediate-p robustness, wherein the intermediate-p robustness is an expectation of a p-norm of a loss, where 1<p<∞;
- increasing a level of perturbation stress applied to a new perturbation for one or more successive iterations;
- and
- in response to exceeding a first threshold associated with convergence of the neural network, outputting a trained neural network utilizing the updated parameters.
A system including a machine-learning network, comprising:
- a data storage interface configured to receive input data from a sensor, wherein the sensor includes a camera, a radar, a sonar, or a microphone;
- a processor, in communication with the data storage interface, wherein the processor is programmed to:
- receive the input data, wherein the input data is indicative of image, radar, sonar, or sound information;
- initiate a random perturbation sample associated with the input data, wherein the random perturbation sample is derived from path sampling;
- for one or more iterations, executing one or more steps including:
- increase a level of perturbation stress for the random perturbation sample;
- compute a loss value associated with the random perturbation sample;
- determine a gradient of the loss value associated with one or more parameters of the neural network;
- update the one or more parameters utilizing the gradient;
- increasing a level of perturbation stress applied to a new perturbation to be utilized;
- increasing a level and evaluate an intermediate-p robustness for one or more perturbations utilizing an estimator, wherein the intermediate-p robustness is an expectation of a p-norm of a loss, where 1<p<∞;
- and in response to exceeding a first threshold associated with convergence of the neural network, output a trained neural network utilizing updated parameters.
A computer-program product in a computer comprising a non-transitory computer readable storage medium storing instruction, that when executed by the computer, cause the computer to:
- for one or more iterations, update parameters associated with a machine-learning network utilizing perturbations for input data received from a sensor, wherein the sensor includes a camera, a radar, a sonar, or a microphone, and wherein the perturbations are sampled utilizing Markov chain Monte Carlo;
- identify a loss value associated with each perturbation in each iteration;
- evaluate the machine learning network by identifying an average loss value across each iteration and outputting the average loss value;
- determine a gradient of the loss value associated with one or more parameters of the machine-learning network, the gradient of the loss value associated with an intermediate-p robustness, wherein the intermediate-p robustness is an expectation of a p-norm of a loss, where 1<p<∞;
- update the one or more parameters utilizing the gradient;
- and
- output a trained machine-learning network utilizing updated parameters and upon convergence to a first threshold.
Dependencies are inferred from the claim text.