Method and system for multi-sensor fusion using transform learning

A system for multi-sensor fusion comprising:

• a memory storing instructions;
• one or more communication interfaces;
• and
• one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
• receive a plurality of training data (X1, X2, . . . Xn) from a plurality of sensors connected to a monitored system with a training output (y);
• and
• perform a joint optimization of a set of parameters including (i) sensor specific transforms (T1, T2, . . . Tn) and (ii) sensor specific coefficients (Z1, Z2, . . . Zn), wherein each of the sensor specific transforms and the sensor specific coefficients correspond to a training data in the plurality of training data (X1, X2, . . . Xn), (iii) a fusing transform (T(f)), (iv) a fusing coefficient (Z(f)), and (v) a weight matrix (w), and wherein the joint optimization comprises:
• initializing the sensor specific transforms (T1, T2, . . . Tn) and the fusing transform (T(f)) with a random matrix comprising real numbers between 0 and 1;
• estimating the sensor specific coefficients (Z1, Z2, . . . Zn) based on the initialized sensor specific transforms (T1, T2, . . . Tn) and a corresponding training data from the plurality of training data (X1, X2, . . . Xn);
• estimating the fusing coefficient (Z(f)) based on the initialized fusing transform (T(f)) and the estimated sensor specific coefficients (Z1, Z2, . . . Zn);
• estimating the weight matrix (w) based on the training output (y) and the estimated fusing coefficient (Z(f));
• and
• iteratively performing joint learning using the initialized parameters and the estimated parameters from the set of parameters in a first iteration and learnt parameters thereafter until a termination criterion is met, the joint learning comprising:
• learning each of the sensor specific transforms (T1, T2, . . . Tn) based on a corresponding sensor specific coefficient (Z1, Z2, . . . Zn) and the plurality of training data (X1, X2, . . . Xn);
• learning each of the sensor specific coefficients (Z1, Z2, . . . Zn) based on the fusing transform (T(f)), a corresponding sensor specific transform (T1, T2, . . . Tn), the fusing coefficient (Z(f)) and a corresponding training data from the plurality of training data (X1, X2, . . . Xn), and remaining of the sensor specific coefficients (Z1, Z2, . . . Zn);
• learning the fusing transform (T(f)) based on the sensor specific coefficient (Z1, Z2, . . . Zn) and the fusing coefficient (Z(f));
• learning the fusing coefficient (Z(f)) based on the fusing transform (T(f)), the sensor specific coefficient (Z1, Z2, . . . Zn), the weight matrix (w) and the training output (y);
• and
• learning the weight matrix (w) based on the fusing coefficient (Z(f)) and the training output (y);
• wherein the termination criterion is one of (i) completion of a predefined number of iterations (Maxiter) and (ii) difference of the fusing transform (T(f)) of a current iteration and the fusing transform (T(f)) of a previous iteration being less than an empirically determined threshold value (Tol);
• to obtain jointly (i) the learnt sensor specific transforms (T1, T2, . . . Tn), (ii) the learnt fusing transform (T(f)) and (iii) the learnt weight matrix (w) for the monitored system being sensed by the plurality of sensors.

A computer program product comprising a non-transitory computer readable medium having a computer readable program embodied therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

• receive, via one or more hardware processors, a plurality of training data (X1, X2, . . . Xn) from a plurality of sensors connected to a monitored system with a training output (y);
• and
• perform, via the one or more hardware processors, a joint optimization of a set of parameters including (i) sensor specific transforms (T1, T2, . . . Tn) and (ii) sensor specific coefficients (Z1, Z2, . . . Zn), wherein each of the sensor specific transforms and the sensor specific coefficients correspond to a training data in the plurality of training data (X1, X2, . . . Xn), (iii) a fusing transform (T(f)), (iv) a fusing coefficient (Z(f)), and (v) a weight matrix (w), and wherein the joint optimization comprises:
• initialize the sensor specific transforms (T1, T2, . . . Tn) and the fusing transform (T(f)) with a random matrix comprising real numbers between 0 and 1;
• estimate the sensor specific coefficients (Z1, Z2, . . . Zn) based on the initialized sensor specific transforms (T1, T2, . . . Tn) and a corresponding training data from the plurality of training data (X1, X2, . . . Xn);
• estimate the fusing coefficient (Z(f)) based on the initialized fusing transform (T(f)) and the estimated sensor specific coefficients (Z1, Z2, . . . Zn);
• estimate the weight matrix (w) based on the training output (y) and the estimated fusing coefficient (Z(f));
• and
• iteratively perform joint learning using the initialized parameters and the estimated parameters from the set of parameters, in a first iteration and learnt parameters thereafter until a termination criterion is met, the joint learning comprising:
• learning each of the sensor specific transforms (T1, T2, . . . Tn) based on a corresponding sensor specific coefficient (Z1, Z2, . . . Zn) and the plurality of training data (X1, X2, . . . Xn);
• learning each of the sensor specific coefficients (Z1, Z2, . . . Zn) based on the fusing transform (T(f)), a corresponding sensor specific transform (T1, T2, . . . Tn), the fusing coefficient (Z(f)) and a corresponding training data from the plurality of training data (X1, X2, . . . Xn), and remaining of the sensor specific coefficients (Z1, Z2, . . . Zn);
• learning the fusing transform (T(f)) based on the sensor specific coefficient (Z1, Z2, . . . Zn) and the fusing coefficient (Z(f));
• learning the fusing coefficient (Z(f)) based on the fusing transform (T(f)), the sensor specific coefficient (Z1, Z2, . . . Zn), the weight matrix (w) and the training output (y);
• and
• learning the weight matrix (w) based on the fusing coefficient (Z(f)) and the training output (y);
• wherein the termination criterion is one of (i) completion of a predefined number of iterations (Maxiter) and (ii) difference of the fusing transform (T(f)) of a current iteration and the fusing transform (T(f)) of a previous iteration being less than an empirically determined threshold value (Tol);
• to obtain jointly (i) the learnt sensor specific transforms (T1, T2, . . . Tn), (ii) the learnt fusing transform (T(f)) and (iii) the learnt weight matrix (w) for the monitored system being sensed by the plurality of sensors.