Apparatus and method for processing multi-type sensor signal on the basis of multi-modal deep learning

Abstract

An apparatus for processing multi-type sensor signals on the basis of multi-modal deep learning, the apparatus including: an individual sensor failure diagnosis unit configured to measure a normality of a sensor output of a single modal sensor at each sampling period, and sense an abnormal operation of the sensor on the basis of the measured normality; an inter-sensor mutual failure diagnosis unit including a multi-modal deep auto encoder, and configured to learn a correlation existing between multi-modalities, extract shared representation between modalities from multi-modal inputs on the basis of the learned correlation, and perform an inter-sensor mutual failure diagnosis; and a reconstruction target sensor output value reconstructing unit configured to, when an output value of a specific sensor is missing, predict and reconstruct the output value of the sensor using other sensor information using the shared representation extracted from other modal sensors.

Dependencies are inferred from the claim text.