Neural network training method and device

A neural network training device comprising:

• a memory storing one or more instructions;
• and
• at least one processor configured to execute the one or more instructions stored in the memory,
• wherein, when the at least one processor executes the one or more instructions, the at least one processor is further configured to:
• obtain target modality signals of a first domain aligned in a time order and auxiliary modality signals of a second domain that are not aligned in the time order,
• extract characteristic information of the target modality signals using a first neural network model,
• estimate the time order of the auxiliary modality signals using a second neural network model, and
• train the first neural network model based on a result of the estimation and the characteristic information.

A non-transitory computer-readable storage medium configured to store one or more computer programs including instructions that, when executed by at least one processor, configure the at least one processor for:

• obtaining target modality signals of a first domain aligned in a time order and auxiliary modality signals of a second domain that are not aligned in the time order;
• extracting characteristic information of the target modality signals using a first neural network model;
• estimating the time order of the auxiliary modality signals using a second neural network model;
• and
• training the first neural network model based on a result of the estimation and the characteristic information.