A method for generating a displacement map of a first input dataset and a second input dataset of an input dataset pair, each input dataset having at least one spatial dimension and/or time dimension, the method comprising the steps of
- processing the first input dataset and the second input dataset by means of a neural network based feature extractor (25) so as to generate a feature map hierarchy (50) comprising a base pair of feature maps (20e, 30e) and a refinement pair of feature maps (20b, 30b, 20c, 30c, 20d, 30d), each pair of feature maps (20b, 30b, 20c, 30c, 20d, 30d, 20e, 30e) constituting one level of the feature map hierarchy (50), the refinement pair of feature maps (20b, 30b, 20c, 30c, 20d, 30d) having less coarse resolution in all of their at least one spatial dimension and/or time dimension than the base pair of feature maps (20e, 30e);
- generating an initial displacement map in a displacement generating operation for the base pair of feature maps (20e, 30e) of the feature map hierarchy (50) based on matching a first feature map of the base pair of feature maps (20e, 30e) with a second feature map of the base pair of feature maps (20e, 30e);
- in a displacement refinement operation,
- upscaling the initial displacement map in all of its at least one spatial dimension and/or time dimension to the scale of the refinement pair of feature maps (20b, 30b, 20c, 30c, 20d, 30d) of the feature map hierarchy (50) with a respective upscaling factor and multiplying the values of the initial displacement map with the respective upscaling factor to generate an upscaled initial displacement map,
- generating a warped version of the first feature map of the refinement pair of feature maps (20b, 30b, 20c, 30c, 20d, 30d) by carrying out a warping operation using the upscaled initial displacement map on a first feature map of the refinement pair of feature maps (20b, 30b, 20c, 30c, 20d, 30d) of the feature map hierarchy (50),
- matching the warped version of the first feature map of the refinement pair of feature maps (20b, 30b, 20c, 30c, 20d, 30d) with the second feature map of the refinement pair of feature maps (20b, 30b, 20c, 30c, 20d, 30d) so as to obtain a correction displacement map for the warped version of the first feature map of the refinement pair of feature maps (20b, 30b, 20c, 30c, 20d, 30d) and the second feature map of the refinement pair of feature maps (20b, 30b, 20c, 30c, 20d, 30d), which correction displacement map is added to the upscaled initial displacement map so as to obtain an updated displacement map.