US 2026/0065653 A1Published application

Methods and systems for image co-registration of multi-modal temporal sensing

AssigneeTata Consultancy Services Limited

InventorsSYED MUJIBUL ISLAM, SHUBHRANGSHU GHOSH, ABHISHEK ROY CHOUDHURY, ARPAN PAL

Life of this patent

Estimating the term…

Loading patent…

Abstract

The disclosure generally relates to methods and systems for image co-registration of multi-modal temporal sensing. Conventional techniques for image co-registration of multi-modality images focus on either the spatial or temporal domain and thus are not of high accuracy and do not preserve both global and local characteristics for the matching. The present disclosure solves the technical problems in the art for image co-registration of multi-modal temporal sensing using a deep-learning based multi-input-output encoder-decoder network with a Gabor Jet Model. The deep-learning based multi-input-output encoder-decoder network is utilized for the feature extraction. A distinctive Gabor-jet layer of the Gabor Jet Model is utilized for the similarity matching. The Gabor-jet layer generates a Gabor jet graph which provides sparse feature points for matching between matching images and reference images.

Claims12

Dependent claims

A processor-implemented method, comprising:

receiving for a predefined region of interest, via one or more hardware processors, a first image set and a second image set, wherein the first image set comprises a plurality of reference images and the second image set comprises a plurality of matching images, and wherein the first image set and the second image set are orthorectified to each other with one-to-one association;
dividing, via the one or more hardware processors, (i) each reference image of the plurality of reference images to obtain one or more reference image patches associated with each reference image, and (ii) each matching image of the plurality of matching images to obtain one or more matching image patches associated with each matching image, using an image splitting technique;
extracting in a homogeneous latent space, via the one or more hardware processors, using an encoder of a deep-learning based multi-input-output encoder-decoder network, (i) a plurality of reference feature maps associated with each reference image patch of the one or more reference image patches associated with each of the plurality of reference images, (ii) a plurality of matching image feature maps associated with each matching image patch of the one or more matching image patches associated with each of the plurality of matching images;
determining, via the one or more hardware processors, a similarity score between (i) each reference image of the plurality of reference images present in the first image set and (ii) a matching image associated with each reference image and is of the plurality of matching images present in the second image set, by employing a Gabor-Jet based similarity matching technique, using the plurality of reference feature maps associated with each reference image patch of the one or more reference image patches associated with each of the plurality of reference images and the plurality of matching image feature maps associated with each matching image patch of the one or more matching image patches associated with each of the plurality of matching images;
and
generating, via the one or more hardware processors, one or more reference-matching co-registered images for the predefined region of interest, from the plurality of reference images present in the first image set and the plurality of matching images present in the second image set, based on the similarity score between (i) each reference image present in the first image set and (ii) the matching image associated with each reference image and is of the plurality of matching images present in the second image set.

3 more elements

A system, comprising:

a memory storing instructions;
one or more input/output (I/O) interfaces;
and
one or more hardware processors coupled to the memory via the one or more I/O interfaces, wherein the one or more hardware processors are configured by the instructions to:
receive for a predefined region of interest, a first image set and a second image set, wherein the first image set comprises a plurality of reference images and the second image set comprises a plurality of matching images, and wherein the first image set and the second image set are orthorectified to each other with one-to-one association;
divide (i) each reference image of the plurality of reference images to obtain one or more reference image patches associated with each reference image, and (ii) each matching image of the plurality of matching images to obtain one or more matching image patches associated with each matching image, using an image splitting technique;
extract in a homogeneous latent space, using an encoder of a deep-learning based multi-input-output encoder-decoder network, (i) a plurality of reference feature maps associated with each reference image patch of the one or more reference image patches associated with each of the plurality of reference images, (ii) a plurality of matching image feature maps associated with each matching image patch of the one or more matching image patches associated with each of the plurality of matching images;
determine a similarity score between (i) each reference image of the plurality of reference images present in the first image set and (ii) a matching image associated with each reference image and is of the plurality of matching images present in the second image set, by employing a Gabor-Jet based similarity matching technique, using the plurality of reference feature maps associated with each reference image patch of the one or more reference image patches associated with each of the plurality of reference images and the plurality of matching image feature maps associated with each matching image patch of the one or more matching image patches associated with each of the plurality of matching images;
and
generate one or more reference-matching co-registered images for the predefined region of interest, from the plurality of reference images present in the first image set and the plurality of matching images present in the second image set, based on the similarity score between (i) each reference image present in the first image set and (ii) the matching image associated with each reference image and is of the plurality of matching images present in the second image set.

7 more elements

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

receiving for a predefined region of interest, a first image set and a second image set, wherein the first image set comprises a plurality of reference images and the second image set comprises a plurality of matching images, and wherein the first image set and the second image set are orthorectified to each other with one-to-one association;
dividing, (i) each reference image of the plurality of reference images to obtain one or more reference image patches associated with each reference image, and (ii) each matching image of the plurality of matching images to obtain one or more matching image patches associated with each matching image, using an image splitting technique;
extracting in a homogeneous latent space using an encoder of a deep-learning based multi-input-output encoder-decoder network, (i) a plurality of reference feature maps associated with each reference image patch of the one or more reference image patches associated with each of the plurality of reference images, (ii) a plurality of matching image feature maps associated with each matching image patch of the one or more matching image patches associated with each of the plurality of matching images;
determining a similarity score between (i) each reference image of the plurality of reference images present in the first image set and (ii) a matching image associated with each reference image and is of the plurality of matching images present in the second image set, by employing a Gabor-Jet based similarity matching technique, using the plurality of reference feature maps associated with each reference image patch of the one or more reference image patches associated with each of the plurality of reference images and the plurality of matching image feature maps associated with each matching image patch of the one or more matching image patches associated with each of the plurality of matching images;
and
generating one or more reference-matching co-registered images for the predefined region of interest, from the plurality of reference images present in the first image set and the plurality of matching images present in the second image set, based on the similarity score between (i) each reference image present in the first image set and (ii) the matching image associated with each reference image and is of the plurality of matching images present in the second image set.

3 more elements

Dependencies are inferred from the claim text.

Facts

Application: 19/248,167Filed 24 Jun 2025
Priority: IN 20242106441727 Aug 2024
Published: 5 Mar 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification17 codes

G06T 7/37
Determination of transform parameters for the alignment of images, i.e. image registration: using transform domain methods
G06V 10/7747
Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting: organisation of the process, e.g. bagging or boosting
G06T 7/337
Using feature-based methods: involving reference images or patches
G06V 10/761
Image or video pattern matching; Proximity measures in feature spaces: proximity, similarity or dissimilarity measures