System and method of grounded large vision-language model for remote sensing

A unified framework system for a computer implemented artificial intelligent assistant to perform multiple tasks for remote sensing, comprising:

• task input field for receiving a task identity;
• a global image encoder configured to receive a remote-sensing image and a user query and encode patch-level tokens [“based on the remote-sensing image and the user query”—is the encoding carried out on the image and query?] at a high resolution via interpolation positional encodings using the remote-sensing image and user query;
• an MLP adapter configured to receive the patch-level tokens and adapt the patch-level tokens to a language space;
• and
• a large language model configured to generate natural language responses interleaved with corresponding object locations based on the language space and task specific prompts,
• wherein the global image encoder further includes a region input field for receiving region location parameters, and
• wherein the framework system is configured to switch based on the received task identity between different types of remote sensing visual interpretation tasks.