# US 2021/0035265 A1 · Cloud computing flexible large area mosaic engine

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0035265 A1
- Document ID: US20210035265A1
- Status: Published application
- Kind: A1
- Application number: 16/964,292
- Filed: 2019-01-28
- Published: 2021-02-04
- Earliest claimed: 2018-01-26
- Assignees: DigitalGlobe, Inc.
- Applicants: Digitalglobe, Inc.
- Inventors: PATRICK YOUNG; PETER SCHMITT; TIM HARRIS
- Web page: https://patentgrep.com/patent/US20210035265A1

## CPC

- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- G09G 5/14: Display of multiple viewports
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06T 7/187: involving region growing; involving region merging; involving connected component labelling
- G06T 7/207: for motion estimation over a hierarchy of resolutions (multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53)
- G09G 2352/00: Parallel handling of streams of display data
- G09G 2370/20: Details of the management of multiple sources of image data
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods for large-scale geospatial mosaic generation with image processing in a cloud computing environment. The approaches described herein specifically leverage scalable cloud computing features to facilitate highly parallel, granular image processing. Front-end image processing techniques allow for generation of a user interface that may provide automated material selection with human operator refinement. The user interface may be a web-based design that provides browse version images at lower resolutions to improve performance of the user interface and generating mosaic recipe. In turn, the mosaic recipe may be provided to back-end image processing that coordinates strip-level jobs and tile-level jobs and highly parallel fashion scalable cloud computing nodes. In turn, very large-scale mosaic images may be generated from geospatial images in computationally and cost-effective manner.

## Claims

### Claim 1

A method for generating a geospatial mosaic image from a plurality of geospatial images, comprising:
executing a front-end image processing system comprising a user interface to generate a mosaic recipe for an area of interest, comprising:
defining the area of interest as a selected geographic area, wherein the area of interest is divided into geographically indexed tiles;
accessing image storage having geospatial images stored therein;
retrieving the geospatial images from the image storage corresponding to the area of interest;
rendering browse version images of at least a portion of the geospatial images from the image storage in a map display of the user interface, wherein the browse version images comprise low resolution image versions of corresponding geospatial images in the image storage; and
receiving a selection of one or more geospatial images for each geographically indexed tile based on the browse version images to define a mosaic recipe of the geospatial images for use in generation of an orthomosaic images; and
executing a back-end image processing system to compile an orthomosaic image based on the mosaic recipe, comprising:
retrieving the geospatial images corresponding to the mosaic recipe from a full resolution image storage and storing the geospatial images in scalable cloud storage;
processing the geospatial images as the geospatial images are received at the scalable cloud storage using an image strip processor, the processing at least comprising dividing the geospatial images into image tiles of corresponding geographic relation to the geographically indexed tiles and storing the image tiles in scalable cloud storage;
determining, in response to storing of the image tiles for each said geospatial image, available tile-level operations capable of being performed for each geographically indexed tile based availability of said image tiles corresponding to the mosaic recipe for a given geographically indexed tile;
generating a queue of operations comprising the available tile-level operations to be performed relative to the image tiles; and
executing the queue of operations using a scalable cloud computing cluster with parallel operation execution at a plurality of compute nodes of the scalable cloud computing cluster for non-dependent tile-based operations to generate the mosaic.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
automatically populating the area of interest with browse version images corresponding to said geospatial images in the map display of the user interface based on metadata information regarding the geospatial images corresponding to the area of interest.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
searching the image store of geospatial images based on metadata information;
presenting geospatial image search results comprising browse version images to the user in the map display, wherein the selection includes a said geospatial image of the geospatial image search results.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
creating an image stack of the browse version images comprising a plurality of zoom levels of the map display.

### Claim 5 (depends on 4)

The method of claim 4, wherein a change to the selection of the one or more geospatial images is reflected in the image stack at the plurality of zoom levels.

### Claim 6 (depends on 1)

The method of claim 1, wherein the geospatial images from the image store comprise a previously generated geospatial mosaic.

### Claim 7 (depends on 1)

The method of claim 1, wherein the processing further includes the image strip processor applying dynamic range adjustment to the geospatial images.

### Claim 8 (depends on 7)

The method of claim 7, wherein the processing further includes the image strip processor performing orthorectification, atmospheric compensation, and pan sharpening on the geospatial images.

### Claim 9 (depends on 1)

The method of claim 1, wherein the tile-level operations comprise intra-tile cutline generation, inter-tile cutline generation, and cutline blending.

### Claim 10 (depends on 1)

The method of claim 1, wherein the processing of the geospatial images using the image strip processor is governed by a directed acyclic graph (DAG).

### Claim 11 (depends on 10)

The method of claim 10, wherein the processing of the geospatial images comprise performing strip-level operations in relation to the temporal order in which the geospatial images are received at the scalable cloud storage with parallel operations performed according to the DAG.

### Claim 12 (depends on 9)

The method of claim 9, wherein the tile-level operations are governed by a directed acyclic graph (DAG) comprising parent operations and dependent child operations, wherein the parallel operation execution is according to the DAG.

### Claim 13 (depends on 1)

The method of claim 1, wherein the number of compute nodes used in said executing the queue of operations is based on a number of operations in the queue.

### Claim 14

A system for generation of a geospatial mosaic image from a plurality of geospatial images, comprising:
a front-end image processing system portion, comprising:
an image server executed in a scalable cloud computing cluster to request and serve geospatial images between network locations including requesting high resolution geospatial images from image storage;
a stack processing module operative to receive the high resolution geospatial images from the image server and generate an image stack comprising low resolution browse version images corresponding to the high resolution geospatial images and to store the browse version images in a scalable cloud storage environment; and
a user interface module operative to generate a web-based interface comprising a map display of an area of interest that is divided into geographically indexed tiles, the map display including said browse version images of corresponding geographical relation to the area of interest, the user interface module also operative to generate a mosaic recipe of said high resolution geospatial images for use in generation of a geospatial mosaic based on a selection of the browse version images displayed, the mosaic recipe defining the high resolution geospatial images for each geographically indexed tile to be used in generation of the geospatial mosaic;
a back-end image processing system portion, comprising:
a management module operative to receive the mosaic recipe from the front-end image processing system;
a strip operation management module operative to request high resolution geospatial imagery at the direction of the management module and generate a strip processing queue for strip-level jobs to be performed relative to the high resolution geospatial imagery at least including dividing the geospatial images into image tiles of corresponding geographic relation to the geographically indexed tiles;
a tile operation management module operative to generate a tile processing queue for tile-level jobs to be performed relative to the image tiles at the direction of the management module, wherein the queue includes parallel operation of non-dependent tile-level jobs; and
a scalable cloud computing cluster operative to dedicate computing nodes for execution of said strip-level jobs and said tile-level jobs, said computing nodes operative to perform said strip-level jobs including the dividing of the geospatial images into tiles and to perform said tile-level jobs.

### Claim 15 (depends on 14)

The system of claim 14, wherein the user interface module is operative to automatically populate the area of interest of the map display with browse version images based on metadata information regarding the geospatial images corresponding to the area of interest.

### Claim 16 (depends on 15)

The system of claim 15, wherein the image storage is searchable by the user interface module to define alternative browse version images for at least a portion of the area of interest.

### Claim 17 (depends on 14)

The system of claim 14, wherein the image stack comprises a plurality of zoom levels of the map display.

### Claim 18 (depends on 17)

The system of claim 17, wherein the stack processing module is operative to propagate a change in the selection of the browse version images through all of the zoom levels of the image stack.

### Claim 19 (depends on 14)

The system of claim 14, wherein the high resolution geospatial images comprise at least one previously generated geospatial mosaic.

### Claim 20 (depends on 14)

The system of claim 14, wherein the strip-level jobs include applying dynamic range adjustment to the high resolution geospatial imagery.

### Claim 21 (depends on 20)

The system of claim 20, wherein the strip-level jobs include orthorectification, atmospheric compensation, and pan sharpening on the high resolution geospatial images.

### Claim 22 (depends on 14)

The system of claim 14, wherein the tile-level jobs include intra-tile cutline generation, inter-tile cutline generation, and cutline blending.

### Claim at position 23

23-25. (canceled)

### Claim 26 (depends on 14)

The system of claim 14, wherein the strip operation management module is operative to generate a directed acyclic graph (DAG) comprising parent strip operations and dependent child strip operations and the strip processing queue is based on the strip-level jobs capable of being performed using the geospatial images as retrieved from the image storage.

### Claim 27 (depends on 24)

The system of claim 24, wherein the tile operation management module is operative to generate a directed acyclic graph (DAG) comprising parent tile operations and dependent child tile operations and the tile processing queue is based on tile-level jobs capable of being performed using the image tiles as processed by the scalable cloud computing cluster.
