# US 2008/0260202 A1 · System and method for constructing photorealistic mosaics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0260202 A1
- Document ID: US20080260202A1
- Status: Published application
- Kind: A1
- Application number: 11/963,347
- Filed: 2007-12-21
- Published: 2008-10-23
- Earliest claimed: 2006-12-26
- Inventors: Alan Steven Roth
- Web page: https://patentgrep.com/patent/US20080260202A1

## CPC

- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- G06V 10/24: Aligning, centring, orientation detection or correction of the image
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems and methods for constructing photorealistic mosaics are described. One embodiment of the invention includes capturing an image of a sheet that includes a plurality of pieces of material, determining the location of each piece of material on the sheet from the captured image, extracting images of each piece of material on the sheet from the captured image, storing an image of each piece of material and information concerning the location of the piece of material in a database and using the images in the database and the target image to identify pieces of material to be used in the construction of the photorealistic mosaic.

## Claims

### Claim 1

A process for constructing a photorealistic mosaic of a target image, comprising:
capturing an image of a sheet that includes a plurality of pieces of material;
determining the location of each piece of material on the sheet from the captured image;
extracting images of each piece of material on the sheet from the captured image;
storing an image of each piece of material and information concerning the location of the piece of material in a database; and
using the images in the database and the target image to identify pieces of material to be used in the construction of the photorealistic mosaic.

### Claim 2 (depends on 1)

The process of claim 1, wherein the pieces of material are arranged on the sheet in a regular array.

### Claim 3 (depends on 1)

The process of claim 1, wherein capturing an image of a sheet of material further comprises capturing an image of the sheet of material using a flat bed scanner.

### Claim 4 (depends on 1)

The process of claim 1, wherein capturing an image of a sheet of material further comprises capturing an image of the sheet of material using a digital camera.

### Claim 5 (depends on 1)

The process of claim 1, wherein determining the location of each piece of material on the sheet from the captured image further comprises:
creating a segmentation image from the captured image; and
determining unique regions within the captured image using the segmentation image.

### Claim 6 (depends on 5)

The process of claim 5, wherein determining the location of each piece of material on the sheet from the captured image further comprises:
determining the average area of each unique region;
discarding unique regions that have an area that are less than a predetermined ratio of the average area; and
identifying the remaining unique regions as pieces of material.

### Claim 7 (depends on 6)

The process of claim 6, wherein extracting images of each piece of material on the sheet from the captured image further comprises:
determining the boundaries of each identified piece of material; and
capturing an image that encompasses at least the boundaries of the identified piece of material.

### Claim 8 (depends on 6)

The process of claim 6, wherein extracting images of each piece of material on the sheet from the captured image further comprises straightening the image.

### Claim 9 (depends on 1)

The process of claim 1, wherein storing an image of each piece of material and information concerning the location of the piece of material in a database further comprises storing information in the database concerning the sheet on which the piece of material is located and information identifying the location of the piece of material on the sheet.

### Claim 10 (depends on 1)

The process of claim 1, wherein storing an image of each piece of material and information concerning the location of the piece of material in a database further comprises storing multiple images of the piece of material, where each image is an image of the piece of material with a different orientation.

### Claim 11

A process for constructing a photorealistic mosaic of a target image, comprising:
creating a database of images of an inventory of pieces of material;
dividing the target image into an array of regions;
comparing at least a subset of the images in the database with each region;
selecting a piece of material for each region based upon the comparisons; and
developing an assembly plan for retrieving pieces of information from the inventory and using them in the construction of the photorealistic mosaic of the target image.

### Claim 12 (depends on 11)

The process of claim 11, wherein dividing the target image into an array of regions further comprises scaling the target image.

### Claim 13 (depends on 11)

The process of claim 11, wherein dividing the target image into an array of regions further comprises dividing the target image into a regular array of regions.

### Claim 14 (depends on 11)

The process of claim 11, wherein dividing the target image into an array of regions further comprises:
performing edge detection to define lines within the target image; and
dividing the target image into an array of regions, where the array includes boundaries between regions that follow the lines defined within the target image.

### Claim 15 (depends on 11)

The process of claim 11, wherein comparing at least a subset of the images in the database with each region further comprises:
selecting a region of the target image;
defining a sub-region within the region of the target image;
selecting an image of a piece of material from the database; and
comparing the sub-region of the target image with the corresponding portion of the image of a piece of material.

### Claim 16 (depends on 15)

The process of claim 15, wherein comparing the sub-region of the target image with the corresponding portion of the image of a piece of material further comprises determining the root mean square error for each of the color components of the sub-region of the target image and the corresponding portion of the image of the piece of material.

### Claim 17 (depends on 15)

The process of claim 15, wherein comparing at least a subset of the images in the database with each region further comprises:
defining multiple sub-regions for each region of the target image;
repeating the comparison of each sub-region to each corresponding portion of the image of the piece of material; and
recording information concerning the overall match between the region of the target image and the image of the piece of material.

### Claim 18 (depends on 17)

The process of claim 17, wherein selecting a piece of material for each region based upon the comparisons further comprises:
determining the region and the piece of material that are the closest match;
selecting the piece of material;
removing the selected piece of material from the available pieces; and
determining which of the remaining regions and available pieces of material is the closest match.

### Claim 19

A process for constructing a photorealistic mosaic of a target image, comprising:
creating a database of images of an inventory of pieces of material and information concerning the location of the pieces of material within an inventory of pieces of material;
using the images in the database and the target image to select pieces of material to be used in the construction of the photorealistic mosaic of the target image; and
creating instructions for retrieving pieces of material from an inventory and for placing the pieces of material in the photorealistic mosaic.

### Claim 20 (depends on 19)

The process of claim 19, wherein creating instructions for retrieving pieces of material from an inventory and for orienting and placing the pieces of material in the photorealistic mosaic further comprises creating at least one document that includes information concerning a subset of pieces of material to be used in the construction of the photorealistic image, the location of the pieces of material within the inventory, the placement of each piece of material within the photorealistic mosaic and the orientation with which each of piece material is to be placed within the photorealistic mosaic.

### Claim 21 (depends on 20)

The process of claim 20, wherein creating instructions for retrieving pieces of material from an inventory and for orienting and placing the pieces of material in the photorealistic mosaic further comprises dividing the photorealistic mosaic into subsections and creating an assembly plan for each subsection.

### Claim 22 (depends on 20)

The process of claim 20, wherein the assembly plan for each subsection includes an array of images of pieces of material, information concerning the location of each piece of material within the inventory and information concerning the orientation of each piece of material.

### Claim 23

A process for constructing a photorealistic mosaic of a target image, comprising:
capturing an image of a sheet of a plurality of pieces of material;
determining the location of each piece of material on the sheet from the captured image;
extracting images of each piece of material on the sheet from the captured image;
storing an image of each piece of material and information concerning the location of the piece of material in a database;
dividing the target image into an array of regions;
comparing at least a subset of the images in the database with each region by defining sub-regions within each region of the target image and comparing each sub-region with the corresponding portion of the image of each piece of material in the subset;
selecting a piece of material for each region based upon the comparisons; and
creating instructions for retrieving pieces of material from an inventory and for placing the pieces of material in the photorealistic mosaic.

### Claim 24

A system for constructing a photorealistic mosaic from a target image, comprising:
an inventory of pieces of material;
a database describing the inventory of material;
a computer configured to construct an assembly plan for a mosaic by matching images of pieces of material from the database with regions of the target image.
