# US 2019/0320094 A1 · Systems and methods for imaging fine art paintings

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0320094 A1
- Document ID: US20190320094A1
- Status: Published application
- Kind: A1
- Application number: 16/383,495
- Filed: 2019-04-12
- Published: 2019-10-17
- Earliest claimed: 2018-04-13
- Assignees: Arius Technology Inc.
- Applicants: Arius Technology Inc.
- Inventors: Michael Kevin JACKSON
- Web page: https://patentgrep.com/patent/US20190320094A1

## CPC

- H04N 1/54: Conversion of colour picture signals to a plurality of signals some of which represent particular mixed colours, e.g. for textile printing
- H04N 1/6094: {depending on characteristics of the input medium, e.g. film type, newspaper}
- G01J 3/0208: {using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction}
- G01J 3/0297: {Constructional arrangements for removing other types of optical noise or for performing calibration}
- G01J 3/06: Scanning arrangements {arrangements for order-selection}
- G01J 3/10: Arrangements of light sources specially adapted for spectrometry or colorimetry
- G01J 3/2823: {Imaging spectrometer}
- G01J 3/501: {Colorimeters using spectrally-selective light sources, e.g. LEDs}
- G01N 21/251: {Colorimeters; Construction thereof}
- G01N 21/27: using photo-electric detection (G01N21/31 takes precedence){; circuits for computing concentration (logarithmic circuits G06G7/24; photometric circuits in general G01J)}
- G01N 21/57: Measuring gloss
- G06T 3/14: Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/94: based on local image properties, e.g. for local contrast enhancement
- G06T 7/90: Determination of colour characteristics
- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- H04N 1/3876: {Recombination of partial images to recreate the original image}
- H04N 1/603: {controlled by characteristics of the picture signal generator or the picture reproducer}
- B33Y 50/02: for controlling or regulating additive manufacturing processes
- G01J 2003/104: {Monochromatic plural sources}
- G01N 2201/06113: Coherent sources; lasers
- G01N 2201/12: Circuits of general importance; Signal processing
- H04N 1/6008: {with primary colour signals, e.g. RGB or CMY(K)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N)
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B33Y: ADDITIVE MANUFACTURING, i.e. MANUFACTURING OF THREE-DIMENSIONAL [3D] OBJECTS BY ADDITIVE DEPOSITION, ADDITIVE AGGLOMERATION OR ADDITIVE LAYERING, e.g. BY 3D PRINTING, STEREOLITHOGRAPHY OR SELECTIVE LASER SINTERING

## Abstract

Color values as a function of position within a painting may be determined by acquiring a plurality of sets of images of the painting. Each set of images may correspond to a different region within the painting. Each image in a set of images may be acquired using a different lighting condition. An optical system comprising a telecentric lens may be used to acquire each image. Within a set of images, corresponding lightness values of pixels may be compared. Lightness values varying from other corresponding lightness values more than a threshold amount may be eliminated. The non-eliminated lightness values for a pixel may be used to generate a processed value for the pixel. The processed values for each pixel representing an imaged region may be combined into a single image. Eliminating lightness values may remove specular highlights and/or shadows. Each of the single images corresponding to each set of images may be registered together to generate a color map image of the painting. The generated color map image may be used to correct color data collected by laser scanning the painting. A 3D printer may, for example, use the corrected laser color data in combination with an elevation map of the painting to produce a faithful reproduction of the painting.

## Claims

### Claim 1

A method for generating color map images of paintings, a color map image representing color values of a painting as a function of position in the painting, the method comprising:
for each of a plurality of overlapping regions of the painting acquiring a set of images, the set of images comprising a plurality of images with different ones of the plurality of images acquired while the region is illuminated by light incident on the region of the painting from different directions, the imaging performed by an imaging device including an imaging sensor and an optical system arranged to focus an image of the region of the painting onto the imaging sensor, the optical system having an object space telecentric configuration wherein chief light rays from points in the region that are parallel to an optical axis of the optical system pass centrally through an aperture of the optical system, a front element of the optical system closest to the painting 2 is at least as large as the region;
processing each of the plurality of sets of images to provide a processed image of the corresponding region of the painting from which one or both of specular highlights and shadows are removed or reduced; and
registering the overlapping processed images together to generate the color map image.

### Claim 2 (depends on 1)

The method according to claim 1 wherein processing each of the plurality of sets of images to remove or reduce one or both of specular highlights and shadows comprises comparing lightness values for corresponding pixels of different images in the set of images being processed to identify pixels in each of the images for which lightness values differ by more than a threshold amount from lightness values for the corresponding pixels in other ones of the images.

### Claim 3 (depends on 2)

The method according to claim 2 comprising analyzing surface geometry of the painting in a patch surface surrounding a pixel and based on the geometry determining whether one or more corresponding values of the pixel represent a specular highlight or shadow.

### Claim 4 (depends on 3)

The method according to claim 3 wherein generating the processed image for a region comprises combining corresponding pixel values of the plurality of images while excluding from the combining any pixel values identified as corresponding to a specular highlight or shadow for each pixel in the set of images of the region.

### Claim 5 (depends on 4)

The method according to claim 4 wherein combining the corresponding pixel values comprises determining a single representative value for the corresponding pixel values from the plurality of images.

### Claim 6 (depends on 5)

The method according to claim 5 wherein determining the single representative value comprises one or more of determining an average value, determining a median value and determining a mode value for the corresponding pixel values from the plurality of images.

### Claim 7 (depends on 6)

The method according to claim 6 comprising conditioning the images of the set of images, the conditioning comprising one or more of reducing radial distortion of pixels, compensating for non-perpendicularity of the central optical axis relative to the painting, compensating for non-uniform detection of lighting values and correcting color values.

### Claim 8 (depends on 1)

The method according to claim 1 comprising translating the optical system and imaging sensor relative to the painting to image the different overlapping regions of the painting.

### Claim 9 (depends on 1)

The method according to claim 1 comprising, for each of the plurality of images, controlling a light source to emit light that is incident on the region at an angle to a plane of the painting and from a different direction.

### Claim 10 (depends on 9)

The method according to claim 9 wherein the light source emits collimated light and an angle of incidence of the light beam onto the region is the same across a field of view of the optical system.

### Claim 11 (depends on 10)

The method according to claim 10 wherein the light source comprises a set of light sources arranged in a line.

### Claim 12 (depends on 9)

The method according to claim 9 wherein the emitted light has uniform intensity across a field of view of the optical system.

### Claim 13 (depends on 1)

The method according to claim 1 wherein for a plurality of points on a surface of the painting and radially spaced apart from the central optical axis, the points have differing elevations and the optical system eliminates or reduces radial shifting of colors at the imaging sensor.

### Claim 14 (depends on 1)

The method according to claim 1 wherein for a plurality of points on a surface of the painting, the points having varying elevations, the optical system projects the plurality of points onto an imaging plane of the imaging sensor with a substantially constant magnification as a distance between the front element of the optical system and the painting is adjusted.

### Claim 15 (depends on 1)

The method according to claim 1 wherein the optical system reduces or removes parallax error corresponding to acquiring images of a point on a surface of the painting in different overlapping regions.

### Claim 16 (depends on 1)

The method according to claim 1 further comprising:
scanning the painting with a laser scanner to collect color data representative of the painting;
refining the laser color data with the generated color map image, the refining comprising:
registering the laser color data with the generated color map image;
populating a look-up table using color values from the generated color map image and color values from the laser color data, the look-up table corresponding to a color gamut of pigments used to create the painting; and
replacing one or more color values from the laser color data that do not match color values from the look-up table with color values from the look-up table.

### Claim 17 (depends on 16)

The method according to claim 16 comprising processing the laser color data prior to refining the laser color data, the processing comprising one or more of scaling the laser color data, reducing shading effects present in the laser color data and reducing specular highlights present in the laser color data.

### Claim 18 (depends on 17)

The method according to claim 17 comprising masking out values of pixels in the generated color map image prior to populating the look-up table, the masked out values representing one or more of photometric shading, cast shadows and specular highlights.

### Claim 19 (depends on 18)

The method according to claim 18 wherein the laser scanning comprises scanning the painting at first and second complementary angles.

### Claim 20 (depends on 19)

The method according to claim 19 wherein the laser scanner comprises a 450 nm, a 532 nm and a 638 nm single-mode laser.
