# US 2015/0145882 A1 · Image processing device, image processing method, and program, capable of virtual reproduction of makeup application state

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0145882 A1
- Document ID: US20150145882A1
- Status: Published application
- Kind: A1
- Application number: 14/405,169
- Filed: 2014-04-07
- Published: 2015-05-28
- Earliest claimed: 2013-04-08
- Assignees: Panasonic Intellectual Property Corporation of America
- Applicants: Panasonic Intellectual Property Corporation of America
- Inventors: Phu Nguyen; Hiroto Tomita
- Web page: https://patentgrep.com/patent/US20150145882A1

## CPC

- A45D 44/005: {for selecting or displaying personal cosmetic colours or hairstyle}
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G09G 5/026: {Control of mixing and/or overlay of colours in general (G09G5/022 and G09G5/024 take precedence)}
- H04N 1/46: Colour picture communication systems
- H04N 1/60: Colour correction or control {(H04N1/54 takes precedence)}
- H04N 1/622: {with simulation on a subsidiary picture reproducer}
- A45D: HAIRDRESSING OR SHAVING EQUIPMENT (wigs, toupees or the like A41G3/00, A41G5/00; hairdressers' chairs A47C1/04; hair cutting appliances, razors B26B); EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image processing device generates a make-up image when a make-up coating material is applied to a part of a subject appearing in a source image by processing the source image. A color range of pixels forming a coating material application layer is obtained by expanding a color range of pixels composing the part of the subject by a ratio of a color representing pixels composing the subject to a color of the make-up coating material. Values of pixels of the coating material application layer are obtained by mapping values of pixels of the source image to the color range of the coating material application layer.

## Claims

### Claim 1

An image processing device that performs image processing with respect to a source image including a subject and thereby obtains a make-up image showing the subject in a state in which make-up coating material is applied, the image processing device comprising:
a specification unit that specifies a part of the subject to which the make-up coating material is to be applied;
a generation unit that generates a coating material application layer for the specified part, the coating material application layer being composed from a plurality of pixels having a color range represented by a sample color of the make-up coating material; and
a compositing unit that composites the coating material application layer and the specified part of the source image, obtaining the make-up image, wherein
the color range of the pixels of the coating material application layer is obtained by expanding a color range of a plurality of pixels composing the specified part according to a ratio of a value of the sample color to a representative value of the pixels composing the specified part, and
the generation unit obtains values of the pixels of the coating material application layer by mapping values of pixels composing the source image to the color range of the coating material application layer.

### Claim 2 (depends on 1)

The image processing device of claim 1, wherein
in the source image, the specified part includes highlighted pixels due to lighting by ambient light and shadowed pixels due to interruption of the ambient light, and
the representative value of the pixels composing the specified part is calculated by performing statistical processing with respect to the highlighted pixels and the shadowed pixels.

### Claim 3 (depends on 1)

The image processing device of claim 1, wherein
the image processing device further acquires a distribution map indicating how values of pixels are distributed in a plane coordinate system corresponding to the source image, and
the mapping by the generation unit includes weighting the color range of the coating material application layer using the values of the pixels in the distribution map as weighted coefficients, and applying an offset to values obtained after the weighting.

### Claim 4 (depends on 3)

The image processing device of claim 3, wherein
each of the values indicated in the distribution map is a normalized difference between values of pixels composing the specified part and a minimum value of colors of the pixels composing the specified part, the normalization being performed using the color range of the pixels composing the specified part, and
the offset is a minimum value of a color range of a replacement color.

### Claim 5 (depends on 3)

The image processing device of claim 3, wherein
a sample file is stored in a storage medium that the image processing device accesses, the sample file being a distribution sample of how values of pixels are distributed in a plane, and
the generation unit acquires the distribution map by reading the sample file from the storage medium.

### Claim 6 (depends on 1)

The image processing device of claim 1, wherein
the mapping by the generation unit includes multiplying relative values by a predefined coefficient and adding an offset, each relative value indicating what proportion of the color range of pixels composing the specified part is occupied by a corresponding one of the pixels of the source image.

### Claim 7 (depends on 6)

The image processing device of claim 6, wherein
each of the relative values is a normalized difference between a value of a pixel having a color representative of pixels composing the specified part and a value of a pixel in the specified part, the normalization being performed using a length of the color range of the pixels composing the specified part,
the predefined coefficient is the length of the color range of the pixels composing the specified part, and
the offset is a lower limit of the color range of color in the pixels composing the specified part.

### Claim 8 (depends on 1)

The image processing device of claim 1, wherein
an upper limit of the color range of the coating material application layer is calculated by multiplying a length of the color range of the coating material application layer by a ratio and adding a value of a pixel of a sample color of the make-up coating material, in which the ratio is a ratio of (i) a difference between a representative value of the pixels composing the specified part and an upper limit of the color range of the pixels composing the specified part to (ii) a length of the color range of the pixels composing the specified part, and
a lower limit of the color range of the coating material application layer is calculated by subtracting a length of the color range of the coating material application layer from the upper limit of the color range of the coating material application layer.

### Claim 9 (depends on 1)

The image processing device of claim 1, wherein
the color range of the pixels composing the specified part is a numerical range of a specific type of pixel components among a plurality of types included in the pixels composing the specified part,
the source image includes a plurality of channel images, and
the values of the pixels of the coating material application layer are obtained by mapping each value of a channel image composed of a specific type of pixel component to the color range of the coating material application layer.

### Claim 10 (depends on 9)

The image processing device of claim 9, wherein
the image processing device has a compositing parameter for each component type, and
the compositing by the compositing unit includes weighting and summing the pixel component of the specific type among the channel images included in the coating material application layer and the pixel component of the specific type among the channel images included in the source image, in which the weighting is weighting using a compositing parameter corresponding to the specific type.

### Claim 11 (depends on 9)

The image processing device of claim 9, wherein
the types of pixel component are hue, chroma, and luminance, and the specific type of pixel component is one or a combination of two or more of hue, chroma, and luminance.

### Claim 12 (depends on 1)

The image processing device of claim 1, further comprising:
a first conversion unit that converts a first, a second, and a third pixel component of each pixel composing the specified part to a hue component, a chroma component, and a luminance component, respectively; and
a second conversion unit that converts the hue component, the chroma component, and the luminance component of pixels composing the specified part composited with the coating material application layer to the first, the second, and the third pixel component, respectively.

### Claim 13 (depends on 1)

The image processing device of claim 1, wherein
the specified part to which the make-up coating material is to be applied is lips of the subject,
the specification unit generates an outline defining the lips when specifying the specified part, and
the generation unit, in determining the color range of the pixels of the coating material application layer, specifies a mouth corner defining a shape of a corner portion of the lips from the outline defining the lips, and assigns a low value to pixels in a vicinity of the mouth corner from among pixels composing the lips.

### Claim 14

An image processing method of a computer system that performs image processing with respect to a source image including a subject and thereby obtains a make-up image showing the subject in a state in which make-up coating material is applied, the image processing method comprising:
specifying a part of the subject to which the make-up coating material is to be applied; and
generating a coating material application layer for the specified part, the coating material application layer being composed from a plurality of pixels having a color range represented by a sample color of the make-up coating material, wherein
the coating material application layer is an image that is composited with the source image,
the color range of the pixels of the coating material application layer is obtained by expanding a color range of a plurality of pixels composing the specified part according to a ratio of a value of the sample color to a representative value of the pixels composing the specified part, and
values of the pixels of the coating material application layer are obtained by mapping values of pixels composing the source image to the color range of the coating material application layer.

### Claim 15

A program executed by a computer that causes the computer to perform image processing with respect to a source image including a subject and thereby obtain a make-up image showing the subject in a state in which make-up coating material is applied, the program comprising program code causing the computer to:
specify a part of the subject to which the make-up coating material is to be applied; and
generate a coating material application layer for the specified part, the coating material application layer being composed from a plurality of pixels having a color range represented by a sample color of the make-up coating material, wherein
the coating material application layer is an image that is composited with the source image,
the color range of the pixels of the coating material application layer is obtained by expanding a color range of a plurality of pixels composing the specified part according to a ratio of a value of the sample color to a representative value of the pixels composing the specified part, and
values of the pixels of the coating material application layer are obtained by mapping values of pixels composing the source image to the color range of the coating material application layer.
