# US 2020/0206487 A1 · Device and method for application of topical compositions guided by projected fiducials

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0206487 A1
- Document ID: US20200206487A1
- Status: Published application
- Kind: A1
- Application number: 16/726,587
- Filed: 2019-12-24
- Published: 2020-07-02
- Earliest claimed: 2018-12-27
- Assignees: Johnson & Johnson Consumer Inc.
- Applicants: Johnson & Johnson Consumer Inc.
- Inventors: Albert Durr EDGAR
- Web page: https://patentgrep.com/patent/US20200206487A1

## CPC

- A61B 5/444: {Evaluating skin marks, e.g. mole, nevi, tumour, scar}
- A61M 35/003: {Portable hand-held applicators having means for dispensing or spreading integral media (hand-held massage devices with liquid delivery A61H7/003)}
- A61B 5/103: Measuring devices for testing the shape, pattern, {colour,} size or movement of the body or parts thereof, for diagnostic purposes (measuring instruments specially adapted for dentistry A61C19/04)
- A61B 5/4836: {Diagnosis combined with treatment in closed-loop systems or methods (A61B5/0036 takes precedence)}
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/11: Region-based segmentation
- G06T 2207/30088: Skin; Dermal
- G06T 2207/30204: Marker
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- A61M: DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY (introducing media into or onto the bodies of animals A61D7/00; means for inserting tampons A61F13/26; devices for administering food or medicines orally A61J; containers for collecting, storing or administering blood or medical fluids A61J1/05); DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY (surgery A61B; chemical aspects of surgical articles A61L); DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR {(Electrotherapy, e.g. producing anaesthesia by the use of alternating or intermittent currents A61N1/36021)}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A device and a method are for application of a composition to a treatment surface (e.g., skin). The device may include an optical emitter projecting fiducials to skin; a detector obtaining image data corresponding to an image of an area of skin marked with the fiducials; an applicator applying the composition to a location within the skin area; and a processing arrangement. The processing arrangement is receiving the image data, analyzing the image data to determine a morphology of the skin area based on the fiducials captured within the image, identifying, based on the morphology, a region within the image corresponding to the location aimed by the applicator arrangement, analyzing the image data to determine whether the identified region corresponds to a skin artifact, and directing the applicator arrangement to selectively apply the composition to the location when the skin artifact is detected from the identified region.

## Claims

### Claim 1

A method for application of a composition to a treatment surface of a user, comprising:
projecting, by an optical emitter, a plurality of fiducials to the treatment surface;
obtaining, by a detector arrangement, image data corresponding to an image of an area of the treatment surface marked with the fiducials;
analyzing, by a processing arrangement, the image data to determine a morphology of the area of the treatment surface based on the fiducials captured within the image;
identifying, by the processing arrangement, a region within the image corresponding to a location within the area of the treatment surface aimed for application of the composition by an applicator arrangement, the region being identified based on the morphology of the area of the treatment surface;
analyzing, by the processing arrangement, the image data to determine whether the identified region within the image corresponds to an artifact; and
selectively applying, by an applicator arrangement, the composition to the location within the area of the treatment surface when the artifact is detected from the identified region.

### Claim 2 (depends on 1)

The method of claim 1, wherein the treatment surface is a skin of a face of the user.

### Claim 3 (depends on 1)

The method of claim 1, wherein at least 3 fiducials are projected to the treatment surface.

### Claim 4 (depends on 1)

The method of claim 1, wherein the plurality of fiducials forms a checkerboard pattern.

### Claim 5 (depends on 4)

The method of claim 4, wherein the morphology of the area of the treatment surface is determined using a plurality of fiducial points corresponding to corners of the checkerboard pattern.

### Claim 6 (depends on 1)

The method of claim 1, wherein the morphology is determined by analyzing the image data to determine, for each fiducial, a distance between the optical emitter and a height of a position on the treatment surface marked by each fiducial.

### Claim 7 (depends on 6)

The method of claim 6, wherein the distance is determined by analyzing the image data to compare a fiducial position of each fiducial within the image to calibration data, the calibration data correlating regions within calibration images of a calibration substrate captured by the detector arrangement to distances between the optical emitter and the calibration substrate.

### Claim 8 (depends on 6)

The method of claim 6, wherein the morphology comprises at least one of a curvature and a tilt of the area of the treatment surface.

### Claim 9 (depends on 1)

The method of claim 1, wherein the processing arrangement determines whether the identified region corresponds to the artifact based on a reflectance of the treatment surface detected in the image.

### Claim 10 (depends on 1)

The method of claim 1, wherein the composition is a cosmetic composition comprising a reflectance modifying agent.

### Claim 11 (depends on 1)

The method of claim 1, wherein the composition comprises an active ingredient for treating a skin condition.

### Claim 12 (depends on 1)

The method of claim 1, wherein the analyzing step comprises adjusting the image data to reduce data corresponding to distortions to the fiducials captured within the image, and determining a morphology of the area of the treatment surface based on the fiducials captured within the image using the processed image data.

### Claim 13 (depends on 12)

The method of claim 12, wherein the image data is adjusted using a second set of image data corresponding to at least one component of a second image of the area of the treatment surface obtained by the detector arrangement, wherein the second image is not marked with the fiducials.

### Claim 14

A handheld device for applying a composition to a treatment surface, comprising:
an optical emitter configured to project a plurality of fiducials to the treatment surface;
a detector arrangement configured to obtain image data corresponding to an image of an area of the treatment surface marked with the fiducials;
an applicator arrangement configured to apply the composition to a location within the area of the treatment surface; and
a processing arrangement configured to receive the image data from the detector arrangement, analyze the image data to determine a morphology of the area of the treatment surface based on the fiducials captured within the image, identify a region within the image corresponding to the location the applicator arrangement is configured to apply the composition, the region being identified based on the morphology of the area of the treatment surface, analyze the image data to determine whether the identified region corresponds to an artifact, and direct the applicator arrangement to selectively apply the composition to the location when the artifact is detected from the identified region.

### Claim 15 (depends on 14)

The handheld device of claim 14, wherein the plurality of fiducials comprises at least 3 fiducials.

### Claim 16 (depends on 14)

The handheld device of claim 14, wherein the at least one optical emitter comprises a fiducial light source and a template through which light from the fiducial light source would pass to project the plurality of fiducials to the area of the treatment surface.

### Claim 17 (depends on 14)

The handheld device of claim 14, wherein the plurality of fiducials forms a checkerboard pattern.

### Claim 18 (depends on 14)

The handheld device of claim 14, wherein the detector arrangement comprises at least one light source for delivering light to the area of the treatment surface, and at least one sensor for detecting the light and the projected fiducials from the area of the treatment surface to obtain the image data.

### Claim 19 (depends on 14)

The handheld device of claim 14, wherein the applicator arrangement comprises a nozzle configured to deposit the composition from a pressurized reservoir to form a thin layer of the composition on the treatment surface.

### Claim 20 (depends on 14)

The handheld device of claim 14, wherein the composition comprises at least one of a reflectance modifying agent and an active ingredient for treating a skin condition.

### Claim 21 (depends on 14)

The handheld device of claim 14, wherein the processing arrangement determines the morphology by analyzing the image data to determine, for each fiducial, a distance between the optical emitter and a height of a position on the treatment surface marked by each fiducial.

### Claim 22 (depends on 21)

The handheld device of claim 21, further comprising a computer-accessible medium configured to store calibration data, the calibration data correlating regions within calibration images of a calibration substrate captured by the detector arrangement to distances between the optical emitter and the calibration substrate, and wherein the processing arrangement determines the distance between the optical emitter and the height of the position on the treatment surface marked by each fiducial by comparing a fiducial position of each fiducial within the image to the calibration data.

### Claim 23 (depends on 14)

The handheld device of claim 14, wherein the morphology comprises at least one of a curvature and a tilt of the area of the treatment surface.

### Claim 24 (depends on 14)

The handheld device of claim 14, wherein the processing arrangement determines whether the region within the image includes the artifact based on a reflectance of the treatment surface detected in the image.
