# US 2020/0402233 A1 · Augmented reality systems and methods for fingernail detection and tracking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402233 A1
- Document ID: US20200402233A1
- Status: Published application
- Kind: A1
- Application number: 16/975,081
- Filed: 2019-02-27
- Published: 2020-12-24
- Earliest claimed: 2018-02-27
- Assignees: BeautyUp Technologies, Inc.
- Applicants: Beautyup Technologies, Inc.
- Inventors: Dennis W. Dawson, Jr.; Filippo Rossi; Taha Fayez Taha; Mohamad Fayez Taha
- Web page: https://patentgrep.com/patent/US20200402233A1

## CPC

- G06T 7/0012: {Biomedical image inspection}
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06Q 10/1093: Calendar-based scheduling for persons or groups
- G06T 7/248: {involving reference images or patches}
- G06T 7/73: using feature-based methods
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 40/107: {Static hand or arm}
- H04L 67/60: Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources (admission control or resource allocation H04L47/70)
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30196: Human being; Person
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A user may be directed to the nail application (e.g., a mobile application that provides the various augmented reality and booking features discussed herein) via interactions with a social media site. For example, a user may browse Facebook and see a friend's post or advertisement regarding a nail design, or more generally, some artwork that the user feels would look nice as a nail design. The user may select the artwork and be redirected to the nail application to begin an augmented reality testing of one or more nail artwork designs and/or schedule treatments. In some embodiments, the social media website utilizes an API provided by the data center (e.g. the entity that coordinates the treatment booking process and provides augmented reality software to the users) so that the user can “Touch to try-on” a particular design team in social media by clicking on the design.

## Claims

### Claim 1

A system comprising:
one or more client applications executable by respective communication devices, each communication device comprising one or more processors in communication with an image capture device, the one or more processors configured with processor-executable instructions included in the client application to perform operations comprising:
receiving, from the image capture device, frames of image data obtained at the image capture device, the image data including a finger of a user;
detecting, within individual frames of the image data, a fingernail region of the finger, the fingernail region comprising a plurality of pixels corresponding to an imaged fingernail of the image data;
identifying, within subsequent frames of the image data, the fingernail region, wherein the fingernail region in subsequent frames is positioned differently than in previous frames based on movement of the image capture device and/or finger of the user;
display, on a display of the communication device,
augmented reality images comprising the subsequent frames of the image data overlaid with a virtual nail treatment occupying at least a portion of the one or more fingernail regions, wherein the augmented reality images depict movement of the finger and fingernail while maintaining the virtual nail treatment on the fingernail region; and
an identifier of the virtual nail treatment, the identifier selectable to initiate communication with a scheduling application configured to schedule an appointment with a service provider of the virtual nail treatment; and
transmit, in response to selection of the identifier, request information associated with the virtual nail treatment identification information.

### Claim 2 (depends on 1)

The system of claim 1, further comprising a server comprising one or more processors configured with processor-executable instructions to perform operations comprising:
receiving, via the network, the request information from the communication device;
identifying, based at least in part on the request information, a service provider corresponding to the request; and
transmitting, via the network, a response to the communication device, the response comprising provider information corresponding to the identified service provider.

### Claim 3 (depends on 2)

The system of claim 2, wherein the request information is associated with a request received from a user, at the communication device, to schedule an appointment corresponding to the displayed virtual nail treatment, and wherein the provider information identifies a service provider capable of providing a nail treatment corresponding to the virtual nail treatment.

### Claim 4 (depends on 3)

The system of claim 3, wherein the provider information further comprises scheduling information indicative of one or more available appointment times of the service provider.

### Claim 5 (depends on 4)

The system of claim 4, wherein the one or more processors of the communication device are further configured with processor-executable instructions included in the client application to perform operations comprising:
receiving, via a user interface of the communication device, a selection from the user of one of the one or more available appointment times of the service provider; and
causing, at least in part, the server to send a scheduling request message to a service provider device associated with the service provider.

### Claim 6 (depends on 2)

The system of claim 2, wherein the request information is associated with a request received from a user, at the communication device, to purchase one or more products corresponding to the displayed virtual nail treatment, and wherein the provider information identifies a networked resource at which the one or more products are available for purchase.

### Claim 7 (depends on 6)

The system of claim 6, wherein the one or more processors of the communication device are further configured with processor-executable instructions included in the client application to perform operations comprising:
determining that the provider information is associated with a product provider;
responsive to determining that the provider information is associated with a product provider, causing the communication device to execute a browser application; and
causing the browser application to navigate to the networked resource identified in the provider information.

### Claim 8 (depends on 1)

The system of claim 1, wherein the one or more processors are further configured with processor-executable instructions to perform operations comprising storing, in a data store associated with the communication device, user profile data, the user profile data corresponding to at least one of virtual nail treatment identification information or a provider information previously accessed by the user.
