# US 11,908,052 B2 · System and method for digital makeup mirror > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,052 B2 - Document ID: US11908052B2 - Status: Granted - Kind: B2 - Application number: 15/759,819 - Filed: 2017-06-29 - Published: 2018-09-27 - Granted: 2024-02-20 - Earliest claimed: 2016-06-29 - Assignees: EyesMatch Ltd. - Applicants: EyesMatch Ltd. - Inventors: Ofer Saban; Nissi Vilcovsky - Web page: https://patentgrep.com/patent/US11908052B2 ## CPC - G06T 11/60: Creating or editing images; Combining images with text - G06T 7/11: Region-based segmentation - G06T 7/13: Edge detection - G06T 7/194: involving foreground-background segmentation - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 7/73: using feature-based methods - G06T 11/00: Two-dimensional [2D] image generation - G06V 10/56: relating to colour - G06V 20/20: in augmented reality scenes - G06V 20/64: Three-dimensional [3D] objects - G06V 40/162: {using pixel segmentation or colour matching} - G06V 40/166: {using acquisition arrangements} - G06V 40/167: {using comparisons between temporally consecutive images} - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G06V 40/193: {Preprocessing; Feature extraction} - H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation} - H04N 5/44504: {Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits} - H04N 9/74: for obtaining special effects (H04N9/65 - H04N9/73 take precedence) - H04N 21/42204: {User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor (constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205)} - H04N 23/56: provided with illuminating means - H04N 23/63: by using electronic viewfinders - G06Q 30/0282: Rating or review of business operators or products - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/10016: Video; Image sequence - G06T 2207/30201: Face - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract A computer implemented method for emulating a mirror using camera video stream and a display screen to generate a digital mirror. The digital mirror is specifically configured for headshot applications, such as makeup and eyeglass tryout sessions. Provisions are made for correcting the appearance of the face on the screen. Specific implementations enable tracking the movement of the face or specific features of the face and applying virtual makeup or virtual glass or other accessories or other filters to the face. Also, recording sessions and auto-editing provides the user with easy accessibility to the tutorial session and the products used during the session. Products may be ordered from the user's mobile device at any time. ## Claims ### Claim 1 A digital make-up mirror, comprising: a digital screen; a digital camera positioned to generate video stream of a makeup session on a face of a user; a user interface including pre-programmed buttons indicating various stages of a makeup session; a controller coupled to the digital screen and to the digital camera and preprogrammed to perform the operations comprising: receiving video stream from the digital camera; identify outline of a face within the video stream; translate pixels belonging to the face so that the outline is centered within the digital screen, thereby generating a translated video stream; displaying the translated video stream on the digital screen; breaking the video stream into makeup sub-sessions according to input from the pre-programmed buttons identifying the timing of start of each of the sub-sessions and inserting marks into the video stream with links to the pre-programmed buttons to thereby generate a marked video stream; storing the marked video stream. ### Claim 2 (depends on 1) The digital make-up mirror of claim 1, wherein the controller performs the breaking according to input data entered by a user identifying the timing of start of each of the sub-sessions. ### Claim 3 (depends on 2) The digital make-up mirror of claim 2, wherein the makeup sub-sessions include at least eyes, cheeks, and lips. ### Claim 4 (depends on 1) The digital make-up mirror of claim 1, wherein the controller further performs the step of storing a list of products associated with each makeup sub-sessions. ### Claim 5 (depends on 1) The digital make-up mirror of claim 1, wherein the controller further performs the steps of: identifying outlines of face features in the video stream; and, tracking location of each identified face feature in each frame. ### Claim 6 (depends on 5) The digital make-up mirror of claim 5, wherein the controller applies virtual reality by modifying attributes of pixels belonging to identified face features. ### Claim 7 (depends on 6) The digital make-up mirror of claim 6, wherein the controller applies virtual reality by further constructing a mask corresponding to the identified face features. ### Claim 8 (depends on 1) The digital make-up mirror of claim 1, wherein the controller further performs the step comprising: identify attributes of a make-up within the video stream; and, map the attributes to a database of products; wherein the attributes include at least one of color, transparency and glossiness. ### Claim 9 (depends on 1) The digital make-up mirror of claim 1, wherein the controller further performs the step comprising: identifying a sclera within the frame; identifying an iris within the sclera; and, translating pixels belonging to the iris so that the iris appears central within the sclera. ### Claim 10 (depends on 1) The digital make-up mirror of claim 1, wherein the controller further performs the step comprising: receiving the video stream from the camera; identifying facial features within the video stream; flipping each frame over a vertical axis to replace right side with left side of the image to thereby mimic a mirror image; transforming each frame to emulate an image of the user looking directly at a camera positioned behind the screen; cropping each frame to enable display of the face of the user at the center of the digital screen; and, displaying the video stream on the digital screen after the flipping, transforming and cropping operations. ### Claim 11 (depends on 10) The digital make-up mirror of claim 10, wherein the controller further performs the operation comprising identifying facial features within each frame of the video stream so as to track location of each of the facial features within each frame. ### Claim 12 (depends on 11) The digital make-up mirror of claim 11, wherein the facial features include one or more of face, lips, eyes, and eyebrow. ### Claim 13 (depends on 12) The digital make-up mirror of claim 12, wherein the facial features further include chin and nose. ### Claim 14 (depends on 10) The digital make-up mirror of claim 10, wherein identifying facial features comprises identifying outlines of facial features. ### Claim 15 (depends on 10) The digital make-up mirror of claim 10, wherein identifying facial features comprises identifying pixels belonging to each facial feature. ### Claim 16 (depends on 10) The digital make-up mirror of claim 10, wherein the controller further performs the operation comprising: displaying on the digital screen a color pallet corresponding to a makeup category; enabling the user to designate a selection from the color pallet; digitally applying the selection to one of the facial features corresponding to the makeup category. ### Claim 17 (depends on 16) The digital make-up mirror of claim 16, wherein digitally applying the selection comprises changing attributes of pixels belonging to the facial feature. ### Claim 18 (depends on 16) The digital make-up mirror of claim 16, wherein displaying the color pallet comprises displaying a plurality of colors and a plurality of color attributes. ### Claim 19 (depends on 10) The digital make-up mirror of claim 10, wherein color attributes include at least transparency, splashes of glossiness, and glossiness. ### Claim 20 (depends on 19) The digital make-up mirror of claim 19, wherein the controller further performs the operation comprising enabling the user to modify the attributes after digitally applying the selection. ### Claim 21 (depends on 16) The digital make-up mirror of claim 16, wherein digitally applying the selection further comprises tracking location of each of the facial features within each frame and applying changing digital application of the selection according to movement of the facial feature within each frame. ### Claim 22 (depends on 16) The digital make-up mirror of claim 16, wherein digitally applying the selection using a mask having a shape of a selected facial feature. ### Claim 23 (depends on 16) The digital make-up mirror of claim 16, further comprising a look-up table associating the color pallet and products data. ### Claim 24 (depends on 16) The digital make-up mirror of claim 16, wherein the controller further performs the operation comprising displaying a product corresponding to the designated selection. ### Claim 25 (depends on 10) The digital make-up mirror of claim 10, wherein the controller further performs the operation comprising: displaying on the digital screen a plurality of preprogrammed makeup looks; enabling the user to designate a selection from the makeup looks; digitally applying the selection to user face projected onto the digital screen. ### Claim 26 (depends on 25) The digital make-up mirror of claim 25, wherein the controller further performs the operation comprising displaying on the screen images of the products used in generating the makeup look. ### Claim 27 (depends on 10) The digital make-up mirror of claim 10, further comprising an illumination device comprising a plurality of light sources of a plurality of temperatures. ### Claim 28 (depends on 27) The digital make-up mirror of claim 27, wherein the controller further performs the operation comprising changing lighting intensity of the plurality of light sources to generate a desired overall light temperature. ### Claim 29 (depends on 27) The digital make-up mirror of claim 27, wherein the controller further performs the operation comprising: projecting on the digital screen lighting temperature selection; enabling the user to designate a selection from the lighting temperature selection; changing lighting intensity of the plurality of light sources according to the user designation.