# US 12,008,970 B2 · Display device and control method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,970 B2 - Document ID: US12008970B2 - Status: Granted - Kind: B2 - Application number: 17/727,303 - Filed: 2022-04-22 - Published: 2022-08-04 - Granted: 2024-06-11 - Earliest claimed: 2020-01-21 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Hyunjoon Choo; Hyunsoo Jeong; Jonghwa Choi; Jongjin Ko; Kwangyoun Kim; Jangwoo Lee; Kyungah Chang - Web page: https://patentgrep.com/patent/US12008970B2 ## CPC - G09G 3/36: using liquid crystals - A47G 1/02: Mirrors used as equipment (hand, pocket, or shaving mirrors A45D42/00; {optical viewing means for vehicles B60R1/00}) - G06F 1/3231: Monitoring the presence, absence or movement of users - G06F 1/325: {Power saving in peripheral device} - G06F 1/3265: {Power saving in display device} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units} - G06V 20/64: Three-dimensional [3D] objects - G06V 40/168: {Feature extraction; Face representation} - G06V 40/172: {Classification, e.g. identification} - H04N 7/188: {Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253)} - H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) - H04N 23/56: provided with illuminating means - H04N 23/60: Control of cameras or camera modules - H05B 47/125: by using cameras - G09G 2360/144: the light being ambient light - Y02B 20/40: Control techniques providing energy savings, e.g. smart controller or presence detection - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - A47G: HOUSEHOLD OR TABLE EQUIPMENT (book-ends A47B5/00; knives B26B) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H05B: ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL - Y02B: CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS ## Abstract A display device is provided. The display device includes a mirror display, a three-dimensional camera positioned toward a front direction of the mirror display, and a controller configured to detect facial contours of a user from three-dimensional face images of the user photographed at at least two different time points by the three-dimensional camera, identify a change of the facial contours between the two time points based on the detected facial contours, and control the mirror display to display information about the change of the facial contours. ## Claims ### Claim 1 A display device comprising: a mirror display; a three-dimensional camera disposed toward a front direction of the mirror display; a motion sensor configured to sense a motion of a user; and a controller configured to: detect facial contours of a user from three-dimensional face images of the user that were photographed at least two different time points by the three-dimensional camera, identify a change of the facial contours between the at least two different time points based on the detected facial contours, control the mirror display to display information about the change of the facial contours, activate the three-dimensional camera when the motion sensor detects the motion of the user, detect a predefined gesture from a three-dimensional image photographed by the three-dimensional camera, and activate the mirror display when the predefined gesture is detected. ### Claim 2 (depends on 1) The display device of claim 1, wherein the controller is further configured to control the mirror display to display the information about the change of the facial contours by a numerical value. ### Claim 3 (depends on 1) The display device of claim 1, further comprising: a two-dimensional camera positioned toward the front direction of the mirror display, wherein the controller is further configured to control the mirror display to display the information about the change of the facial contours on a two-dimensional face image of the user photographed by the two-dimensional camera. ### Claim 4 (depends on 3) The display device of claim 3, wherein the controller is further configured to store at least one of the three-dimensional face images or the information about the change of the facial contours for each user. ### Claim 5 (depends on 4) The display device of claim 4, wherein the controller is further configured to recognize the user based on the two-dimensional face image of the user photographed by the two-dimensional camera or the three-dimensional face images of the user photographed by the three-dimensional camera. ### Claim 6 (depends on 5) The display device of claim 5, wherein the controller is further configured to identify the change of the facial contours based on a three-dimensional face image previously stored for the recognized user and a current three-dimensional face image of the user photographed by the three-dimensional camera. ### Claim 7 (depends on 3) The display device of claim 3, wherein the controller is further configured to activate the two-dimensional camera in response to detecting a predefined gesture. ### Claim 8 (depends on 3) The display device of claim 3, wherein the controller is further configured to: detect a user indicating area from a two-dimensional image photographed by the two-dimensional camera or one of the three-dimensional face images photographed by the three-dimensional camera, and control the two-dimensional camera to zoom in the detected user indicating area and photograph the user indicating area. ### Claim 9 (depends on 1) The display device of claim 1, wherein the controller is further configured to: determine a face shape type of the user based on the detected facial contours, and control the mirror display to provide information about a makeup style corresponding to the determined face shape type of the user. ### Claim 10 (depends on 1) The display device of claim 1, wherein the controller is further configured to control the mirror display to provide information about a makeup style corresponding to the change of the facial contours. ### Claim 11 (depends on 1) The display device of claim 1, wherein the mirror display comprises: a half mirror positioned on a front surface of the display device, and a display positioned behind the half mirror. ### Claim 12 (depends on 1) The display device of claim 1, wherein the mirror display comprises a display comprising an upper polarizing plate on which a mirror film is disposed. ### Claim 13 (depends on 1) The display device of claim 1, further comprising: a light positioned toward the front direction, wherein the controller is further configured to control brightness of the light based on a three-dimensional image photographed by the three-dimensional camera. ### Claim 14 (depends on 13) The display device of claim 13, wherein the controller is further configured to: detect approach of the user based on the three-dimensional image, and control the light to emit brighter light in response to detecting the user's approach. ### Claim 15 (depends on 13) The display device of claim 13, wherein the controller is further configured to: determine a distance between the user and the display device based on the three-dimensional image, and control brightness of the light based on the determined distance. ### Claim 16 A method of controlling a display device, the display device comprising a mirror display, a motion sensor configured to sense a motion of a user, a two-dimensional camera, and a three-dimensional camera, the method comprising: photographing a three-dimensional face image of a user at a first time point by using the three-dimensional camera; photographing a three-dimensional face image of the user at a second time point by using the three-dimensional camera; detecting a facial contour of the user from the three-dimensional face image of the user photographed at the first time point; detecting a facial contour of the user from the three-dimensional face image of the user photographed at the second time point; identifying a change of the facial contours between the first time point and the second time point based on the detected facial contours; displaying information about the change of the facial contours on the mirror displays; activating the three-dimensional camera when the motion sensor detects the motion of the user; detecting a predefined gesture from a three-dimensional image photographed by the three-dimensional camera; and activating the mirror display when the predefined gesture is detected. ### Claim 17 (depends on 16) The method of claim 16, wherein the displaying of the information about the change of the facial contours on the mirror display comprises displaying the information about the change of the facial contours by a numerical value. ### Claim 18 (depends on 16) The method of claim 16, wherein the displaying of the information about the change of the facial contours on the mirror display comprises: displaying the information about the change of the facial contours on a two-dimensional face image of the user photographed by the two-dimensional camera. ### Claim 19 (depends on 18) The method of claim 18, further comprising: storing at least one of the three-dimensional face images photographed at the first time point and the second time point or the information about the change of the facial contours for each user. ### Claim 20 (depends on 19) The method of claim 19, further comprising: recognizing the user based on the two-dimensional face image of the user photographed by the two-dimensional camera or the three-dimensional face images of the user photographed by the three-dimensional camera, wherein the identifying of the change of the facial contours between the first time point and the second time point comprises determining the change of the facial contours based on the three-dimensional face image stored for the recognized user and photographed at the first time point and the three-dimensional face image of the user photographed at the second time point by the three-dimensional camera.