# US 12,217,341 B2 · Communicative light assembly system for digital humans > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,341 B2 - Document ID: US12217341B2 - Status: Granted - Kind: B2 - Application number: 17/571,085 - Filed: 2022-01-07 - Published: 2022-07-14 - Granted: 2025-02-04 - Earliest claimed: 2021-01-08 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Pranav K. Mistry; Jiawei Zhang; Seungsu Hong; Abhijit Z. Bendale - Web page: https://patentgrep.com/patent/US12217341B2 ## CPC - G06T 13/00: Animation - H05B 47/105: in response to determined parameters - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/147: using display panels - G06F 3/16: Sound input; Sound output (speech processing G10L) - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06T 13/40: of characters, e.g. humans, animals or virtual beings - G06V 40/168: {Feature extraction; Face representation} - G06V 40/174: {Facial expression recognition} - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - G09F 13/049: {Edge illuminated signs, boards or panels} - G10L 13/02: Methods for producing synthetic speech; Speech synthesisers - G10L 15/063: {Training} - G10L 15/16: using artificial neural networks - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 25/63: for estimating an emotional state - H05B 47/12: by detecting audible sound - H05B 47/125: by using cameras - G06F 3/012: {Head tracking input arrangements} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns - G09G 2370/18: Use of optical transmission of display information - G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser - G10L 2015/0638: {Interactive procedures} - G10L 2015/227: {of the speaker; Human-factor methodology} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H05B: ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G09F: DISPLAYING; ADVERTISING; SIGNS; LABELS OR NAME-PLATES; SEALS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract Communications pertaining to a digital human can include communicating via a lighting system based on determining an aspect of a user based on one or more sensor-generated signals. A communicative lighting sequence can be determined based on the user attribute. The lighting sequence can correspond to a condition of a digital human and can be configured to communicate to the user the condition of the digital human. The light sequence can be generated with an LED array mounted on a digital human display assembly. ## Claims ### Claim 1 A method, comprising: presenting a digital human on a display panel of a digital human display assembly, wherein the digital human is computer-generated and is configured to engage in an interactive dialog with a user; selecting, with a communicative light system (CLS), a selected lighting sequence from a plurality of lighting sequences based on a status of the digital human in responding to a query from the user during the interactive dialog, wherein each lighting sequence of the plurality of lighting sequences is configured to communicate to the user a different status of the digital human as displayed on the display panel and indicating readiness of the digital human to receive a request from the user; and generating, under control of the CLS, the selected lighting sequence using a light emitting diode (LED) array on the digital human display assembly concurrently with displaying the digital human on the display panel, wherein the LED array is separate from the display panel. ### Claim 2 (depends on 1) The method of claim 1, further comprising: determining, with the CLS, an aspect of the user based on one or more sensor-generated signals; wherein the selected lighting sequence is also determined based on the aspect of the user; and wherein the one or more sensor-generated signals comprise a signal generated by a motion sensor indicating that the user is within a predetermined distance of the digital human display assembly. ### Claim 3 (depends on 1) The method of claim 1, further comprising: determining, with the CLS, an aspect of the user based on one or more sensor-generated signals; wherein the selected lighting sequence is also determined based on the aspect of the user; wherein the one or more sensor-generated signals comprise an image captured by a camera; and wherein the aspect of the user includes at least one of a facial feature, a facial expression, or a physical gesture of the user. ### Claim 4 (depends on 1) The method of claim 1, further comprising: determining, with the CLS, an aspect of the user based on one or more sensor-generated signals; wherein the selected lighting sequence is also determined based on the aspect of the user; wherein the one or more sensor-generated signals comprise an audible signal captured by a microphone; and wherein the aspect of the user is determined based on the audible signal and includes at least one of a request or question posed or response to the digital human by the user. ### Claim 5 (depends on 1) The method of claim 1, further comprising: determining, with the CLS, an aspect of the user based on one or more sensor-generated signals; and selecting the digital human that is displayed by the digital human display assembly from a plurality of computer-generated digital humans based on the aspect of the user. ### Claim 6 (depends on 1) The method of claim 1, wherein the selected lighting sequence indicates one of an active status indicating the digital human is ready to receive a request from the user or an inactive status of the digital human. ### Claim 7 (depends on 1) The method of claim 1, wherein the status of the digital human is an action currently engaged in by the digital human. ### Claim 8 (depends on 1) The method of claim 1, wherein the status of the digital human includes at least one of waiting for a response from the user, listening to the user, or speaking by the digital human. ### Claim 9 (depends on 1) The method of claim 1, wherein the status of the digital human includes a status of an artificial intelligence service coupled to the digital human and configured to interpret user input. ### Claim 10 (depends on 1) The method of claim 1, wherein the LED array comprises a plurality of LEDs. ### Claim 11 A system, comprising: one or more sensors; a light emitting diode (LED) array; a display; and a processor operatively coupled to the one or more sensors, the LED array, and the display, wherein the processor is configured to initiate operations including: presenting a digital human on the display, wherein the digital human is computer-generated and is configured to engage in an interactive dialog with a user; selecting a selected lighting sequence from a plurality of lighting sequences based on a status of the digital human in responding to a query from the user during the interactive dialog, wherein each lighting sequence of the plurality of lighting sequences is configured to communicate to the user a different status of the digital human as displayed on the display and indicating readiness of the digital human to receive a request from the user; and generating the selected lighting sequence using the LED array concurrently with displaying the digital human on the display, wherein the LED array is separate from the display. ### Claim 12 (depends on 11) The system of claim 11, wherein the processor is configured to initiate operations further comprising: determining an aspect of the user based on one or more sensor-generated signals; wherein the selected lighting sequence is also determined based on the aspect of the user; and wherein the one or more sensor-generated signals comprise a signal generated by a motion sensor indicating that the user is within a predetermined distance of the one or more sensors. ### Claim 13 (depends on 11) The system of claim 11, wherein the processor is configured to initiate operations further comprising: determining an aspect of the user based on one or more sensor-generated signals; wherein the selected lighting sequence is also determined based on the aspect of the user; wherein the one or more sensor-generated signals comprise an image captured by a camera; and wherein the aspect of the user includes at least one of a facial feature, a facial expression, or a physical gesture of the user. ### Claim 14 (depends on 11) The system of claim 11, wherein the processor is configured to initiate operations further comprising: determining an aspect of the user based on one or more sensor-generated signals; wherein the selected lighting sequence is also determined based on the aspect of the user; wherein the one or more sensor-generated signals comprise an audible signal captured by a microphone; and wherein the aspect of the user is determined based on the audible signal and includes at least one of a request or question posed or response to the digital human by the user. ### Claim 15 (depends on 11) The system of claim 11, wherein the processor is configured to initiate operations further comprising: determining an aspect of the user based on one or more sensor-generated signals; and selecting the digital human that is displayed by the display from a plurality of computer-generated digital humans based on the aspect of the user. ### Claim 16 (depends on 11) The system of claim 11, wherein the selected lighting sequence indicates one of an active status indicating the digital human is ready to receive a request from the user or an inactive status of the digital human. ### Claim 17 (depends on 11) The system of claim 11, wherein the status of the digital human is an action currently engaged in by the digital human. ### Claim 18 (depends on 11) The system of claim 11, wherein the status of the digital human includes at least one of waiting for a response from the user, listening to the user, or speaking by the digital human. ### Claim 19 (depends on 11) The system of claim 11, wherein the status of the digital human includes a status of an artificial intelligence service coupled to the digital human and configured to interpret user input. ### Claim 20 A computer program product, the computer program product comprising: one or more computer-readable storage media and program instructions collectively stored on the one or more computer-readable storage media, wherein the program instructions are executable by a processor to cause the processor to initiate operations including: presenting a digital human on a display panel of a digital human display assembly, wherein the digital human is computer-generated and is configured to engage in an interactive dialog with a user; selecting a selected lighting sequence from a plurality of lighting sequences based on a status of the digital human in responding to a query from the user during the interactive dialog, wherein each lighting sequence of the plurality of lighting sequences is configured to communicate to the user a different status of the digital human as displayed on the display panel and indicating readiness of the digital human to receive a request from the user; and generating the selected lighting sequence using a light emitting diode (LED) array mounted on the digital human display assembly concurrently with displaying the digital human on the display panel, wherein the LED array is separate from the display panel.