# US 10,761,597 B2 · Using augmented reality technology to address negative emotional states > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,761,597 B2 - Document ID: US10761597B2 - Status: Granted - Kind: B2 - Application number: 16/163,681 - Filed: 2018-10-18 - Published: 2020-04-23 - Granted: 2020-09-01 - Earliest claimed: 2018-10-18 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Cheng Xu; Su Liu; Peng Hui Jiang - Web page: https://patentgrep.com/patent/US10761597B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06T 7/90: Determination of colour characteristics - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An emotional state of a user and one or more images of one or more respective objects observed by the user in an environment using an augmented reality device are received. It is determined whether the one or more objects include at least one negative impact object having a source color that can be changed to improve the emotional state of the user. In response to a determination that the one or more objects include at least one negative impact object having a source color that can be changed to improve the emotional state of the user, the augmented reality device is caused to render the at least one negative impact object with a target color assigned to improve the emotional state of the user. ## Claims ### Claim 1 A system for using augmented reality to address negative emotional states, comprising: an augmented reality device associated with a user; and at least one processor device associated with the augmented reality device and operatively coupled to a memory device for storing program code, the at least one processor device being configured to execute program code stored on the memory device to: receive an emotional state of a user and one or more images of one or more respective objects observed by the user in an environment using the augmented reality device; determine, based on a color mapping table and object color rendering criteria, whether the one or more objects include at least one negative impact object having a source color that can be changed to improve the emotional state of the user; in response to a determination that the one or more objects include at least one negative impact object having a source color that can be changed to improve the emotional state of the user, cause the augmented reality device to render the at least one negative impact object with a target color associated with a natural color of the at least one negative impact object assigned to improve the emotional state of the user; monitor a change in the emotional state of the user resulting from the rendering; determine a lack of effectiveness of the rendering based on the change in the emotional state of the user; alter at least one of the color mapping table and the object color rendering criteria to generate an altered color mapping table and altered object color rendering criteria in response to the determination of the lack of effectiveness of the rendering; and cause the augmented reality device to adjust the rendering based on the altered color mapping table and altered object color rendering criteria to obtain an adjusted rendering of the at least one negative impact object to improve the emotional state of the user. ### Claim 2 (depends on 1) The system of claim 1, wherein the at least one processor device is further configured to execute program code stored on the memory device to receive the emotional state of the user from one or more sources in real-time. ### Claim 3 (depends on 2) The system of claim 2, wherein the one or more sources are selected from the group consisting of: communication contexts, biometrics, facial expression, voices, and combinations thereof. ### Claim 4 (depends on 1) The system of claim 1, wherein the at least one processor device is further configured to execute program code stored on the memory device to define the color mapping table and the object color rendering criteria. ### Claim 5 (depends on 1) The system of claim 1, wherein the at least one processor device is further configured to determine that the one or more objects include at least one negative impact object having a source color that can be changed to improve the emotional state of the user by: dividing a given one of the one or more objects into a plurality of sub-objects; recognizing each of the plurality of sub-objects; and identifying a source color of each of the plurality of sub-objects in response to the recognition. ### Claim 6 A computer-implemented method for using augmented reality to address negative emotional states, comprising: receiving an emotional state of a user and one or more images of one or more respective objects observed by the user in an environment using an augmented reality device associated with the user; determining, based on a color mapping table and object color rendering criteria, that the one or more objects include at least one negative impact object having a source color that can be changed to improve the emotional state of the user; causing the augmented reality device to render the at least one negative impact object with a target color associated with a natural color of the at least one negative impact object assigned to improve the emotional state of the user; monitoring a change in the emotional state of the user resulting from the rendering; determining a lack of effectiveness of the rendering based on the change in the emotional state of the user; altering at least one of the color mapping table and the object color rendering criteria to generate an altered color mapping table and altered object color rendering criteria in response to determining the lack of effectiveness of the rendering; and causing the augmented reality device to adjust the rendering based on the altered color mapping table and altered object color rendering criteria to obtain an adjusted rendering of the at least one negative impact object to improve the emotional state of the user. ### Claim 7 (depends on 6) The method of claim 6, further comprising receiving the emotional state of the user from one or more sources in real-time. ### Claim 8 (depends on 7) The method of claim 7, wherein the one or more sources are selected from the group consisting of: communication contexts, biometrics, facial expression, voices, and combinations thereof. ### Claim 9 (depends on 6) The method of claim 6, further comprising defining the emotion impacted color mappings and the object color rendering criteria. ### Claim 10 (depends on 6) The method of claim 6, wherein determining that the one or more objects include at least one negative impact object having a source color that can be changed to improve the emotional state of the user further includes: dividing a given one of the one or more objects into a plurality of sub-objects; recognizing each of the plurality of sub-objects; and identifying a source color of each of the plurality of sub-objects in response to the recognition. ### Claim 11 A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method for using augmented reality to address negative emotional states, the method performed by the computer comprising: receiving an emotional state of a user and one or more images of one or more respective objects observed by the user in an environment using an augmented reality device associated with the user; determining, based on a color mapping table and object color rendering criteria, that the one or more objects include at least one negative impact object having a source color that can be changed to improve the emotional state of the user; causing the augmented reality device to render the at least one negative impact object with a target color associated with a natural color of the at least one negative impact object assigned to improve the emotional state of the user; monitoring a change in the emotional state of the user resulting from the rendering; determining a lack of effectiveness of the rendering based on the change in the emotional state of the user; altering at least one of the color mapping table and the object color rendering criteria to generate an altered color mapping table and altered object color rendering criteria in response to determining the lack of effectiveness of the rendering; and causing the augmented reality device to adjust the rendering based on the altered color mapping table and altered object color rendering criteria to obtain an adjusted rendering of the at least one negative impact object to improve the emotional state of the user. ### Claim 12 (depends on 11) The computer program product of claim 11, wherein the method further comprises receiving the emotional state of the user from one or more sources in real-time. ### Claim 13 (depends on 12) The computer program product of claim 12, wherein the one or more sources are selected from the group consisting of: communication contexts, biometrics, facial expression, voices, and combinations thereof. ### Claim 14 (depends on 11) The computer program product of claim 11, wherein the method further comprises defining the emotion impacted color mappings and the object color rendering criteria. ### Claim 15 (depends on 11) The computer program product of claim 11, wherein determining that the one or more objects include at least one negative impact object having a source color that can be changed to improve the emotional state of the user further includes: dividing a given one of the one or more objects into a plurality of sub-objects; recognizing each of the plurality of sub-objects; and identifying a source color of each of the plurality of sub-objects in response to the recognition.