# US 12,118,318 B2 · Typifying emotional indicators for digital messaging > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,318 B2 - Document ID: US12118318B2 - Status: Granted - Kind: B2 - Application number: 18/206,482 - Filed: 2023-06-06 - Published: 2023-11-16 - Granted: 2024-10-15 - Earliest claimed: 2018-10-05 - Assignees: Capital One Services, LLC - Applicants: Capital One Services, LLC - Inventors: Jeremy Phillips; Andrew Beane - Web page: https://patentgrep.com/patent/US12118318B2 ## CPC - G06F 40/30: Semantic analysis - A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens} - A61B 5/01: Measuring temperature of body parts {; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue} (clinical contact thermometers G01K13/20) - A61B 5/021: Measuring pressure in heart or blood vessels - A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety} - A61B 5/6898: {Portable consumer electronic devices, e.g. music players, telephones, tablet computers} - A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34) - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 40/253: Grammatical analysis; Style critique - G06V 40/174: {Facial expression recognition} - G08B 21/18: Status alarms (G08B21/02 takes precedence) - H04L 51/04: Real-time or near real-time messaging, e.g. instant messaging [IM] - H04L 51/063: Content adaptation, e.g. replacement of unsuitable content - G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract The present disclosure provides computing systems and techniques for indicating an emotional and/or environmental state of a user in a digital messaging application. A computing device can determine an emotional and/or environmental state of a first user responsive to reading or responding to a message and can convey the determined emotional and/or environmental state to a second computing device, to be transiently presented by the second computing device. ## Claims ### Claim 1 An apparatus, comprising: a processor; and a memory coupled to the processor, the memory comprising instructions that when executed by the processor cause the processor to: determine state data based on characteristics of interaction by a user with a messaging device and characteristics of the messaging device in response to the interaction; generate a state indication based in part on the state data via a machine learning model, wherein the state indication comprises an indication of an emotional state of the user of the messaging device, an indication of an environmental state of the user of the messaging device, or both the indication of the emotional state and the indication of the environmental state; provide the state indication to a digital message application of the message device; receive a state indication feedback associated with the state indication; and train the machine learning model with the state indication feedback to create a trained state prediction model to predict state indications. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the state data is related to: a biometric data of the user; a current usage of the messaging device; an image of a user of the message device; or an environmental characteristic associated with the messaging device received from a sensor of the messaging device. ### Claim 3 (depends on 2) The apparatus of claim 2, wherein the indication of the emotional state is based at least in part on the state data related to the biometric data. ### Claim 4 (depends on 2) The apparatus of claim 2, wherein the indication of the emotional state is based at least in part on the state data related to the current usage of the messaging device. ### Claim 5 (depends on 2) The apparatus of claim 2, wherein the indication of the emotional state is based at least in part on the state data related to the image in the state data. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein receipt of the state indication feedback comprises receipt of an alternative state indication. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the state data includes information related to an actively used application executed by the messaging device. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the state indication includes at least one emoji related to the emotional state. ### Claim 9 (depends on 1) The apparatus of claim 1, wherein the state indication includes a plurality of emojis, a first one of the plurality of emojis indicative of the emotional or the environmental state and a second one of the plurality of emojis indicative of a second emotional state or a second environmental state. ### Claim 10 (depends on 1) The apparatus of claim 1, wherein the state indication includes a plurality of punctuation marks. ### Claim 11 (depends on 1) The apparatus of claim 1, wherein the indication of the emotional state of the user of the message, is based, in part, on the state data indicating a state of connection of at least one accessory to the messaging device. ### Claim 12 A non-transitory computer readable medium embodied with programming code executable by a processor that cause the processor to: determine state data based on characteristics of interaction by a user with a messaging device and characteristics of the messaging device in response to the interaction; generate a state indication based in part on the state data via a machine learning model, wherein the state indication comprises an indication of an emotional state of the user of the messaging device, an indication of an environmental state of the user of the messaging device, or both the indication of the emotional state and the indication of the environmental state; provide the state indication to a digital message application of the message device; receive a state indication feedback associated with the state indication; and train the machine learning model with the state indication feedback to create a trained state prediction model to predict state indications. ### Claim 13 (depends on 12) The non-transitory computer readable medium of claim 12, wherein the state data is related to: a biometric data of the user; a current usage of the messaging device; an image of a user of the message device; an environmental characteristic associated with the messaging device received from a sensor of the messaging device; or an actively used application executed by the messaging device. ### Claim 14 (depends on 12) The non-transitory computer readable medium of claim 12, wherein the state indication includes: at least one emoji related to the indication of the emotional state or the environmental state; a plurality of emojis, a first one of the plurality of emojis indicative of a second emotional state or a second environmental state and a second one of the plurality of emojis indicative of a third emotional state or a third environmental state; or a plurality of punctuation marks. ### Claim 15 (depends on 12) The non-transitory computer readable medium of claim 12, wherein the indication of the emotional state of user of the message is based, in part, on the state data indicating an actively used application executed by the messaging device or a state of connection of at least one accessory to the messaging device. ### Claim 16 (depends on 12) The non-transitory computer readable medium of claim 12, wherein receipt of the state indication feedback comprises receipt of an alternative state indication. ### Claim 17 A method comprising: determining state data based on characteristics of interaction by a user with a messaging device and characteristics of the messaging device in response to the interaction; generating a state indication based in part on the state data via a machine learning model, wherein the state indication comprises an indication of an emotional state of the user of the messaging device, an indication of an environmental state of the user of the messaging device, or both the indication of the emotional state and the indication of the environmental state; providing the state indication to a digital message application of the message device; receiving a state indication feedback associated with the state indication; and training the machine learning model with the state indication feedback to create a trained state prediction model to predict state indications. ### Claim 18 (depends on 17) The method of claim 17, wherein the state data is related to: a biometric state of the user of the messaging device; a current usage of the messaging device; an image of the user; an environmental characteristic associated with the messaging device received from a sensor of the messaging device; or an actively used application executed by the messaging device. ### Claim 19 (depends on 18) The method of claim 18, wherein the indication of: the emotional state is based at least in part on the biometric state; the emotional state is based at least in part on the current usage; the emotional state is based at least in part on the image; or the environmental state is based at least in part on the environmental characteristic. ### Claim 20 (depends on 17) The method of claim 17, wherein the state indication includes: at least one emoji related to the emotional state; a plurality of emojis, a first one of the plurality of emojis indicative of a second emotional or a second environmental state and a second one of the plurality of emojis indicative of a third emotional or a third environmental state; or a plurality of punctuation marks.