# US 12,001,755 B2 · Apparatus and method for caring emotion based on vehicle sound > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,755 B2 - Document ID: US12001755B2 - Status: Granted - Kind: B2 - Application number: 17/512,070 - Filed: 2021-10-27 - Published: 2022-11-10 - Granted: 2024-06-04 - Earliest claimed: 2021-05-10 - Assignees: Hyundai Motor Company; Kia Corporation; Seoul National University R&Db Foundation - Applicants: Hyundai Motor Company; Kia Corporation; Seoul National University R&DB Foundation - Inventors: Ki Chang Kim; Eun Soo Jo; Tae Kun Yun; Sang Jin Hong; Dong Chul Park; Kyoung Jin Chang; Ju In Kim; Jin Sung Lee; Myung Hwan Yun; Sung Ho Kim - Web page: https://patentgrep.com/patent/US12001755B2 ## CPC - G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path} - A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety} - A61B 5/18: for vehicle drivers {or machine operators} - A61B 5/4836: {Diagnosis combined with treatment in closed-loop systems or methods (A61B5/0036 takes precedence)} - A61B 5/6893: {Cars} - A61B 5/7267: {involving training the classification device} - G05B 13/027: {using neural networks only} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G07C 5/085: {using electronic data carriers} - G16H 20/70: relating to mental therapies, e.g. psychological therapy or autogenous training - G16H 40/63: for local operation - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - A61B 5/01: Measuring temperature of body parts {; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue} (clinical contact thermometers G01K13/20) - A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)} - A61B 5/4803: {Speech analysis specially adapted for diagnostic purposes} - A61B 2503/22: Motor vehicles operators, e.g. drivers, pilots, captains - G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An apparatus and a method for caring emotion of a driver based on a vehicle sound, may include a detector configured to detect driving information of the vehicle and body information of a user, a non-transitory storage to store instructions, and a processor to execute the instructions. The processor determines an emotional status of the driver by use of at least one of the driving information or the body information, to generate a healing sound by designing a virtual sound based on the driving information and the emotional status of the driver, and to generate and output the healing sound. ## Claims ### Claim 1 An apparatus of caring emotion of a driver based on a vehicle sound, the apparatus comprising: a detector configured to detect driving information of a vehicle and body information of the driver; a non-transitory storage configured to store instructions; and a processor electrically connected to the detector and the non-transitory storage and configured to execute the instructions, wherein the processor is configured to: determine an emotional status of the driver by use of at least one of the driving information or the body information; generating a healing sound by designing a virtual sound according to the driving information and the emotional status of the driver; and outputting the healing sound, and wherein the processor is configured to generate three emotion models by applying previously extracted personalized topics and emotions determined by a deep learning-based emotion analyzing algorithm. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the processor is configured to perform emotion modeling for modeling the emotion of the driver by utilizing the deep learning-based emotion analyzing algorithm. ### Claim 3 (depends on 2) The apparatus of claim 2, wherein the deep learning-based emotion analyzing algorithm is designed based on a conversational memory network (CMN). ### Claim 4 (depends on 2) The apparatus of claim 2, wherein the processor is configured to: determine emotion adjectives based on the generated three emotion models. ### Claim 5 (depends on 4) The apparatus of claim 4, wherein the personalized topics are extracted by utilizing a Latent Dirichelt Allocation (LDA) algorithm. ### Claim 6 (depends on 4) The apparatus of claim 4, wherein the processor is configured to determine the emotion adjectives using a Latent Semantic Analysis (LSA) algorithm and a Girvan-Newman algorithm. ### Claim 7 (depends on 4) The apparatus of claim 4, wherein the processor is configured to determine an emotion correlation equation by analyzing an emotion index based on the driving information and the determined emotion adjectives. ### Claim 8 (depends on 7) The apparatus of claim 7, wherein the processor is configured to analyze the emotion index using a convolutional neural network (CNN) algorithm. ### Claim 9 (depends on 7) The apparatus of claim 7, wherein the processor is configured to design the virtual sound by utilizing the emotion correlation equation as feedback information. ### Claim 10 (depends on 7) The apparatus of claim 7, wherein the emotion correlation equation is determined through a Multiple Regression Analysis (MRA) algorithm. ### Claim 11 (depends on 1) The apparatus of claim 1, wherein the processor is configured to perform at least one of vibration output, scent output, or light driving, in link to the healing sound. ### Claim 12 A method for caring emotion of a driver based on a vehicle sound, the method comprising: detecting driving information of a vehicle and body information of the driver; determining, by a processor, an emotional status of the driver by use of at least one of the driving information or the body information; generating, by the processor, a healing sound by designing a virtual sound according to the driving information and the emotional status of the driver; and outputting, by the processor, the healing sound, wherein the method further includes performing of emotion modeling, and wherein the performing of the emotion modeling includes generating three emotion models by applying previously extracted personalized topics and emotions determined by a deep learning-based emotion analyzing algorithm. ### Claim 13 (depends on 12) The method of claim 12, further including: the performing of the emotion modeling for modeling the emotion of the driver by utilizing the deep learning-based emotion analyzing algorithm. ### Claim 14 (depends on 13) The method of claim 13, wherein the performing of the emotion modeling includes: extracting personalized topics by utilizing a Latent Dirichelt Allocation (LDA) algorithm; deriving emotions by analyzing the emotion by an emotion classifier, based on the extracted personalized topics; and deriving an emotion adjective based on the generated three emotion models. ### Claim 15 (depends on 14) The method of claim 14, wherein the deriving of the emotion adjective includes: deriving the emotion adjective using a Latent Semantic Analysis (LSA) algorithm and a Girvan-Newman algorithm. ### Claim 16 (depends on 14) The method of claim 14, further including: deriving an emotion correlation equation by analyzing an emotion index based on the driving information and the emotion adjective. ### Claim 17 (depends on 16) The method of claim 16, wherein the deriving of the emotion correlation equation includes: analyzing the emotion index using a convolutional neural network (CNN) algorithm. ### Claim 18 (depends on 16) The method of claim 16, wherein the generating of the healing sound includes: designing the virtual sound by utilizing the emotion correlation equation as feedback information. ### Claim 19 (depends on 18) The method of claim 18, wherein the designing of the virtual sound includes: performing at least one of pitch control, gain control, an APS control, frequency filtering, shepard layer control, or volume control for the virtual sound. ### Claim 20 (depends on 12) The method of claim 12, further including: performing, by the processor, at least one of vibration output, scent output, or light driving, in link to the healing sound.