# US 10,481,858 B2 · Generating personalized audio content based on mood

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,481,858 B2
- Document ID: US10481858B2
- Status: Granted
- Kind: B2
- Application number: 15/833,997
- Filed: 2017-12-06
- Published: 2019-06-06
- Granted: 2019-11-19
- Earliest claimed: 2017-12-06
- Assignees: Harman International Industries, Incorporated
- Applicants: Harman International Industries, Incorporated
- Inventors: Adam Boulanger; Joseph Verbeke; Stefan Marti; Davide Di Censo
- Web page: https://patentgrep.com/patent/US10481858B2

## CPC

- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 16/635: Filtering based on additional data, e.g. user or group profiles
- G06F 16/637: {Administration of user profiles, e.g. generation, initialization, adaptation or distribution}
- G06F 16/638: Presentation of query results
- G06F 16/68: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G10H 1/00: Details of electrophonic musical instruments
- G10H 1/0066: {using a MIDI interface}
- G10L 25/63: for estimating an emotional state
- H04H 60/12: wherein another information is substituted for the portion of broadcast information
- H04H 60/16: on playing information
- H04H 60/33: Arrangements for monitoring the users' behaviour or opinions
- H04H 60/35: Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users
- H04H 60/45: for identifying users
- H04H 60/46: for recognising users' preferences
- H04H 60/49: for identifying locations
- H04H 60/59: of video {(recognising characters or patterns in general G06F18/00, G06V20/00)}
- H04H 60/65: for using the result on users' side
- G10H 2240/085: Mood, i.e. generation, detection or selection of a particular emotional content or atmosphere in a musical piece
- H04R 3/12: for distributing signals to two or more loudspeakers {(specially adapted for hearing aids H04R25/407)}
- H04R 2499/13: Acoustic transducers and sound field adaptation in vehicles
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04H: BROADCAST COMMUNICATION (multiplex communication H04J; pictorial communication aspects of broadcast systems H04N)
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

In one embodiment, a personalization application provides emotionally personalized audio content to co-located users. In operation, the personalization application acquires first personalized audio content for a first user based on an emotional state associated with the first user. The personalization application then causes a first audio device associated with the first user to output the first audio content. In addition, the personalization application causes a second audio device associated with a second user that is co-located with the first user to output second audio content while the first audio devices outputs the first audio content. Notably, the second audio content is different from the first audio content.

## Claims

### Claim 1

A method for providing personalized audio content, the method comprising:
determining, via one or more processors, a first emotional state associated with a first user based on sensor data from one or more sensors;
determining that a first version of first source audio content is targeted for output to the first user;
acquiring a second version of the first source audio content, wherein the second version of the first source audio content comprises a multi-track version of the first source audio content or a Musical Instrument Digital Interface (MIDI) version of the first source audio content;
performing one or more audio operations on the second version of the first source audio content based on the first emotional state to generate first personalized audio content;
causing a first audio device associated with the first user to output the first personalized audio content; and
causing a second audio device associated with a second user that is co-located with the first user to output second audio content while the first audio device outputs the first personalized audio content, wherein the second audio content is different from the first personalized audio content.

### Claim 2 (depends on 1)

The method of claim 1, further comprising acquiring the second audio content, wherein the second audio content is based on a second emotional state associated with the second user.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first user and the second user are occupants of a vehicle.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first audio device comprises at least one of an earbud and a speaker.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first personalized audio content includes at least one of musical content, speech content, and noise cancellation content.

### Claim 6 (depends on 1)

The method of claim 1, wherein performing one or more audio operations on the second version of the first source audio content comprises modifying a first characteristic of the second version of the first source audio content based on the first emotional state to generate the first personalized audio content.

### Claim 7 (depends on 6)

The method of claim 6, wherein the first characteristic comprises a frequency characteristic, a musical arrangement selection, an instrument selection, a pitch, a word choice, a type of information, a tone of voice, or a type of voice.

### Claim 8 (depends on 1)

The method of claim 1, further comprising determining the first emotional state based on at least one of image data, biological data, auditory data, tactile data, and electronic activity data.

### Claim 9 (depends on 1)

The method of claim 1, wherein performing one or more audio operations on the second version of the first source audio content comprises determining a goal emotional state based on the first emotional state and modifying the second version of the first source audio content based on the goal emotional state to generate the first personalized audio content.

### Claim 10

One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to provide personalized audio content by performing the steps of:
determining, via one or more processors, a first emotional state associated with a first user based on sensor data from one or more sensors;
determining that a first version of first source audio content is targeted for output to the first user;
acquiring a second version of the first source audio content, wherein the second version of the first source audio content comprises a multi-track version of the first source audio content or a Musical Instrument Digital Interface (MIDI) version of the first source audio content;
performing one or more audio operations on the second version of the first source audio content based on the first emotional state to generate first personalized audio content;
acquiring second personalized audio content based on a second emotional state associated with a second user that is co-located with the first user;
causing a first audio device associated with the first user to output the first personalized audio content; and
causing a second audio device associated with the second user to output the second personalized audio content while the first audio device outputs the first personalized audio content.

### Claim 11 (depends on 10)

The one or more computer-readable storage media of claim 10, wherein performing one or more audio operations on the second version of the first source audio content comprises modifying a first characteristic of the second version of the first source audio content based on the first emotional state to generate the first personalized audio content.

### Claim 12 (depends on 11)

The one or more computer-readable storage media of claim 11, wherein acquiring the second personalized audio content comprises determining that the first source audio content is also targeted for output to the second user, and modifying at least one of the first characteristic and a second characteristic of the first source audio content based on the second emotional state to generate the second personalized audio content.

### Claim 13 (depends on 11)

The one or more computer-readable storage media of claim 11, wherein acquiring the second personalized audio content comprises determining that second source audio content is targeted for output to the second user, and modifying at least one of the first characteristic and a second characteristic of the second source audio content based on the second emotional state to generate the second personalized audio content.

### Claim 14 (depends on 10)

The one or more computer-readable storage media of claim 10, wherein acquiring the second version of the first source audio content comprises selecting the second version of the first source audio content based on the first emotional state.

### Claim 15 (depends on 10)

The one or more computer-readable storage media of claim 10, wherein the first user and the second user are occupants of a vehicle.

### Claim 16 (depends on 10)

The one or more computer-readable storage media of claim 10, wherein the first audio devices comprise at least one of an earbud and a speaker.

### Claim 17 (depends on 10)

The one or more computer-readable storage media of claim 10, wherein the first personalized audio content includes at least one of musical content, speech content, and noise cancellation content.

### Claim 18

A system for providing personalized audio content, the system comprising:
a memory storing a personalization application; and
a processor coupled to the memory, wherein, when executed by the processor, the personalization application causes the processor to:
for a first user included in a plurality of co-located users, determine an emotional state associated with the first user based on sensor data from one or more sensors;
for the first user, determine a goal emotional state based on the emotional state associated with the first user and modify first source audio content targeted for output to the first user based on the goal emotional state to generate first personalized audio content;
cause a first audio device associated with the first user to output the first personalized audio content; and
cause a second audio device associated with a second user included in the plurality of co-located users to output the source audio content or second personalized audio content that is different from the first personalized audio content while the first audio device outputs the first personalized audio content.

### Claim 19 (depends on 18)

The system of claim 18, wherein the plurality of co-located users are occupants of a vehicle.