# US 2026/0094587 A1 · System and method for creating music-aware virtual assistants

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0094587 A1
- Document ID: US20260094587A1
- Status: Published application
- Kind: A1
- Application number: 19/343,886
- Filed: 2025-09-29
- Published: 2026-04-02
- Earliest claimed: 2024-09-27
- Assignees: Carnegie Mellon University
- Applicants: Carnegie Mellon University
- Inventors: Alexander Wang; David Lindbauer; Chris Donahue
- Web page: https://patentgrep.com/patent/US20260094587A1

## CPC

- G10H 1/0025: {Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece (automatically producing a series of tones G10H1/26)}
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 13/047: Architecture of speech synthesisers
- G10L 13/10: Prosody rules derived from text; Stress or intonation
- G10H 2220/011: Lyrics displays, e.g. for karaoke applications
- G10H 2250/455: Gensound singing voices, i.e. generation of human voices for musical applications, vocal singing sounds or intelligible words at a desired pitch or with desired vocal effects, e.g. by phoneme synthesis
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A system and method provide musically integrated notifications on a user’s device, such as a phone or laptop. The notifications are received as text-based notifications and converted to speech notifications, which are typically done by virtual assistants running on the device. In the system and method disclosed herein, the speech notification undergoes further processing to match the context of music playing on the user’s device. In addition, the system and method consider the prosody of the notification to increase intelligibility of the musically integrated notification, decreasing the perceived disruption to the user.

## Claims

### Claim 1

A system for providing musical notifications comprising:
an input module configured to receive a text-based notification and user music;
a pre-processing module configured to separate the user music into vocals and musical accompaniment and to convert the notification into speech;
a synthesis module configured to generate a melody based on the notification and user music, and to create melodic speech by mapping syllables of the notification to the generated melody; and
an output module configured to integrate the melodic speech and the generated melody into the user music, creating a musically integrated speech notification.

### Claim 2 (depends on 1)

The system of claim 1, wherein the pre-processing module further comprises:
a music information component configured to identify information comprising at least one of melody, chords, beats, and general structure of the user music.

### Claim 3 (depends on 1)

The system of claim 1, wherein the pre-processing module includes a text-to-speech system to convert the notification into speech.

### Claim 4 (depends on 2)

The system of claim 2, wherein the music information component retrieves the information from a database and the information further comprises a click track.

### Claim 5 (depends on 2)

The system of claim 2, wherein the music information component generates the information based on the user music.

### Claim 6 (depends on 1)

The system of claim 1, wherein the synthesis module further comprises:
a prosody-informed melody generation component configured to create the generated melody based on a prosody of the text-based notification and the information related to the user music; and
a musical voice synthesis component configured to create the melodic speech by mapping syllables of the speech to the generated melody.

### Claim 7 (depends on 6)

The system of claim 6, wherein the prosody of the text-based notification comprises a spoken rhythm of the notification.

### Claim 8 (depends on 6)

The system of claim 6, wherein the melodic speech conforms to the generated melody with increased intelligibility compared to a speech generated by a singing voice synthesis system.

### Claim 9 (depends on 6)

The system of claim 6, wherein syllables are marked by identifying estimating an onset time of phonemes and grouping the phonemes into syllables.

### Claim 10 (depends on 1)

The system of claim 1, wherein the generated melody can be inserted at an arbitrary location in time of the user music.

### Claim 11 (depends on 6)

The system of claim 6, wherein the generated melody has one note for each syllable of the melodic speech.

### Claim 12 (depends on 6)

The system of claim 6, wherein a pitch and duration of each syllable in the melodic speech is remapped to match the generated melody.

### Claim 13 (depends on 1)

The system of claim 1, wherein the output module further comprises:
a component configured to overlay the melodic speech onto the user music by slightly decreasing a volume of the user music.

### Claim 14 (depends on 13)

The system of claim 13, further comprising:
a component configured to replace original vocals in the user music with the melodic speech.

### Claim 15

A method for providing musical notifications, comprising:
receiving notification text and user music;
separating the user music into vocals and instrumental accompaniment;
converting the notification text into a spoken message;
generating a melody based on the notification text and user music;
creating melodic speech by mapping syllables of the notification text to the generated melody; and
integrating the melodic speech into the user music, resulting in a musically integrated speech notification.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
identifying information comprising at least one of melody, chords, beats, and general structure of the user music.

### Claim 17 (depends on 15)

The method of claim 15, further comprising:
generating the melody that matches a prosody of the notification text; and
creating the melodic speech by mapping syllables of the notification text to the generated melody.

### Claim 18 (depends on 15)

The method of claim 15, further comprising:
overlaying the melodic speech onto the user music by slightly decreasing the volume of the user music; and
replacing original vocals in the user music with the melodic speech.
