System and method of GPT driven cinematic music generation through text processing

A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method for generating background music tailored for a movie scene, in a smart audio-visual display device, the method comprising:

• receiving, from an input source, a video of the movie scene;
• detecting, by processing circuitry, speech signals in the movie scene;
• extracting, by the processing circuitry, visual information and, when speech signals are detected, spoken dialogue from the movie scene;
• generating, by the processing circuitry, descriptive text from the visual information;
• detecting, by the processing circuitry, emotion categories based on the visual information;
• transcribing, by the processing circuitry, the spoken dialogue into transcribed text;
• translating, via a large language model (LLM) by the processing circuitry, the descriptive text, emotion categories and transcribed text into text-based musical instrument conditions;
• guiding, by the musical instrument conditions, a text-to-music model in the processing circuitry, to generate audio tokens that resonate with the movie scene;
• and
• outputting, by the audio-visual display device, music signals in accordance with the audio tokens in synchronism with the movie scene.