# US 2018/0027347 A1 · Method and system for analysing sound

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0027347 A1
- Document ID: US20180027347A1
- Status: Published application
- Kind: A1
- Application number: 15/676,042
- Filed: 2017-08-14
- Published: 2018-01-25
- Earliest claimed: 2011-06-10
- Assignees: X-System Limited
- Applicants: X-System Limited
- Inventors: Nigel OSBORNE; Robert ASHCROFT; Paul ROBERTSON; Peter KINGSLEY
- Web page: https://patentgrep.com/patent/US20180027347A1

## CPC

- G16H 20/70: relating to mental therapies, e.g. psychological therapy or autogenous training
- G06F 16/683: using metadata automatically derived from the content
- G10H 1/0008: {Associated control or indicating means}
- G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/50: for simulation or modelling of medical disorders
- H04R 29/00: Monitoring arrangements; Testing arrangements {(for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308)}
- G10H 2210/066: for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
- G10H 2210/071: for rhythm pattern analysis or rhythm style recognition
- G10H 2220/371: Vital parameter control, i.e. musical instrument control based on body signals, e.g. brainwaves, pulsation, temperature or perspiration; Biometric information
- G10H 2220/376: using brain waves, e.g. EEG
- G10H 2240/085: Mood, i.e. generation, detection or selection of a particular emotional content or atmosphere in a musical piece
- G10H 2240/135: Library retrieval index, i.e. using an indexing scheme to efficiently retrieve a music piece
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

The present invention relates to a method and system for analysing audio (eg. music) tracks. A predictive model of the neuro-physiological functioning and response to sounds by one or more of the human lower cortical, limbic and subcortical regions in the brain is described. Sounds are analysed so that appropriate sounds can be selected and played to a listener in order to stimulate and/or manipulate neuro-physiological arousal in that listener. The method and system are particularly applicable to applications harnessing a biofeedback resource.

## Claims

### Claim 1

A computer implemented system for analysing sounds, such as audio tracks, or any other types of sounds, the system including a processor programmed for automatically analysing sounds according to musical parameters derived from or associated with a predictive model of the neuro-physiological functioning and response to sounds by one or more of the human lower cortical, limbic and subcortical regions in the brain;
and in which the system analyses sounds so that appropriate sounds can be selected and played to a listener in order to stimulate and/or manipulate neuro-physiological arousal in that listener.

### Claim 2 (depends on 1)

The system of claim 1 in which the system is adapted to automatically analyse sounds and store the results of that analysis in a database so that appropriate sounds can subsequently be selected from that database and played to a listener to provide a desired stimulation and/or manipulation of neuro-physiological arousal in that listener.

### Claim 3 (depends on 1)

The system of claim 1 where the musical parameters relate to rhythmicity.

### Claim 4 (depends on 1)

The system of claim 1 where the musical parameters relate to harmonicity, being the degree of correspondence to the harmonic series.

### Claim 5 (depends on 1)

The system of claim 1 where the musical parameters relate to turbulence, being a measure of rate of change and extent of change in musical experience.

### Claim 6 (depends on 3)

The system of claim 3, which predictively models primitive spinal pathways and the pre-motor loop (such as the basal ganglia, vestibular system, cerebellum), all concerned with primal responses to rhythmic impulses, by analysing beat induction, using a specifically calibrated onset window.

### Claim 7 (depends on 3)

The system of claim 3, which predictively models rhythmic pattern recognition and retention regions (such as the secondary auditory cortex of the temporal lobes) by using self-similarity/auto-correlation algorithms.

### Claim 8 (depends on 3)

The system of claim 3 which predictively models the activation of mirror neuron systems, which detect the power, trajectory and intentionality of rhythmic activity, through one or more of: indices of rhythmic power, including computation of volume levels, volume peak density, “troughs”, or the absence of energy and, dynamic profiles of performance energy.

### Claim 9 (depends on 3)

The system of claim 3 which predictively models activation of mirror neuron systems by analysing the profile of expenditure of energy (precipitous for high arousal, smooth for low) before and in between onsets, important mirror neuron information, by a computation of profiles of energy flow leading to significant articulations.

### Claim 10 (depends on 4)

The system of claim 4 which predictively models the functioning and response of Heschl's Gyms to sound by determining levels of harmonicity and inharmonicity.

### Claim 11 (depends on 4)

The system of claim 4 which detects a principal fundamental through calculation of the harmonic product spectrum, then establishes degrees of harmonicity both within and among the spectra of different fundamentals.

### Claim 12 (depends on 11)

The system of claim 11 in which the detection of a principal fundamental and the establishment of degrees of harmonicity is applied both “vertically” to instantaneous moments, and “horizontally” to progressions of pitches and spectra in time (related to the tonotopic mapping of the area around Heschl's Gyms) and expressed in terms of linear harmonic cost, which represents both the rate at which the fundamental is changing, and the harmonic distance of the changes.

### Claim 13 (depends on 12)

The system of claim 12 which predictively models the neurophysiological sensing of simple timbre by Heschl's gyms, superior temporal sulcus, circular insular sulcus by analysing windows of vertical harmonicity at instantaneous moments.

### Claim 14 (depends on 12)

The system of claim 12 which predictively models melodic and harmonic progressions in terms of how far each STFT time slices deviates from the simple ratios of the harmonic series: Linear harmonic cost arises from STFT time slices whose fundamental frequency differs from that of the previous slice; Time slices with no change in fundamental have a cost of zero.

### Claim 15 (depends on 5)

The system of claim 5 in which turbulence combines indices of change in rhythmicity and harmonicity, with auditory brainstem and cortical activity innervating the amygdala, hippocampus and core emotional regions affecting neurotransmission and endocrine systems, including the HPA axis, dopamine circuits and levels of, for example, norepinephrine, melatonin and oxytocin.

### Claim 16 (depends on 1)

The system of claim 1 where the analysis of sounds operates in real-time on locally stored music data and the system includes software, firmware and/or hardware running on a personal computing device.

### Claim 17 (depends on 1)

The system of claim 1 in which data defining the rhythmicity, harmonicity and turbulence for a particular sound, such as a music track or portion of that track, are combined in such a manner as to give a single output representing arousal.

### Claim 18 (depends on 3)

The system of claim 3 that determines rhythmicity using an equation that relates R to B and S, such as the equation R=√B*Ŝ2, and where R is rhythmicity, B is beats per minute, S is the mean of the beat strength.

### Claim 19 (depends on 3)

The system of claim 3 that determines rhythmicity using an equation that relates R to B and S, such as the equation I=C/10−H, where I is inharmonicity, C is linear harmonic cost and H is instantaneous harmonicity.

### Claim 20 (depends on 5)

The system of claim 5 that determines turbulence using an equation that links T to H and P, such as T=dH/dt*P, where T is turbulence, H is harmonicity and P is energy during peak volume.

### Claim 21 (depends on 1)

The system of claim 1 in which the values of rhythmicity, harmonicity and, if applicable, turbulence for a given track are combined and mapped to an n dimensional point p characterising physiological state E and where the system determines excitement using an equation that links E to I, R and T, such as where E=(10*I*R)+T, where E is the excitement state, R is the rhythmicity, I is the inharmonicity and T is the turbulence.

### Claim 22 (depends on 1)

The system of claim 1, wherein the sound is analysed for the purpose of navigation, or discovery, or retrieval, or selection, or matching for a specific requirement, or for playlist creation, or for entraining affect or for entraining mood.

### Claim 23 (depends on 1)

The system of claim 1 wherein the system is for selecting music tracks for playback to a human subject according to a preselected desired arousal state of the human subject, the music tracks being selected according to the model of human neuro-physiological response to music, the neuro-physiological model being used to select the music tracks for playback according to the neuro-physiological response to the music tracks predicted by the neuro-physiological model.

### Claim 24 (depends on 1)

The system of claim 1 wherein a primary music library and analysis software resides on a computer operable, remotely or locally, by a human subject, with the ability to transfer a selection of music to a personal music player device, which then generates a dynamic playlist based on the available music.

### Claim 25 (depends on 1)

The system of claim 1 wherein the model of human neuro-physiological response to sound is refined through machine learning, such as linear regressive and/or neural network approaches.

### Claim 26 (depends on 1)

The system of claim 1 wherein a sensor is used to measure the state of arousal of a human subject, and music categorised by predictive modelling of the model of human neuro-physiological response is streamed or otherwise provided in order to achieve the preselected desired arousal state of the human subject.

### Claim 27 (depends on 1)

The system of claim 1 wherein the system comprises a sensor such that once the sensor is activated the system measures a human subject's initial level of neuro-physiological arousal and the system automatically constructs a playlist that will first mirror this level of arousal, then direct the human subject towards, and help to maintain them at, the preselected desired arousal state of the human subject.

### Claim 28 (depends on 1)

The system of claim 1 where a playlist is created in order to entrain or maintain arousal and direct state of mind and/or affect.

### Claim 29 (depends on 1)

The system of claim 1 wherein measurement of a level of neuro-physiological arousal of an individual is automated and is expressed as a value in order to enable it to be mirrored by a musical effect of any one of a theoretically unlimited number of pieces of music in a database.

### Claim 30 (depends on 1)

The system as claimed in claim 1, operable to detect emotional arousal parameters information of an audio track and further operable to embed the information into the audio track or into an electronic link to the audio track or as metadata associated with the track.

### Claim 31 (depends on 1)

The system as claimed in claim 1 operable to enable an automated search of music stored on remote or local databases for music with signatures meeting defined criteria.

### Claim 32 (depends on 1)

The system as claimed in claim 1 operable to share arousal values in a social networking application.

### Claim 33 (depends on 1)

The system as claimed in claim 1 operable to share arousal values (a) associated with a user interacting with a search engine such as Google®, with that search engine, so that the search engine can then use those values to optimise search and/or advertisement selection by the search engine.

### Claim 34

A method for analysing sound, such as audio tracks or other sound, comprising the step of the sound being automatically analysed according to a predictive model of human neuro-physiological functioning and response to sounds by one or more of the human lower cortical, limbic and subcortical regions in the brain.

### Claim 35 (depends on 34)

The method for analysing sound, such as audio tracks or other sound, as claimed in claim 34, the sound or tracks being for playback to a human subject according to a preselected desired arousal state of the human subject, comprising the steps of:
(i) storing a set of individual audio tracks operable for selection for playback;
(ii) predicting a neuro-physiological response to the individual audio tracks according to a neuro-physiological model of the functioning and response of one or more of the human lower cortical, limbic and subcortical regions in the brain to sounds;
(iii) receiving a selected desired arousal state of the human subject, and
(iv) selecting audio tracks according to the predicted neuro-physiological response to the individual music tracks, and according to the selected desired arousal state of the human subject.

### Claim 36

A computer-implemented method of creating a playlist of tracks generated by automatically or manually analysing musical parameters derived from a predictive model of human lower cortical, limbic and subcortical neuro-physiological functioning and response to the pieces of music in order to entrain arousal and direct state of mind and/or affect.
