# US 2019/0378383 A1 · Systems and methods for transient processing of an audio signal for enhanced haptic experience

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378383 A1
- Document ID: US20190378383A1
- Status: Published application
- Kind: A1
- Application number: 16/435,341
- Filed: 2019-06-07
- Published: 2019-12-12
- Earliest claimed: 2018-06-07
- Assignees: Lofelt GmbH
- Applicants: Lofelt GmbH
- Inventors: Daniel Büttner; Sebastian Roos
- Web page: https://patentgrep.com/patent/US20190378383A1

## CPC

- G06N 7/02: using fuzzy logic (computing arrangements based on biological models G06N3/00; computing arrangements using knowledge-based models G06N5/00)
- G08B 6/00: Tactile signalling systems, e.g. tactile personal calling systems
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/065: Analogue means
- G06N 3/08: Learning methods
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N 20/20: Ensemble learning
- G10H 1/0091: {Means for obtaining special acoustic effects (combined with modulation G10H1/043)}
- G10H 1/12: by filtering complex waveforms (G10H1/14, G10H1/16 take precedence)
- G10H 1/46: Volume control
- H03G 3/3005: {in amplifiers suitable for low-frequencies, e.g. audio amplifiers (H03G3/32, H03G3/34 take precedence)}
- H03G 9/14: for gain control and tone control
- H04R 1/1008: {Earpieces of the supra-aural or circum-aural type}
- H04R 1/1075: {Mountings of transducers in earphones or headphones}
- H04R 5/033: Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)}
- H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)}
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G10H 2210/281: Reverberation or echo
- G10H 2210/315: Dynamic effects for musical purposes, i.e. musical sound effects controlled by the amplitude of the time domain audio envelope, e.g. loudness-dependent tone colour or musically desired dynamic range compression or expansion
- G10H 2220/311: with controlled tactile or haptic feedback effect; output interfaces therefor
- G10H 2250/025: Envelope processing of music signals in, e.g. time domain, transform domain or cepstrum domain
- H03G 5/165: {Equalizers; Volume or gain control in limited frequency bands}
- H04R 3/04: for correcting frequency response
- H04R 2205/022: Plurality of transducers corresponding to a plurality of sound channels in each earpiece of headphones or in a single enclosure
- H04R 2400/03: Transducers capable of generating both sound as well as tactile vibration, e.g. as used in cellular phones
- H04R 2430/03: Synergistic effects of band splitting and sub-band processing
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- H03G: CONTROL OF AMPLIFICATION
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Systems and methods for generating a haptic output from an audio signal having a continuous stream of sampled digital audio data are provided. A haptic processing system receives the digital audio data, analyses the digital audio data for processing and extracts haptic signals for generating a haptic effect through an actuator. The method includes passing the digital audio signal on through dynamic processor(s), adjusting the dynamic range of the digital audio signal, extracting the signal envelope of the audio data, synthesising low-frequency signals from the extracted signal envelope, and enhancing the low-frequency content using a resonator. The haptic output is generated by mixing the digital audio signal with outputs from the different modules of the haptic processing system. An analytics module monitors, controls and adjusts the processing of the digital audio signal at the noise gate module, the compressor module and the envelope module to enhance the haptic output.

## Claims

### Claim 1

A haptic processing method for processing a transient present in an audio input signal for generating haptic output, the haptic processing method comprising:
receiving the input audio signal simultaneously in a digital signal processing chain and an analytics processor;
processing the input audio signal in the analytics processor to detect the presence of a transient;
passing the input audio signal simultaneously on to a fast envelope follower and a slow envelope follower;
calculating the difference in output values between the fast envelope follower and slow envelope follower;
calculating a transient score based on the difference in the calculated values;
receiving a transient threshold;
deriving a transient binary based on the transient score, wherein the transient binary is one (1) when the transient score is above the transient threshold, and the transient binary is zero (0), when the transient score is below the transient threshold;
calculating a transient output by multiplying the transient binary with the transient score; and
providing the transient output to adjust the processing of haptic data in the digital signal processing chain.

### Claim 2 (depends on 1)

The haptic processing method of claim 1, wherein the analytics processor comprises a transient analysis detector and an input signal analysis and RMS generator.

### Claim 3 (depends on 2)

The haptic processing method of claim 2, wherein the transient analysis detector and input signal analysis and RMS generator are associated with a predictive and intelligence processor configured to gather information for processing of the transients in the input audio signal.

### Claim 4 (depends on 1)

The haptic processing method of claim 1, wherein the input signal analysis and RMS generator temporarily sets the root mean square value to zero (0), when the transient is detected in the input audio signal.

### Claim 5 (depends on 1)

The haptic processing method of claim 1, wherein the analytics processor controls the opening and closing of the dual filter.

### Claim 6 (depends on 5)

The haptic processing method of claim 5, wherein the dual filter shifts from lower cut-off frequency to upper cut-off frequency, when a transient is detected by the analytics processor.

### Claim 7 (depends on 1)

The haptic processing method of claim 1, wherein the difference of the fast envelope follower and the slow envelope follower output values in decibels represents the transient score.

### Claim 8 (depends on 1)

The haptic processing method of claim 1, wherein the transient threshold is set using a preset processor.

### Claim 9

A haptic processing system for processing the transients in an audio input signal for generating haptic output data, the haptic processing system comprising:
a digital signal processing chain comprising one or more dynamic processors; and
an analytics processor comprising at least a transient detector, the transient detector performing the steps of:
detecting a transient signal in the input audio signal by passing the input audio signal simultaneously on to a fast envelope follower and a slow envelope follower,
calculating the difference in output values between the fast envelope follower and the slow envelope follower,
calculating a transient score based on the difference in the calculated values;
receiving a transient threshold,
deriving a transient binary based on the transient score, wherein the transient binary is one (1) when the transient score is above the transient threshold and zero (0), when the transient score is below transient threshold,
calculating a transient output by multiplying the transient binary with the transient score, and
providing the transient output to an envelope follower and a dual filter, and the binary score to the transient analysis detector to adjust the processing of a haptic data in the digital signal processing chain.

### Claim 10 (depends on 9)

The haptic processing system of claim 9, wherein the transient analysis detector and an input signal analysis and RMS generator are associated with a predictive and intelligence processor configured to gather information associated with transients in the input audio signal.

### Claim 11 (depends on 9)

The haptic processing system of claim 9, wherein the input signal analysis and RMS generator temporarily sets the root mean square value to zero (0), when the transient is detected in the input audio signal.

### Claim 12 (depends on 9)

The haptic processing system of claim 9, wherein the analytics processor controls the opening and closing of the dual filter.

### Claim 13 (depends on 12)

The haptic processing system of claim 12, wherein the dual filter shifts from lower cut-off frequency to upper cut-off frequency, when the transient is detected by the analytics processor.

### Claim 14 (depends on 9)

The haptic processing system of claim 9, wherein the difference of the fast envelope follower and the slow envelope follower output values in decibels represents the transient score.

### Claim 15 (depends on 9)

The haptic processing system of claim 9, wherein the transient binary is calculated based on the transient score.

### Claim 16 (depends on 9)

The haptic processing system of claim 9, wherein the transient threshold is set using a preset processor.

### Claim 17

A computer program product, the computer program product comprising a non-transitory computer-readable storage medium and a processor, the non-transitory computer readable storage medium storing executable code, when executed by at least one processor cause the processor to perform the steps of:
receiving an input audio signal simultaneously in a digital signal processing chain and an analytics processor;
processing the input audio signal in the analytics processor to detect the presence of transient;
passing the input audio signal simultaneously on to a fast envelope follower and a slow envelope follower;
calculating the difference in output values between the fast envelope follower and slow envelope follower;
calculating a transient score based on the difference in the calculated values;
receiving a transient threshold;
deriving a transient binary based on the transient score, wherein the transient binary is one (1) when the transient score is above the transient threshold, and the transient binary is zero (0) when the transient score is below the transient threshold;
calculating a transient output by multiplying the transient binary with the transient score; and
providing the transient output to an envelope follower and a dual filter, and the binary score to the transient analysis detector to adjust the processing of a haptic data in the digital processing chain.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein the analytics processor comprises a transient analysis detector and an input signal analysis and RMS generator.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein the transient analysis detector and the input signal analysis and RMS generator are associated with a predictive and intelligence processor configured to gather information from the input audio signal.

### Claim 20 (depends on 17)

The computer program product of claim 17, wherein the input signal analysis and RMS generator temporarily sets the root mean square value to zero (0), when the transient is detected in the input audio signal.

### Claim 21 (depends on 17)

The computer program product of claim 17, wherein the analytics processor controls the opening and closing of the dual filter.

### Claim 22 (depends on 21)

The computer program product of claim 21, wherein the dual filter shifts from lower cut-off frequency to upper cut-off frequency, when the transient is detected by the analytics processor.

### Claim 23 (depends on 17)

The computer program product of claim 17, wherein the difference of the fast envelope follower and the slow envelope follower output values in decibels is associated with the transient score.

### Claim 24 (depends on 17)

The computer program product of claim 17, wherein the transient binary is calculated based on the transient score.

### Claim 25 (depends on 17)

The computer program product of claim 17, wherein the transient threshold is set using a preset processor.
