# US 2019/0379977 A1 · Systems and methods for generating haptic output for enhanced user experience

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0379977 A1
- Document ID: US20190379977A1
- Status: Published application
- Kind: A1
- Application number: 16/435,353
- Filed: 2019-06-07
- Published: 2019-12-12
- Earliest claimed: 2018-06-07
- Assignees: Lofelt GmbH
- Applicants: Lofelt GmbH
- Inventors: Daniel Büttner; Sebastian Roos
- Web page: https://patentgrep.com/patent/US20190379977A1

## CPC

- G06N 7/02: using fuzzy logic (computing arrangements based on biological models G06N3/00; computing arrangements using knowledge-based models G06N5/00)
- G08B 6/00: Tactile signalling systems, e.g. tactile personal calling systems
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/065: Analogue means
- G06N 3/08: Learning methods
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N 20/20: Ensemble learning
- G10H 1/0091: {Means for obtaining special acoustic effects (combined with modulation G10H1/043)}
- G10H 1/12: by filtering complex waveforms (G10H1/14, G10H1/16 take precedence)
- G10H 1/46: Volume control
- H03G 3/3005: {in amplifiers suitable for low-frequencies, e.g. audio amplifiers (H03G3/32, H03G3/34 take precedence)}
- H03G 9/14: for gain control and tone control
- H04R 1/1008: {Earpieces of the supra-aural or circum-aural type}
- H04R 1/1075: {Mountings of transducers in earphones or headphones}
- H04R 5/033: Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)}
- H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)}
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G10H 2210/281: Reverberation or echo
- G10H 2210/315: Dynamic effects for musical purposes, i.e. musical sound effects controlled by the amplitude of the time domain audio envelope, e.g. loudness-dependent tone colour or musically desired dynamic range compression or expansion
- G10H 2220/311: with controlled tactile or haptic feedback effect; output interfaces therefor
- G10H 2250/025: Envelope processing of music signals in, e.g. time domain, transform domain or cepstrum domain
- H03G 5/165: {Equalizers; Volume or gain control in limited frequency bands}
- H04R 3/04: for correcting frequency response
- H04R 2205/022: Plurality of transducers corresponding to a plurality of sound channels in each earpiece of headphones or in a single enclosure
- H04R 2400/03: Transducers capable of generating both sound as well as tactile vibration, e.g. as used in cellular phones
- H04R 2430/03: Synergistic effects of band splitting and sub-band processing
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- H03G: CONTROL OF AMPLIFICATION
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Systems and methods for generating a haptic output from an audio signal having a continuous stream of sampled digital audio data are provided. A haptic processing system receives the digital audio data, analyses the digital audio data for processing and extracts haptic signals for generating a haptic effect through an actuator. The method includes passing the digital audio signal on through dynamic processor(s), adjusting the dynamic range of the digital audio signal, extracting the signal envelope of the audio data, synthesising low-frequency signals from the extracted signal envelope, and enhancing the low-frequency content using a resonator. The haptic output is generated by mixing the digital audio signal with outputs from the different modules of the haptic processing system. An analytics module monitors, controls and adjusts the processing of the digital audio signal at the noise gate module, the compressor module and the envelope module to enhance the haptic output.

## Claims

### Claim 1

A haptic processing method for producing haptic output data from an input audio signal, the haptic processing method comprising:
receiving the input audio signal in a digital signal processing chain and an analytics processor;
processing the input audio signal simultaneously in the digital signal processor and the analytics processor, wherein the digital signal processor comprises at least one dynamic processor, and wherein the analytics processor independently processes the input audio signal to set a gain and a threshold in the at least one dynamic processor to control the processing of the input audio signal in the digital signal processing chain;
mixing the received audio signal in a mixer; and
producing the haptic output in one or more actuators.

### Claim 2 (depends on 1)

The haptic processing method of claim 1, wherein the digital signal processing chain further comprises a dry/wet processor, a noise gate, a compressor, an envelope follower, a frequency oscillator, a resonance processor, a detector, and a preset.

### Claim 3 (depends on 2)

The haptic processing method of claim 2, wherein the preset is configured to control the input audio signal processing in at least one of the mixer, the low pass filter, the limiter, or the bell filter.

### Claim 4 (depends on 2)

The haptic processing method of claim 2, wherein the digital signal processing chain further includes a dual filter, wherein the dual filter receives at least one control parameter from the analytics processor and the preset.

### Claim 5 (depends on 4)

The haptic processing method of claim 4, wherein the preset provides the at least one control parameter to the dry/wet processor.

### Claim 6 (depends on 2)

The haptic processing method of claim 2, wherein the dry/wet processor receives the input audio signal and the output from the at least one dynamic processor, wherein the ratio of dry signal to wet signal is controlled by a user interface.

### Claim 7 (depends on 2)

The haptic processing method of claim 2, wherein the resonance processor performs at least one or more functions comprising: (a) amplify suppressed low frequencies; (b) excitate physical models of string, membrane, plate, or other musical instrument models based on the input audio signal; or (c) add low frequencies to the output signal.

### Claim 8 (depends on 2)

The haptic processing method of claim 2, wherein the mixer is configured to mix output signal from the dry/wet processor, the frequency oscillator, and the resonance processor.

### Claim 9 (depends on 1)

The haptic processing method of claim 1, wherein the analytics processor comprises a peak analyzer, a transient detector, an RMS generator, a sidechain processor, a detector and threshold processor, a predictive and intelligence processor, a gain controller, and a frequency gain controller.

### Claim 10 (depends on 1)

The haptic processing method of claim 1, wherein the analytics processor performs the processing of the input audio signal independently of the digital signal processor and passes parameters by implementing function calls.

### Claim 11 (depends on 1)

The haptic processing method of claim 1, wherein the analytics processor provides control parameters to the envelope follower.

### Claim 12 (depends on 9)

The haptic processing method of claim 9, wherein the parameters passed to the at least one dynamic processor is determined by performing steps:
analyzing the input audio signal to calculate a peak amplitude value and a root mean square value for a fixed time period with a varying window size;
storing the calculated peak amplitude value and the calculated root mean square value to set a threshold level and a gain of the at least one dynamic processor by:
(a) determining the difference between the peak amplitude value and a pre-determined gain,
(b) determining the sum of the root mean square value and the pre-determined gain, wherein the pre-determined gain subtracted from the peak amplitude values and the pre-determined gain added to the root mean square value is a constant, and
(c) determining the minimum of the two values determined at step (a) and step (b); and
applying the minimum values to dynamically modulate the threshold level or the gain of the at least one dynamic processor.

### Claim 13 (depends on 1)

The haptic processing method of claim 1, wherein the input audio signal is processed in real time or non-real time.

### Claim 14 (depends on 2)

The haptic processing method of claim 2, wherein the dry/wet processor provides a preset control accessible by a user to control the mixing of a dry signal and a wet signal.

### Claim 15 (depends on 1)

The haptic processing method of claim 1, wherein the at least one dynamic processor is a compressor.

### Claim 16 (depends on 1)

The haptic processing method of claim 1, wherein the at least one dynamic processor is a noise gate.

### Claim 17 (depends on 1)

The haptic processing method of claim 1, wherein the analytics processor sets at least one parameter in the digital signal processing chain to control the audio signal processing.

### Claim 18 (depends on 2)

The haptic processing method of claim 2, wherein the detector implements a power saving mode by detecting input audio signal and the output signal.

### Claim 19 (depends on 2)

The haptic processing method of claim 2, wherein the power is switched to battery saving mode in absence of input audio signal and the output signal.

### Claim 20 (depends on 1)

The haptic processing method of claim 1, wherein the input audio signal feed to the analytics processor is processed using predictive algorithms to predict at least the threshold value and gain.

### Claim 21

A haptic processing system for producing a haptic output data from an input audio signal, the haptic processing system comprising:
a digital signal processing chain and an analytics processor configured to simultaneously receive the input audio signal, wherein the digital signal processor comprise at least one dynamic processor, and wherein the analytics processor separately processes the input audio signal to determine at least one parameter to set at least a gain or a threshold in the at least one dynamic processor;
a mixer configured to mix signals; and
one or more actuators to produce the haptic output data.

### Claim 22 (depends on 21)

The haptic processing system of claim 21, further comprising an envelope follower, wherein the analytics processor provides control parameters to the envelope follower.

### Claim 23 (depends on 21)

The haptic processing system of claim 21, wherein the parameters passed to the at least one dynamic processor is determined by performing the steps of:
analyzing the input audio signal to calculate a peak amplitude value and a root mean square value for a fixed time period with a varying window size;
storing the calculated peak amplitude value and the calculated root mean square value to set a threshold level and a gain of the at least one dynamic processor by:
(a) determining the difference between the peak amplitude value and a pre-determined gain,
(b) determining the sum of the root mean square value and the pre-determined gain, wherein the pre-determined gain subtracted from the peak amplitude values and the pre-determined gain added to the root mean square value is a constant, and
(c) determining the minimum of the two values determined at step (a) and step (b); and
applying the minimum values to dynamically modulate the threshold level or the gain of the at least one dynamic processor.

### Claim 24 (depends on 21)

The haptic processing system of claim 21, wherein the digital signal processing chain further comprises a dry/wet processor, a noise gate, a compressor, an envelope follower, a frequency oscillator, a resonance processor, and a preset.

### Claim 25 (depends on 24)

The haptic processing system of claim 24, wherein the preset is configured to control at least one of the mixer, a low pass filter, a limiter, or a bell filter.

### Claim 26 (depends on 21)

The haptic processing system of claim 21, wherein the digital signal processing chain further includes a dual filter, wherein the dual filter receives control parameters from the analytics processor and the preset.

### Claim 27 (depends on 24)

The haptic processing system of claim 24, wherein the preset provides control parameters to the dry/wet processor.

### Claim 28 (depends on 24)

The haptic processing system of claim 24, wherein the dry/wet processor receives the input audio signal and the output from the at least one dynamic processor, wherein the ratio of dry signal to wet signal is controlled by a user interface.

### Claim 29 (depends on 24)

The haptic processing system of claim 24, wherein the resonance processor performs at least one or more functions comprising: (a) amplify suppressed low frequencies; (b) excitate physical models of string, membrane, plate or other musical instrument models based on the input audio signal; or (c) add low frequencies to the output signal.

### Claim 30 (depends on 21)

The haptic processing system of claim 21, wherein the mixer is configured to mix output signals from the dry/wet processor, the frequency oscillator, and the resonance processor.

### Claim 31 (depends on 21)

The haptic processing system of claim 21, wherein the analytics processor comprises a peak analyzer, a transient detector, an RMS generator, a sidechain processor, a detector and threshold processor, a predictive and intelligence processor, a gain controller, and a frequency gain controller.

### Claim 32

A computer program product, the computer program product comprising a non-transitory computer-readable storage medium and a processor, the non-transitory computer readable storage medium storing executable code, when executed by at least one processor causes the processor to perform the steps of:
receive an input audio signal simultaneously in a digital signal processing chain and an analytics processor, wherein the digital signal processor comprises at least one dynamic processor, and wherein the analytics processor separately processes the input audio signal to determine at least one parameter to set at least a gain in the at least one dynamic processor to control the processing of the haptic data;
mix the received audio signal to in a mixer, which receives at least one control parameter from the analytics processor; and
produce a haptic output using at least one actuator.

### Claim 33 (depends on 32)

The computer program product of claim 32, wherein the analytics processor provides control parameters to an envelope follower.

### Claim 34 (depends on 32)

The computer program product of claim 32, wherein the analytics processor comprises a peak analyzer, a transient detector, an RMS generator, a sidechain processor, a detector and threshold processor, a predictive and intelligence processor, a gain controller, and a frequency gain controller.

### Claim 35 (depends on 32)

The computer program product of claim 32, wherein the parameters passed to the at least one dynamic processor is determined by performing the steps of:
analyzing the input audio signal to calculate a peak amplitude value and a root mean square value for a fixed time period with a varying window size;
storing the calculated peak amplitude value and the calculated root mean square value to set a threshold level and a gain of the at least one dynamic processor by:
(a) determining the difference between the peak amplitude value and a pre-determined gain,
(b) determining the sum of the root mean square value and the pre-determined gain, wherein the pre-determined gain subtracted from the peak amplitude values and the pre-determined gain added to the root mean square value is a constant, and
(c) determining the minimum of the two values determined at step (a) and step (b); and
applying the minimum values to dynamically modulate the threshold level or the gain of the at least one dynamic processor.

### Claim 36 (depends on 34)

The computer program product of claim 34, wherein predictive and intelligence processor implements machine learning algorithms to perform the determination of at least one parameter to be passed to the at least one dynamic processor.

### Claim 37 (depends on 32)

The computer program product of claim 32, wherein the digital signal processing chain further comprises a dry/wet processor, a noise gate, a compressor, an envelope follower, a frequency oscillator, a resonance processor, and a preset.

### Claim 38 (depends on 37)

The computer program product of claim 37, wherein the preset is configured to control at least one of the mixer, a low pass filter, a limiter, or a bell filter.

### Claim 39 (depends on 32)

The computer program product of claim 32, wherein the digital signal processing chain further includes a dual filter, wherein the dual filter receives control parameters from the analytics processor and the preset.

### Claim 40 (depends on 32)

The computer program product of claim 32, wherein the preset provides control parameters to the dry/wet processor.

### Claim 41 (depends on 37)

The computer program product of claim 37, wherein the dry/wet processor receives the input audio signal and the output from the at least one dynamic processor, wherein the ratio of dry signal to wet signal is controlled by a user interface.

### Claim 42 (depends on 37)

The computer program product of claim 37, wherein the resonance processor performs at least one or more functions comprising: (a) amplify suppressed low frequencies; (b) excitate physical models of string, membrane, plate or other musical instrument models based on the input audio signal; or (c) add low frequencies to the output signal.

### Claim 43 (depends on 32)

The computer program product of claim 32, wherein the mixer is configured to mix output signals from the dry/wet processor, the frequency oscillator, and the resonance processor.
