# US 2021/0297780 A1 · Audio signal processor and generator

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0297780 A1
- Document ID: US20210297780A1
- Status: Published application
- Kind: A1
- Application number: 16/332,680
- Filed: 2017-09-13
- Published: 2021-09-23
- Earliest claimed: 2016-09-13
- Assignees: VisiSonics Corporation
- Applicants: VisiSonics Corporation
- Inventors: Dmitry N. Zotkin; Nail A. Gumerov; Ramani Duraiswami
- Web page: https://patentgrep.com/patent/US20210297780A1

## CPC

- H04R 1/406: {microphones}
- H04R 5/027: Spatial or constructional arrangements of microphones, e.g. in dummy heads
- H04R 1/222: {for microphones (H04R1/24, H04R1/26 take precedence)}
- H04R 1/326: {for microphones (H04R1/34 and H04R1/40 take precedence)}
- H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)}
- H04S 3/00: Systems employing more than two channels, e.g. quadraphonic (H04S5/00, H04S7/00 take precedence)
- H04R 2201/401: 2D or 3D arrays of transducers
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- H04S 2420/11: Application of ambisonics in stereophonic audio systems
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)
- H04S: STEREOPHONIC SYSTEMS

## Abstract

A spatial-audio recording system includes a spatial-audio recording device including a plurality of microphones, and a computing device. The computing device is configured to determine a plane-wave transfer function for the spatial-audio recording device based on a physical shape of the spatial-audio recording device and to expand the plane-wave transfer function to generate a spherical-harmonics transfer function corresponding to the plane-wave transfer function. The computing device is further configured to retrieve a plurality of signals captured by the microphones, determine spherical-harmonics coefficients for an audio signal based on the plurality of captured signals and the spherical-harmonics transfer function, and generate the audio signal based on the determined spherical-harmonics coefficients.

## Claims

### Claim 1

A spatial-audio recording system, comprising:
a spatial-audio recording device comprising a plurality of microphones; and
a computing device configured to:
determine a plane-wave transfer function for the spatial-audio recording device based on a physical shape of the spatial-audio recording device;
expand the plane-wave transfer function to generate a spherical-harmonics transfer function corresponding to the plane-wave transfer function;
retrieve a plurality of signals captured by the microphones;
determine spherical-harmonics coefficients for an audio signal based on the plurality of captured signals and the spherical-harmonics transfer function; and
generate the audio signal based on the determined spherical-harmonics coefficients.

### Claim 2 (depends on 1)

The system of claim 1, wherein:
the computing device is further configured to generate the audio signal based on the determined spherical-harmonics coefficients by performing processes that include converting the spherical-harmonics coefficients to ambisonics coefficients.

### Claim 3 (depends on 1)

The system of claim 1, wherein:
the computing device is configured to determine the spherical-harmonics coefficients by performing processes that include setting a measured audio field based on the plurality of signals equal to an aggregation of a signature function comprising the spherical-harmonics coefficients and the spherical-harmonics transfer function.

### Claim 4 (depends on 3)

The system of claim 3, wherein:
the computing device is further configured to determine the signature function comprising spherical-harmonics coefficients by expanding a signature function that describes a plane wave strength as a function of direction over a unit sphere into the signature function comprising spherical-harmonics coefficients.

### Claim 5 (depends on 1)

The system of claim 1, wherein:
the computing device is configured to determine the plane-wave transfer function for the spatial-audio recording device by performing operations that comprise implementing a fast multipole-accelerated boundary element method, or based on previous measurements of the spatial-audio recording device.

### Claim 6 (depends on 1)

The system of claim 1, wherein:
the plurality of microphones are distributed over a non-spherical surface of the spatial-audio recording device.

### Claim 7 (depends on 1)

The system of claim 1, wherein:
the computing device is configured to determine the spherical-harmonics coefficients based on the plurality of captured signals and the spherical-harmonics transfer function by performing operations that comprise implementing a least-squares technique.

### Claim 8 (depends on 1)

The system of claim 1, wherein:
the computing device is configured to determine a frequency-space transform of one or more of the captured signals.

### Claim 9 (depends on 1)

The system of claim 1, wherein:
the computing device is configured to generate the audio signal corresponding to an audio field generated by one or more external sources and substantially undisturbed by the spatial-audio recording device.

### Claim 10 (depends on 1)

The system of claim 1, wherein the spatial-audio recording device is a panoramic camera.

### Claim 11 (depends on 1)

The system of claim 1, wherein the spatial-audio recording device is a wearable device.

### Claim 12

A method of generating an audio signal, comprising:
determining a plane-wave transfer function for a spatial-audio recording device comprising a plurality of microphones based on a physical shape of the spatial-audio recording device;
expanding the plane-wave transfer function to generate a spherical-harmonics transfer function corresponding to the plane-wave transfer function;
retrieving a plurality of signals captured by the microphones;
determining spherical-harmonics coefficients based on the plurality of captured signals and the spherical-harmonics transfer function; and
generating an audio signal based on the determined spherical-harmonics coefficients.

### Claim 13 (depends on 12)

The method of claim 12, wherein:
the generating the audio signal based on the determined spherical-harmonics coefficients comprises converting the spherical-harmonics coefficients to ambisonics coefficients.

### Claim 14 (depends on 12)

The method of claim 12, wherein:
the determining the plane-wave transfer function for the spatial-audio recording device comprises implementing a fast multipole-accelerated boundary element method, or based on previous measurements of the spatial-audio recording device.

### Claim 15 (depends on 12)

The method of claim 12, wherein:
determining the spherical-harmonics coefficients comprises setting a measured audio field based on the plurality of signals equal to an aggregation of a signature function comprising the spherical-harmonics coefficients and the spherical-harmonics transfer function.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
determining the signature function comprising spherical-harmonics coefficients by expanding a signature function that describes a plane-wave strength as a function of direction over a unit sphere into the signature function comprising spherical-harmonics coefficients.

### Claim 17 (depends on 12)

The method of claim 12, wherein:
the spherical-harmonics transfer function corresponding to the plane-wave transfer function satisfies the equation:
H
⁡
(
k
,
s
,
r
j
)
=
∑
n
=
0
p
-
1
⁢
∑
m
=
-
n
n
⁢
H
n
m
⁡
(
k
,
r
j
)
⁢
Y
n
m
⁡
(
s
)
,
where H(k,s,rj) is the plane-wave transfer function, Hnm(k, rj) constitute the spherical-harmonics transfer function, Ynm(s) are orthonormal complex spherical harmonics, k is a wavenumber of the captured signals, s is a vector direction from which the captured signals are arriving, n is a degree of a spherical mode, m is an order of a spherical mode, and p is a predetermined truncation number.

### Claim 18 (depends on 12)

The method of claim 12, wherein:
the signature function comprising spherical-harmonics coefficients is expressed in the form:
μ
⁡
(
k
,
s
)
=
∑
n
=
0
p
-
1
⁢
∑
m
=
-
n
n
⁢
C
n
m
⁡
(
k
)
⁢
Y
n
m
⁡
(
s
)
,
where μ(k,s) is the signature function, Cnm(k) constitute the spherical-harmonics coefficients, Ynm(s) are orthonormal complex spherical harmonics, k is a wavenumber of the captured signals, s is a vector direction from which the captured signals are arriving, n is a degree of a spherical mode, m is an order of a spherical mode, and p is a predetermined truncation number.

### Claim 19 (depends on 12)

The method of claim 12, wherein the spatial-audio recording device is a panoramic camera.

### Claim 20 (depends on 12)

The method of claim 12, wherein the spatial-audio recording device is a wearable device.

### Claim 21

A spatial-audio recording device comprising:
a plurality of microphones; and
a computing device configured to:
determine a plane-wave transfer function for the spatial-audio recording device based on a physical shape of the spatial-audio recording device;
expand the plane-wave transfer function to generate a spherical-harmonics transfer function corresponding to the plane-wave transfer function;
retrieve a plurality of signals captured by the microphones;
determine spherical-harmonics coefficients based on the plurality of captured signals and the spherical-harmonics transfer function;
convert the spherical-harmonics coefficients to ambisonics coefficients; and
generate an audio signal based on the ambisonics coefficients.

### Claim 22 (depends on 21)

The spatial-audio recording device of claim 21, wherein:
the computing device is configured to determine the plane-wave transfer function for the spatial-audio recording device based on a mesh representation of the physical shape of the spatial-audio recording device.

### Claim 23 (depends on 21)

The spatial-audio recording device of claim 21, wherein:
the audio signal is an augmented audio signal.

### Claim 24 (depends on 21)

The spatial-audio recording device of claim 21, wherein:
the microphones are distributed over a non-spherical surface of the spatial-audio recording device.

### Claim 25 (depends on 21)

The spatial-audio recording device of claim 21, wherein the spatial-audio recording device is a panoramic camera.

### Claim 26 (depends on 21)

The spatial-audio recording device of claim 21, wherein the spatial-audio recording device is a wearable device.
