# US 2019/0379993 A1 · System and method to capture image of pinna and characterize human auditory anatomy using image of pinna

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0379993 A1
- Document ID: US20190379993A1
- Status: Published application
- Kind: A1
- Application number: 16/542,930
- Filed: 2019-08-16
- Published: 2019-12-12
- Earliest claimed: 2016-11-13
- Assignees: EmbodyVR, Inc.
- Applicants: EmbodyVR, Inc.
- Inventors: Kapil Jain
- Web page: https://patentgrep.com/patent/US20190379993A1

## CPC

- A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles}
- A61B 5/12: Audiometering
- A61B 5/121: {evaluating hearing capacity}
- A61B 5/6815: {Ear}
- A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems}
- G01B 7/14: for measuring distance or clearance between spaced objects or spaced apertures (G01B7/30 takes precedence)
- G06T 1/0007: {Image acquisition}
- G06T 7/73: using feature-based methods
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- H04R 1/005: {using digitally weighted transducing elements}
- H04R 1/1008: {Earpieces of the supra-aural or circum-aural type}
- H04R 1/1058: {Manufacture or assembly}
- H04R 1/22: for obtaining desired frequency characteristic only {(circuit for combining transducers having different responses H04R3/00; for hearing aids H04R25/407)}
- H04R 1/32: for obtaining desired directional characteristic only {(specially adapted for hearing aids H04R25/40)}
- H04R 1/323: {for loudspeakers (H04R1/34 and H04R1/40 take precedence)}
- H04R 3/04: for correcting frequency response
- H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)}
- H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- H04S 7/301: {Automatic calibration of stereophonic sound system, e.g. with test microphone}
- H04S 7/304: {For headphones}
- H04S 7/306: {For headphones}
- A61B 5/4005: {for evaluating the sensory system (A61B5/12, A61B5/4058 take precedence)}
- A61B 5/7415: {Sound rendering of measured values, e.g. by pitch or volume variation (A61B5/741 takes precedence)}
- A61B 6/5217: {extracting a diagnostic or physiological parameter from medical diagnostic data}
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- H04R 1/1016: {Earpieces of the intra-aural type}
- H04R 5/033: Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)}
- H04R 11/02: Loudspeakers
- H04R 2201/029: Manufacturing aspects of enclosures transducers
- H04R 2225/77: Design aspects, e.g. CAD, of hearing aid tips, moulds or housings
- H04S 7/30: {Control circuits for electronic adaptation of the sound field}
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)
- H04S: STEREOPHONIC SYSTEMS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

An image of a pinna is captured. Based on the image of the pinna, a non-linear transfer function is determined which characterizes how sound is transformed at the pinna. A signal is output indicative of one or more audio cues to facilitate spatial localization of sound via the pinna, where the one or more audio cues is based on the non-linear transfer function.

## Claims

### Claim 1

A method comprising:
capturing an image of a reference pinna;
based on the image of the reference pinna, determining a non-linear transfer function which characterizes how sound is transformed at the reference pinna, wherein determining the non-linear transfer function comprises inputting the image into a neural network model which outputs the non-linear transfer function; and
outputting, by a transducer, a signal indicative of one or more audio cues to facilitate spatial localization of sound via the reference pinna, wherein the one or more audio cues is based on the non-linear transfer function.

### Claim 2 (depends on 1)

The method of claim 1, wherein capturing the image of the reference pinna comprises determining, by a proximity sensor, a distance between a mobile device and the reference pinna; and capturing, by an image sensor of the mobile device, the image of the reference pinna when a proximity sensor indicates that the mobile device is at a given linear distance from the reference pinna.

### Claim 3 (depends on 1)

The method of claim 1, wherein capturing the image of the reference pinna comprises positioning a mobile device having a camera proximate to the reference pinna.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining the non-linear transfer function comprises extracting characteristics of one or more features of the reference pinna based on the image of the reference pinna.

### Claim 5 (depends on 1)

The method of claim 1, wherein capturing the image of the reference pinna comprises capturing, via a mobile device, a plurality of images of the reference pinna at a plurality of orientations, wherein the plurality of images of the reference pinna at the plurality of orientations indicates at least a relative distance between features of the reference pinna and a depth of features of the reference pinna.

### Claim 6 (depends on 5)

The method of claim 5, wherein capturing the image of the reference pinna comprises combining the plurality of images of the reference pinna to form a multi-dimensional image of the reference pinna; and wherein determining the non-linear transfer function comprises inputting the multi-dimensional image into the neural network model.

### Claim 7 (depends on 1)

The method of claim 1, wherein the image of the reference pinna is associated with a first individual and the neural network model is fit to a plurality of HRTFs and associated images of pinna associated with second individuals other than the first individual.

### Claim 8

One or more non-transitory machine-readable media comprising computer instructions stored in memory and executable by a processor, the computer instructions to:
capture an image of a reference pinna;
based on the image of the reference pinna, determine a non-linear transfer function which characterizes how sound is transformed at the reference pinna, wherein determining the non-linear transfer function comprises inputting the image into a neural network model which outputs the non-linear transfer function; and
output, by a transducer, a signal indicative of one or more audio cues to facilitate spatial localization of sound via the reference pinna, wherein the one or more audio cues is based on the non-linear transfer function.

### Claim 9 (depends on 8)

The one or more non-transitory machine-readable media of claim 8, wherein the computer instructions to capture the image of the reference pinna comprises computer instructions to determine, by a proximity sensor, a distance between a mobile device and the reference pinna; and capture, by an image sensor of the mobile device, the image of the reference pinna when a proximity sensor indicates that the mobile device is at a given linear distance from the reference pinna.

### Claim 10 (depends on 8)

The one or more non-transitory machine-readable media of claim 8, wherein the computer instructions to capture the image of the reference pinna comprises receiving the image of the reference pinna from a mobile device having a camera positioned proximate to the reference pinna.

### Claim 11 (depends on 8)

The one or more non-transitory machine-readable media of claim 8, wherein the computer instructions to determine the non-linear transfer function comprises computer instructions to extract characteristics of one or more features of the reference pinna based on the image of the reference pinna.

### Claim 12 (depends on 8)

The one or more non-transitory machine-readable media of claim 8, wherein the computer instructions to capture the image of the reference pinna comprises computer instructions to capture, via a mobile device, a plurality of images of the reference pinna at a plurality of orientations, wherein the plurality of images of the reference pinna at the plurality of orientations indicates at least a relative distance between features of the reference pinna and a depth of features of the reference pinna.

### Claim 13 (depends on 12)

The one or more non-transitory machine-readable media of claim 12, wherein the computer instructions to capture the image of the reference pinna comprises computer instructions to combine the plurality of images of the reference pinna to form a multi-dimensional image of the reference pinna; and wherein the computer instructions to determine the non-linear transfer function comprises computer instructions to input the multi-dimensional image into the neural network model.

### Claim 14 (depends on 8)

The one or more non-transitory machine-readable media of claim 8, wherein the image of the reference pinna is associated with a first individual and the neural network model is fit to a plurality of HRTFs and associated images of pinna associated with second individuals other than the first individual.

### Claim 15

A system comprising:
a mobile device having a camera;
computer instructions stored in memory and executable by a processor to perform the functions of:
capturing, by the camera of the mobile device, an image of a reference pinna;
based on the image of the reference pinna, determining a non-linear transfer function which characterizes how sound is transformed at the reference pinna, wherein determining the non-linear transfer function comprises inputting the image into a neural network model which outputs the non-linear transfer function; and
outputting a signal indicative of one or more audio cues to facilitate spatial localization of sound via the reference pinna, wherein the one or more audio cues is based on the non-linear transfer function.

### Claim 16 (depends on 15)

The system of claim 15, wherein the computer instructions to capture the image of the reference pinna comprises receiving the image of the reference pinna from the mobile device having the camera positioned proximate to the reference pinna.

### Claim 17 (depends on 15)

The system of claim 15, wherein the computer instructions to determine the non-linear transfer function comprises computer instructions to extract characteristics of one or more features of the reference pinna based on the image of the reference pinna.

### Claim 18 (depends on 15)

The system of claim 15, wherein the computer instructions to capture the image of the reference pinna comprises computer instructions to capture, via the mobile device, a plurality of images of the reference pinna at a plurality of orientations, wherein the plurality of images of the reference pinna at the plurality of orientations indicates at least a relative distance between features of the reference pinna and a depth of features of the reference pinna.

### Claim 19 (depends on 18)

The system of claim 18, wherein the computer instructions to capture the image of the reference pinna comprises computer instructions to combine the plurality of images of the reference pinna to form a multi-dimensional image of the reference pinna; and wherein the computer instructions to determine the non-linear transfer function comprises computer instructions to input the multi-dimensional image into the neural network model.

### Claim 20 (depends on 15)

The system of claim 15, wherein the image of the reference pinna is associated with a first individual and the neural network model is fit to a plurality of HRTFs and associated images of pinna associated with second individuals other than the first individual.