Back to results
US 10,313,822 B2Granted
Image and audio based characterization of a human auditory system for personalized audio reproduction
AssigneeEmbodyVR, Inc.
InventorKapil Jain
Life of this patent
Estimating the term…
Abstract
An image sensor in a first earcup captures an image of a pinna. First sound is output by a transducer in a second earcup located at the pinna and respective second sound is detected by each of one or more microphones in the second earcup located at the pinna. Based on the captured image and the respective second audio sound from each of the one or more microphones, a non-linear transfer function is determined which characterizes how sound is transformed by the pinna. A signal is generated indicative of one or more audio cues for spatializing third sound based on the determined non-linear transfer function.
Claims22
A method comprising:
- capturing, by an image sensor in a first earcup, a first image of a pinna;
- outputting, by a first transducer in a second earcup located at the pinna, first sound;
- detecting, by each of one or more microphones in the second earcup located at the pinna, respective second sound based on the output of the first sound;
- based on the captured first image and the respective second sound from each of the one or more microphones, determining a first non-linear transfer function which characterizes how sound is transformed by the pinna;
- generating a signal indicative of one or more audio cues for spatializing third sound based on the first non-linear transfer function;
- and
- outputting, by a second transducer in the first or second earcup located on the pinna, the signal indicative of one or more audio cues to facilitate spatial localization of the third sound via the pinna;
- wherein determining the first non-linear transfer function comprises:
- comparing characteristics of features of the captured first image to a second image associated with a second non-linear transfer function;
- defining a first correlation score based on the comparison of the characteristics of the features of the first image to the second image;
- comparing the detected respective second sound of each of the one or more microphones to a given one or more frequency responses of the second non-linear transfer function;
- defining a second correlation score based on the comparison of the detected respective second sound to the given one or more frequency responses of the second non-linear transfer function;
- and
- based on the first correlation score, the second correlation score, and the second non-linear transfer function, determining the first non-linear transfer function.
11 more elements
One or more non-transitory computer readable media comprising program code stored in memory and executable by a processor, the program code to:
- capture, by an image sensor, a first image of a pinna;
- output, by a first transducer in an earcup located at the pinna, first sound;
- detect, by each of one or more microphones in the earcup located at the pinna, respective second sound based on the output of the first sound;
- based on the captured first image and the respective second sound from each of the one or more microphones, determine a first non-linear transfer function which characterizes how sound is transformed by the pinna;
- generate a signal indicative of one or more audio cues for spatializing third sound based on the determined first non-linear transfer function;
- and
- output, by a second transducer in the earcup located on the pinna, the signal indicative of one or more audio cues to facilitate spatial localization of the third sound via the pinna;
- wherein determining the first non-linear transfer function comprises:
- comparing characteristics of features of the captured first image to a second image associated with a second non-linear transfer function;
- defining a first correlation score based on the comparison of the characteristics of the features of the captured first image to the second image;
- comparing the detected respective second sound of each of the one or more microphones to a given one or more frequency responses of the second non-linear transfer function;
- defining a second correlation score based on the comparison of the detected respective second sound to the given one or more frequency responses of the second non-linear transfer function;
- and
- based on the first correlation score, the second correlation score, and the second non-linear transfer function, determining the first non-linear transfer function.
11 more elements
A system comprising:
- an earcup;
- and
- computer instructions stored in memory and executable by a processor to perform the functions of:
- receiving, from an image sensor, a first image of a pinna;
- outputting, by a first transducer in the earcup located at the pinna, first sound;
- receiving, from each of one or more microphones in the earcup located at the pinna, respective second sound based on the output of the first sound;
- based on the first image and the respective second sound from each of the one or more microphones, determining a first non-linear transfer function which characterizes how sound is transformed by the pinna;
- generating a signal indicative of one or more audio cues for spatializing third sound based on the determined first non-linear transfer function;
- and
- outputting, by a second transducer in the earcup located on the pinna, the signal indicative of one or more audio cues to facilitate spatial localization of the third sound via the pinna;
- wherein determining the first non-linear transfer function comprises:
- comparing characteristics of features of the first image to a second image associated with a second non-linear transfer function;
- defining a first correlation score based on the comparison of the characteristics of the features of the first image to the second image;
- comparing the detected respective second sound of each of the one or more microphones to a given one or more frequency responses of the second non-linear transfer function;
- defining a second correlation score based on the comparison of the detected respective second sound to the given one or more frequency responses of the second non-linear transfer function;
- and
- based on the first correlation score, the second correlation score, and the second non-linear transfer function, determining the first non-linear transfer function.
14 more elements
Dependencies are inferred from the claim text.