Back to results
US 2023/0148241 A1Published application
Gaze-guided audio
AssigneeMeta Platforms Technologies, LLC
InventorsSebastian Sztuk, Salvael Ortega Estrada
Life of this patent
Estimating the term…
Abstract
A gaze direction of a user is determined by an eye-tracking system of a head mounted device. Audio data generated by at least one microphone is captured. Gaze-guided audio is generated from the audio data based on the gaze direction of the user.
Claims17
A head mounted device comprising:
- an eye-tracking system including one or more sensors configured to determine a gaze direction of an eye in an eyebox region of the head mounted device;
- a first microphone configured to record first audio data from first sound waves, wherein the first microphone is oriented to receive the first sound waves originating from a first audio zone of an external environment of the head mounted device;
- a second microphone configured to record second audio data from second sound waves, wherein the second microphone is oriented to receive the second sound waves originating from a second audio zone of an external environment of the head mounted device;
- a camera to capture a video file;
- a memory;
- and
- processing logic configured to:
- receive the gaze direction from the eye-tracking system;
- and
- select a primary microphone between the first microphone and the second microphone to generate gaze-guided audio, wherein the primary microphone is selected based on the gaze direction received from the eye-tracking system;
- and
- store the gaze-guided audio to the memory as an audio portion of the video file that was captured by the camera of the head mounted device contemporaneously with the gaze-guided audio, wherein the audio portion of the video file is received from the selected primary microphone.
9 more elements
2-5. (canceled)
A head mounted device comprising:
- an eye-tracking system including one or more sensors configured to determine a gaze direction of an eye in an eyebox region of the head mounted device;
- a plurality of microphones, wherein the microphones in the plurality of microphones are directionally oriented to capture audio data from a plurality of different audio zones of an external environment of the head mounted device;
- a camera for capturing a video file;
- memory;
- and
- processing logic configured to:
- receive the gaze direction from the eye-tracking system;
- and
- generate gaze-guided audio from the audio data based on the gaze direction;
- and
- store the gaze-guided audio to the memory as an audio portion of the video file that was captured by the camera of the head mounted device contemporaneously with the audio data, wherein the audio portion of the video file includes selected audio data from a portion of the plurality of microphones selected based on a direction a user was gazing while the camera was recording the video file.
8 more elements
(canceled)
(canceled)
(canceled)
(canceled)
A method of operating a head mounted device, the method comprising:
- determining a gaze direction of an eye of a user of the head mounted device, wherein an eye-tracking system of the head mounted device determines the gaze direction of the user;
- capturing a video file with a camera contemporaneously with capturing audio data from a plurality of microphones of the head mounted device, wherein the microphones in the plurality are directionally oriented to capture the audio data from a plurality of different audio zones of an external environment of the head mounted device, wherein the camera is included in the head mounted device;
- generating gaze-guided audio from the audio data based on the gaze direction of the user, wherein generating the gaze-guided audio includes selectively amplifying audio from the microphones in the plurality of microphones that are oriented to receive sound waves from audio zones that correspond with a gaze vector representative of the gaze direction of the user;
- and
- storing the gaze-guided audio to a memory as an audio portion of the video file.
2 more elements
(canceled)
(canceled)
(canceled)
Dependencies are inferred from the claim text.