Audio-triggered augmented reality eyewear device
AssigneeSnap Inc.
InventorDavid Meisenholder
Life of this patent
Estimating the term…
Abstract
Systems, methods, and non-transitory computer readable media for augmenting scenes viewed thorough displays of an eyewear devices with audio-related image information. Scenes may be augmented by capturing, via a camera of the eyewear device, initial images of a scene, identifying features within the initial images; receiving audio-related image information (e.g., lyrics and/or images), registering the audio-related image information to the identified features, creating audio-based visual overlays including the audio-related image information registered to the identified features, and displaying the audio-based visual overlays over the scene.
Claims21
A system comprising:
- an eyewear device, including:
- a frame;
- a temple extending from a lateral side of the frame;
- a see-through optical assembly supported by the frame, the see-through optical assembly having a viewing area for viewing a scene and configured to present audio-based visual overlays to a user over the scene in the viewing area;
- and
- a camera connected to the frame or the temple to capture initial images of the scene;
- a processor connected to the camera;
- a memory accessible to the processor;
- and
- programming in the memory, wherein execution of the programming by the processor configures the system to perform functions, including functions to:
- capture, via the camera, the initial images of the scene;
- identify features within the initial images;
- receive an image that is at least one of related to or synchronized to a prerecorded or live audio signal;
- match display parameters identifying at least one of a feature or position of the image that is at least one of related to or synchronized to a prerecorded or live audio signal to one or more of the identified features within the initial images;
- create the audio-based visual overlays, the audio-based visual overlays including the image that is at least one of related to or synchronized to the prerecorded or live audio signal that has been matched to the one or more of the identified features within the initial images;
- and
- display, via the see-through optical assembly, the audio-based visual overlays to the user over the scene in the viewing area.
A method comprising:
- capturing, via a camera of an eyewear device, initial images of a scene in a viewing area;
- identifying features within the initial images;
- receiving an image that is at least one of related to or synchronized to a prerecorded or live audio signal;
- matching display parameters identifying at least one of a feature or position of the image that is at least one of related to or synchronized to a prerecorded or live audio signal to one or more of the identified features within the initial images;
- creating audio-based visual overlays, the audio-based visual overlays including the image that is at least one of related to or synchronized to the prerecorded or live audio signal that has been matched to the one or more of the identified features within the initial images;
- and
- displaying, via a see-through optical assembly of the eyewear device, the audio-based visual overlays to the user over the scene in the viewing area.
A non-transitory computer-readable medium storing program code which, when executed, is operative to cause an electronic processor to perform the steps of:
- capturing, via a camera of an eyewear device, initial images of a scene in a viewing area;
- identifying features within the initial images;
- receiving an image that is at least one of related to or synchronized to a prerecorded or live audio signal;
- matching display parameters identifying at least one of a feature or position of the image that is at least one of related to or synchronized to a prerecorded or live audio signal to one or more of the identified features within the initial images;
- creating audio-based visual overlays, the audio-based visual overlays including the image that is at least one of related to or synchronized to the prerecorded or live audio signal that has been matched to the one or more of the identified features within the initial images;
- and
- displaying, via a see-through optical assembly of the eyewear device, the audio-based visual overlays to the user over the scene in the viewing area.
Dependencies are inferred from the claim text.