Back to results
US 12,761,742 B2Granted
Controller movement tracking with light emitters
AssigneeMentor Acquisition One, LLC
InventorsRobert Michael Lohse, John D. Haddick
Life of this patent
Estimating the term…
Abstract
A head-worn computer includes a camera system positioned to capture a surrounding environment in front of a user, a processor that identifies a position of a plurality of light emitters mounted on a hand-held controller from images captured by the camera system and tracks the position of the plurality of light emitters as the hand-held controller moves in the surrounding environment and interprets the tracked position as positional changes of the hand-held controller. The processor uses the position of the plurality of light emitters as markers in three dimensional space, the markers used as an anchor for virtual content presented in a see-through display of the head-worn computer.
Claims27
A system comprising:
- a head wearable device configured to be worn on a head of a user, the head wearable device including:
- a microphone array comprising a plurality of microphones,
- a camera system configured to capture images of a surrounding environment of a user of the wearable head device,
- a see-through display, and
- one or more processors in communication with the one or more microphones, the camera system and the see-through display, the one or more processors being configured to perform a method comprising:
- activating one or more light emitting elements of a plurality of light emitting elements of an input object carried by or held in a hand of the user, the plurality of light emitting elements comprising a first light emitting element of a first wavelength and a second light emitting element of a second wavelength different from the first wavelength,
- activating the camera system to capture an image comprising a visually distorted view of the one or more activated light emitting elements of the input object, wherein the visual distortion is introduced by the camera system,
- identifying a relationship between a movement in the surrounding environment and a movement of the light emitting elements of the input object, wherein the movement in the surrounding environment is smaller than the movement of the light emitting elements of the input object due to, the visual distortion introduced by a lens of the camera system, wherein the camera system is configured to use the visual distortion to increase a sensitivity of the camera system by causing the camera system to exaggerate movement of the light emitting elements of the input object by the user,
- deactivating the one or more previously activated light emitting elements of the input object,
- activating one or more audio emitters of the input object,
- detecting acoustic signals emitted by the one or more audio emitters of the input object and triangulating the detected acoustic signals using the microphone array,
- determining, based on the identified relationship, a position of the input object in the surrounding environment at a first time and determining a predicted position of the input object at a second time later than the first time, wherein said determining the predicted position is based at least in part on
- an input from an orientation sensor of the input object,
- the identified relationship between the movement in the surrounding environment and the exaggerated movement of the activated light emitting elements of the input object, and
- a result of triangulating the detected acoustic signals output by the audio emitters of the input object,
- determining, based on the predicted position of the input object, a display position of virtual content in the see-through display of the wearable head device, and
- presenting the virtual content at the display position in the see-through display of the wearable head device by anchoring the virtual content to the input object based on the predicted position of the input object.
14 more elements
A method comprising:
- activating one or more of a plurality of light emitting elements of an input object carried by or held in a hand of a user of a head wearable device, the plurality of light emitting elements comprising a first light emitting element of a first wavelength and a second light emitting element of a second wavelength different from the first wavelength;
- activating a camera system of a head wearable device configured to capture images of a surrounding environment of a user of the wearable head device, the activated camera system capturing an image comprising a visually distorted view of the one or more activated light emitting elements of the input object wherein the visual distortion is introduced by the camera system;
- identifying a relationship between a movement in the surrounding environment and a movement of the light emitting elements of the input object, wherein the movement in the surrounding environment is smaller than the movement of the light emitting elements of the input object due to the visual distortion introduced by a lens of the camera system, wherein the camera system is configured to use the visual distortion to increase a sensitivity of the camera system by causing the camera system to exaggerate movement of the light emitting elements of the input object by the user,
- deactivating the one or more previously activated light emitting elements of the input object;
- activating one or more audio emitters of the input object;
- detecting acoustic signals emitted by the one or more audio emitters of the input object by a microphone array of the head wearable device, the microphone array including a plurality of microphones;
- determining, based on the identified relationship, a position of the input object at a first time and determining a predicted position of the input object at a second time later than the first time, wherein said determining the predicted position is based at least in part on
- an input from an orientation sensor of the input object,
- the identified relationship between the movement in the surrounding environment and the exaggerated movement of the activated light emitting elements of the input object, and
- a result of triangulating the detected acoustic signals output by the audio emitters of the input object;
- determining, based on the predicted position of the input object, a display position of virtual content in a see-through display of the wearable head device;
- and
- presenting the virtual content at the display position in the see-through display of the wearable head device by anchoring the virtual content to the input object based on the predicted position of the input object.
10 more elements
A non-transitory computer readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising:
- activating one or more of light emitting elements of an input object carried by or held in a hand of a user of a head wearable device, the plurality of light emitting elements comprising a first light emitting element of a first wavelength and a second light emitting element of a second wavelength different from the first wavelength;
- activating a camera system of a head wearable device configured to capture images of a surrounding environment of a user of the wearable head device, the activated camera system capturing an image comprising a visually distorted view of the one or more activated light emitting elements of the input object wherein the visual distortion is introduced by the camera system;
- identifying a relationship between a movement in the surrounding environment and a movement of the light emitting elements of the input object, wherein the movement in the surrounding environment is smaller than the movement of the light emitting elements of the input object due to the visual distortion introduced by a lens of the camera system, wherein the camera system is configured to use the visual distortion to increase a sensitivity of the camera system by causing the camera system to exaggerate movement of the light emitting elements of the input object by the user,
- deactivating the one or more previously activated light emitting elements of the input object;
- activating one or more audio emitters of the input object;
- detecting acoustic signals emitted by the one or more audio emitters of the input object by a microphone array of the head wearable device, the microphone array including a plurality of microphones;
- determining, based on the identified relationship, a position of the input object at a first time and determining a predicted position of the input object at a second time later than the first time, wherein said determining the predicted position is based at least in part on
- an input from an orientation sensor of the input object,
- the identified relationship between the movement in the surrounding environment and the exaggerated movement of the activated light emitting elements of the input object, and
- a result of triangulating the detected acoustic signals output by the audio emitters of the input object;
- determining, based on the predicted position of the input object, a display position of virtual content in a see-through display of the wearable head device;
- and
- presenting the virtual content at the display position in the see-through display of the wearable head device by anchoring the virtual content to the input object based on the predicted position of the input object.
10 more elements
Dependencies are inferred from the claim text.