Steerable camera for AR hand tracking
AssigneeSnap Inc.
InventorsDaniel Colascione, Patrick Timothy McSweeney Simons, Weston Welge, Ramzi Zahreddine
Life of this patent
Estimating the term…
Abstract
A system for hand tracking for an Augmented Reality (AR) system. The AR system uses a camera of the AR system to capture tracking video frame data of a hand of a user of the AR system. The AR system generates a skeletal model based on the tracking video frame data and determines a location of the hand of the user based on the skeletal model. The AR system causes a steerable camera of the AR system to focus on the hand of the user.
Claims20
A computer-implemented method comprising:
- capturing, by a non-steerable camera of an Augmented Reality (AR) system, video frame data of a real-world scene environment from a perspective of a user of the AR system, the video frame data including portions of one or more hands of the user, and the non-steerable camera having a field of view equal to a field of view of the AR system;
- determining a location of one or more hands of the user based on the video frame data captured by the non-steerable camera;
- generating steering command data based on the location of the one or more hands of the user;
- and
- steering an optical axis of a steerable camera of the AR system based on the steering command data to focus the steerable camera on the one or more hands of the user.
A machine comprising:
- at least one processor;
- and
- at least one memory storing instructions that, when executed by the at least one processor, cause the machine to perform operations comprising:
- capturing, by a non-steerable camera of an Augmented Reality (AR) system, video frame data of a real-world scene environment from a perspective of a user of the AR system, the video frame data including portions of one or more hands of the user, and the non-steerable camera having a field of view equal to a field of view of the AR system;
- determining a location of one or more hands of the user based on the video frame data captured by the non-steerable camera;
- generating steering command data based on the location of the one or more hands of the user;
- and
- steering an optical axis of a steerable camera of the AR system based on the steering command data to focus the steerable camera on the one or more hands of the user.
A non-transitory machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
- capturing, by a non-steerable camera of an Augmented Reality (AR) system, video frame data of a real-world scene environment from a perspective of a user of the AR system, the video frame data including portions of one or more hands of the user, and the non-steerable camera having a field of view equal to a field of view of the AR system;
- determining a location of one or more hands of the user based on the video frame data captured by the non-steerable camera;
- generating steering command data based on the location of the one or more hands of the user;
- and
- steering an optical axis of a steerable camera of the AR system based on the steering command data to focus the steerable camera on the one or more hands of the user.
Dependencies are inferred from the claim text.