Gaze-directed photography

A gaze-directed photography system configured to control a camera with gaze tracking, the system comprising:

• at least one camera system having a first dynamic field of view, the at least one camera system including a hand-held physical camera system;
• a gaze-tracking device to track a gaze of a user, the gaze-tracking device being separate from the hand-held physical camera system, and the gaze-tracking device further comprising:
• at least one physical eye-tracking camera configured to track movement of at least one eye of the user;
• and
• a physical reference camera having a second dynamic field of view independent of the first dynamic field of view, the physical reference camera being physically attached to the gaze-tracking device, and the physical reference camera facing outwards from the user in a direction of the gaze of the user;
• a processor;
• and
• a memory coupled to the processor, the memory storing instructions which are executable by the processor to perform a method comprising:
• determining a target of the gaze of the user on the second dynamic field of view via the gaze-tracking device, the first dynamic field of view and the second dynamic field of view each being independent of a dynamic position of the target;
• geometrically mapping the target and a plurality of pixels surrounding the target from the second dynamic field of view of the physical reference camera to a plurality of pixels within the first dynamic field of view of the at least one camera system, wherein the geometrically mapping uses an approximation of an epipolar geometry relationship between the first dynamic field of view and the second dynamic field of view;
• adjusting at least one parameter of an image from the at least one camera system based on the mapped target and the plurality of pixels surrounding the target to include the dynamic position of the target into the first dynamic field of view;
• and
• capturing one or more images via the at least one camera system, the one or more captured images optimized for the target and framed in a manner that is substantially independent of a head position of the user.

A non-transitory computer readable medium storing computer-executable instructions that, when executed, cause a processor to:

• communicate, by the processor, with at least one physical camera system having a first field of view, a physical reference camera having a second field of view, and an eye-tracking camera, wherein the first field of view is independent from the second field of view, and the first and the second fields of view are dynamic;
• receive, by the processor, gaze data from the eye-tracking camera, the gaze data representing a gaze point;
• receive, by the processor, at least one of an image and a video from the at least one physical camera system and at least one of an image and a video from the physical reference camera;
• identify, by the processor, a target on the second dynamic field of view within the at least one of an image and a video from the physical reference camera, a dynamic position of the target being independent of the first dynamic field of view and the second field of view;
• map, by the processor, the target and a plurality of pixels surrounding the target from the second dynamic field of view from the at least one of an image and a video from the physical reference camera to a plurality of pixels within the at least one of an image and a video from the at least one physical camera system, wherein the mapping uses an approximation of an epipolar geometry relationship between the first dynamic field of view and the second dynamic field of view;
• track, by the processor, the identified target;
• adjust at least one parameter of an image from the at least one camera system based on the tracking of the identified target and the mapping to include the dynamic position of the target into the first dynamic field of view;
• and
• capture one or more images optimized for the identified target via the at least one physical camera system, the one or more captured images optimized for the identified target and framed in a manner that is substantially independent of a head position of a user.