Back to results
US 2023/0410314 A1Published application
Gaze-based control of device operations
AssigneeSnap Inc.
InventorFedir POLIAKOV
Life of this patent
Estimating the term…
Abstract
Systems, devices, media, and methods are presented for gaze-based control of device operations. One method includes receiving a video stream from an imaging device, the video stream depicting one or more eyes, determining a gaze direction for the one or more eyes depicted in the video stream, detecting a change in the gaze direction of the one or more eyes, and triggering an operation in a client device based on the change in the gaze direction.
Claims20
A method for segmenting a video stream, comprising:
- receiving, at a first client device, a first portion of the video stream from an imaging device;
- generating a binary mask indicative of pixels representing an object of interest within one or more images of the first portion of the video stream, the binary mask comprising a set of first pixels having a first value;
- determining an approximate center of mass of the set of first pixels;
- receiving, at the first client device, one or more images of a second portion of the video stream;
- receiving, at the first client device, at least one of the binary mask or tracking data representative of a location of the object of interest within the one or more images of the first portion of the video stream;
- modifying, using the at least one of the binary mask or tracking data, one or more values for pixels of the binary mask representing the object of interest within the second portion of the video stream to generate a modified video stream;
- and
- determining a direction of the object of interest relative to the imaging device and a display device in the modified video stream by calculating a line of sight extending from the center of mass of the set of first pixels to the display device.
5 more elements
A system, comprising:
- one or more processors;
- and
- a memory accessible by the one or more processors, the memory including instructions for execution by the one or more processors that configure the one or more processors to segment a video stream by performing operations including:
- receiving a first portion of the video stream from an imaging device;
- generating a binary mask indicative of pixels representing an object of interest within one or more images of the first portion of the video stream, the binary mask comprising a set of first pixels having a first value;
- determining an approximate center of mass of the set of first pixels;
- receiving one or more images of a second portion of the video stream;
- receiving at least one of the binary mask or tracking data representative of a location of the object of interest within the one or more images of the first portion of the video stream;
- modifying, using the at least one of the binary mask or tracking data, one or more values for pixels of the binary mask representing the object of interest within the second portion of the video stream to generate a modified video stream;
- and
- determining a direction of the object of interest relative to the imaging device and a display device in the modified video stream by calculating a line of sight extending from the center of mass of the set of first pixels to the display device.
8 more elements
A non-transitory machine-readable storage medium storing processor executable instructions that, when executed by a processor of a machine, cause the machine to segment a video stream by performing operations comprising:
- receiving, at a first client device, a first portion of the video stream from an imaging device;
- generating a binary mask indicative of pixels representing an object of interest within one or more images of the first portion of the video stream, the binary mask comprising a set of first pixels having a first value;
- determining an approximate center of mass of the set of first pixels;
- receiving, at the first client device, one or more images of a second portion of the video stream;
- receiving at least one of the binary mask or tracking data representative of a location of the object of interest within the one or more images of the first portion of the video stream;
- modifying, using the at least one of the binary mask or tracking data, one or more values for pixels of the binary mask representing the object of interest within the second portion of the video stream to generate a modified video stream;
- and
- determining a direction of the object of interest relative to a display device in the modified video stream by calculating a line of sight extending from the center of mass of the set of first pixels to the display device.
5 more elements
Dependencies are inferred from the claim text.