Systems and methods of direct pointing detection for interaction with a digital device

A non-transitory computer readable medium storing instructions which, when executed, cause at least one processor to perform operations for recognizing an aimed point or area in a space, the operations comprising:

• obtaining, using an image sensor, at least one image of the space;
• processing the at least one image by at least one processor operatively connected to the image sensor and obtaining data indicative of more than one location of at least one pointing element in the space;
• obtaining data indicative of a location of at least one predefined body part of a user in the space;
• determining by the at least one processor, a location of the aimed point or area in the space using a combination of:
• the data indicative of the more than one location of the at least one pointing element,
• the data indicative of the location of the at least one predefined body part, and
• a plurality of selected position data components associated with a motion path of the at least one pointing element during a pointing gesture,
• wherein the at least one processor determines the location of the aimed point or area by:
• determining a respective viewing ray for each of the plurality of selected position data components, the respective viewing ray extending from the location of the at least one predefined body part through the location of the at least one pointing element, and intersecting a plane on which at least one graphical element is displayed, thus yielding a respective candidate plane, wherein the location of the at least one pointing element corresponds to the given position data component;
• determining an overlapping area between the respective candidate planes, and
• determining the aimed point or area using data indicative of the overlapping area;
• determining a change in a motion vector of the at least one pointing element during the pointing gesture, wherein the change in the motion vector relates to a change in a direction of the motion vector in at least one of a vertical axis or a horizontal axis;
• determining a user selection of an icon or a graphical element, based on the change in the motion vector and the determined location of the aimed point or area;
• and
• executing a predefined command or message associated with the selected icon or the graphical element.

A system for recognizing an aimed point or area in a space, the system comprising at least one processor configured to:

• obtain, using an image sensor, at least one image of the space;
• process the at least one image by at least one processor operatively connected to the image sensor and obtaining data indicative of more than one location of at least one pointing element in the space;
• obtain data indicative of a location of at least one predefined body part of a user in the space;
• determine by the at least one processor, a location of the aimed point or area in the space using a combination of:
• the data indicative of the more than one location of the at least one pointing element,
• the data indicative of the location of the at least one predefined body part, and
• a plurality of selected position data components associated with a motion path of the at least one pointing element during a pointing gesture,
• wherein the at least one processor determines the location of the aimed point or area by:
• determining a respective viewing ray for each of the plurality of selected position data components, the respective viewing ray extending from the location of the at least one predefined body part through the location of the at least one pointing element, and intersecting a plane on which at least one graphical element is displayed, thus yielding a respective candidate plane, wherein the location of the at least one pointing element corresponds to the given position data component;
• determining an overlapping area between the respective candidate planes, and
• determining the aimed point or area using data indicative of the overlapping area;
• determine a change in a motion vector of the at least one pointing element during the pointing gesture, wherein the change in the motion vector relates to a change in a direction of the motion vector in at least one of a vertical axis or a horizontal axis;
• determining a user selection of an icon or a graphical element, based on the change in the motion vector and the determined location of the aimed point or area;
• and
• execute a predefined command or message associated with the selected icon or the graphical element.