Gesture-Based Interface Plugin and Integration SDK for Web and App Environments

A computer-implemented method for enabling touchless gesture-based interaction with a digital user interface, the method comprising:

• capturing, via a camera interface of a client device, a sequence of image frames representing hand movements of a user;
• processing, by a gesture recognition engine executed on the client device, the sequence of image frames to extract hand landmark data;
• classifying, by the gesture recognition engine, the hand landmark data into one or more predefined gesture types;
• associating, by a gesture-to-interface mapping module, each of the one or more predefined gesture types with a corresponding user interface action;
• binding, by the gesture-to-interface mapping module, the corresponding user interface action to at least one target element of a Document Object Model (DOM) using selector-based identifiers;
• generating, by an event injection module, one or more synthetic interface events corresponding to the associated user interface action;
• and applying the one or more synthetic interface events to the at least one target element within the DOM to execute the user interface action without physical touch input..

A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a client device, cause the client device to perform operations comprising:

• capturing a sequence of image frames representing hand movements of a user;
• extracting hand landmark data from the sequence of image frames using a gesture recognition engine;
• classifying the hand landmark data into one or more predefined gesture types;
• mapping each of the one or more predefined gesture types to a corresponding user interface action;
• binding the corresponding user interface action to at least one target element of a Document Object Model (DOM) using selector-based identifiers;
• generating one or more synthetic interface events corresponding to the user interface action;
• and injecting the one or more synthetic interface events into the DOM to cause execution of the user interface action without physical touch input.