Detecting Ink Gestures based on Spatial and Image Data Processing

A method for processing an ink document, comprising:

• receiving a particular ink stroke most recently drawn by the a using an ink capture device, within the ink document;
• identifying a grouping of ink strokes for analysis that includes at least the particular ink stroke;
• identifying spatial information that describes a sequence of positions traversed by the user in drawing the grouping of ink strokes using the ink capture device;
• converting the grouping of ink strokes into image information;
• mapping the spatial information into a first feature embedding;
• mapping the image information into a second feature embedding;
• and
• classifying a gesture associated with the grouping of ink strokes based on a combination of the first feature embedding and the second feature embedding.

A computer-readable storage medium for storing computer-readable instructions, the computer-readable instructions, when executed by one or more hardware processors, performing a method that comprises:

• receiving a particular ink stroke most recently drawn by a user using an ink capture device, within an ink document;
• identifying a spatial cluster of ink strokes for analysis that includes at least the particular ink stroke, said identifying including:
• identifying a spatial scope of analysis that encompasses the particular ink stroke;
• extending the spatial scope of analysis;
• and
• identifying any other ink stroke that is included in the spatial scope of analysis as extended, and adding said any other ink stroke to the spatial cluster of ink strokes;
• and
• classifying a gesture associated with the spatial cluster of ink strokes based on spatial information pertaining to the spatial cluster of ink strokes and image information pertaining to the spatial cluster of ink strokes,
• the spatial information describing a sequence of positions traversed by the user in drawing the spatial cluster of ink strokes using the ink capture device, and
• the image information produced by rendering the spatial cluster of ink storages into an image.