# US 10,310,618 B2 · Gestures visual builder tool > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,310,618 B2 - Document ID: US10310618B2 - Status: Granted - Kind: B2 - Application number: 14/985,775 - Filed: 2015-12-31 - Published: 2017-07-06 - Granted: 2019-06-04 - Earliest claimed: 2015-12-31 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Kfir Karmon; Eyal Krupka; Yuval Tzairi; Uri Levanon; Shelly Horowitz - Web page: https://patentgrep.com/patent/US10310618B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 8/34: Graphical or visual programming - G06V 10/40: Extraction of image or video features - G06V 40/11: {Hand-related biometrics; Hand pose recognition} - G06V 40/113: {Recognition of static hand signs} - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A system for creating hand gestures representations, comprising an interface for interacting with a user, a storage storing a plurality of discrete pose values and discrete motion values, a memory storing a gesture visual builder code, one or more processors coupled to the interface, storage and memory to execute the gesture visual builder code allowing the user to create hand gesture. The gesture visual builder code comprising code instructions to present the user with a GUI which displays a hierarchical menu driven interface, code instructions to receive iteratively user instructions from the user using the hierarchical menu driven interface, for creating a logical sequence of hand gesture by defining one or more hand pose features records and hand motion features records and code instructions to generate a code segment defining the one or more hand pose/motion features records through the discrete pose/motion values respectively. ## Claims ### Claim 1 A system for creating hand gestures representations, comprising: an interface for interacting with a user; a storage storing a plurality of discrete pose values and a plurality of discrete motion values; a memory storing a gesture visual builder code; at least one processor coupled to said interface, said storage and said memory, enabled to execute said gesture visual builder code to allow said user to create code segments representing at least one hand gesture, said gesture visual builder code comprising: code instructions to present a graphical user interface (GUI) to said user, said GUI displays a hierarchical menu driven interface; code instructions to receive iteratively a plurality of user instructions indicated by said user using said hierarchical menu driven interface, for creating a logical sequence of hand poses and hand motions constructing at least one hand gesture by defining at least one of a plurality of hand pose features records and a plurality of hand motion features records; and code instructions to generate a software code segment describing said at least one hand gesture by defining said at least one hand pose features record through at least one of said plurality of discrete pose values and said at least one hand motion features record through at least one of said plurality of discrete motion values; wherein said generated software code segment is adapted to be executed by at least one application for implementing a gesture Human Machine Interface (HMI). ### Claim 2 (depends on 1) The system of claim 1, wherein each of said discrete pose values is indicative of a corresponding one of a plurality of hand pose features and each of said discrete motion values is indicative of a corresponding one of a plurality of hand motion features. ### Claim 3 (depends on 2) The system of claim 2, wherein said hand pose feature is a member selected from a group consisting of: active hand, palm direction, palm rotation, flexion of at least one finger, direction of at least one finger, relative location between at least two fingers and tangency between at least two fingers. ### Claim 4 (depends on 2) The system of claim 2, wherein said hand motion feature is a member selected from a group consisting of: motion properties and motion script, said motion script defines at least one of: hand motion and motion of at least one finger. ### Claim 5 (depends on 1) The system of claim 1, wherein said logical sequence which represents said at least one hand gesture is defined by a unique finite state machine model. ### Claim 6 (depends on 1) The system of claim 1, further comprising said code segment conforms to a gesture application user interface (API). ### Claim 7 (depends on 1) The system of claim 1, further comprising said GUI displays said logical sequence so that each of said at least one hand pose features record and said at least one hand motion features record is assigned with a construction area which is displayed with an associated instance of said hierarchical menu driven interface. ### Claim 8 (depends on 7) The system of claim 7, further comprising said associated instance is a hierarchical arc menus structure comprising a plurality of menus shaped as arcs each having at least one of a plurality of feature fields, said hierarchical arc menus structure having at least one of a plurality of levels: a first level arc menu which is displayed by said GUI adjacent to said construction area; and at least one of a plurality of next level arc menus, each of said plurality of next level arc menus is adjacent to a respective previous level arc menu and having a larger radius from a center of said construction area; wherein each of said plurality of feature fields is associated with a gesture feature which is assigned at least one of: said plurality of discrete pose values and said plurality of discrete motion values. ### Claim 9 (depends on 8) The system of claim 8, wherein said each next level arc menu is presented by said GUI following an indication in said at least one field of said previous level arc menu, said indication is made by said user. ### Claim 10 (depends on 8) The system of claim 8, further comprising said at least one feature field is actively displayed to indicate availability of said associated gesture creation operation. ### Claim 11 (depends on 1) The system of claim 1, further comprising at least one of: said plurality of hand pose features records and said plurality of hand motion features records, is included in said logical sequence by capturing at least one of hand pose and hand motion, said at least one of hand pose and hand motion are articulated by said user and captured by at least one imaging device. ### Claim 12 A computer implemented method for creating hand gestures representations, comprising: using an application code executed on at least one processor for: presenting a graphical user interface (GUI) to a user, said GUI displays a hierarchical menu driven interface; receiving iteratively a plurality of user instructions indicated by said user using said hierarchical menu driven interface, for creating a logical sequence of hand poses and hand motions constructing at least one hand gesture by defining at least one of a plurality of hand pose features records and a plurality of hand motion features records; and generating a software code segment describing said at least one hand gesture by defining said at least one hand pose features record through at least one of a plurality of discrete pose values and said at least one hand motion features record through at least one of a plurality of discrete motion values; wherein said generated software code segment is adapted to be executed by at least one application for implementing a gesture Human Machine Interface (HMI). ### Claim 13 (depends on 12) The computer implemented method of claim 12, wherein each of said discrete pose values is indicative of a corresponding one of a plurality of hand pose features and each of said discrete motion values is indicative of a corresponding one of a plurality of hand motion features. ### Claim 14 (depends on 13) The computer implemented method of claim 13, wherein said hand pose feature is a member selected from a group consisting of: active hand, hand direction, palm rotation, flexion of at least one finger, direction of at least one finger, relative location between at least two fingers and tangency between at least two fingers. ### Claim 15 (depends on 13) The computer implemented method of claim 13, wherein said hand motion feature is a member selected from a consisting of: motion properties and motion script, said motion script defines at least one of: hand motion and motion of at least one finger. ### Claim 16 A software program product for creating hand gestures representations, comprising: a non-transitory computer readable storage medium; first program instructions to present a graphical user interface (GUI) to a user, said GUI displays a hierarchical menu driven interface; second program instructions code instructions to receive iteratively a plurality of user instructions indicated by said user using said hierarchical menu driven interface, for creating a logical sequence of hand poses and hand motions constructing at least one hand gesture by defining at least one of a plurality of hand pose features records and a plurality of hand motion features records; and third program instructions code instructions to generate a software code segment describing said at least one hand gesture by defining said at least one hand pose features record through at least one of a plurality of discrete pose values and said at least one hand motion features record through at least one of a plurality of discrete motion values; wherein said generated software code segment is adapted to be executed by at least one application for implementing a gesture Human Machine Interface (HMI); and wherein said first, second and third program instructions are executed by at least one computerized processor from said non-transitory computer readable storage medium. ### Claim 17 (depends on 16) The software program product of claim 16, wherein each of said discrete pose values is indicative of a corresponding one of a plurality of hand pose features and each of said discrete motion values is indicative of a corresponding one of a plurality of hand motion features. ### Claim 18 (depends on 17) The software program product of claim 17, wherein said hand pose feature is a member selected from a group consisting of: active hand, palm direction, palm rotation, flexion of at least one finger, direction of at least one finger, relative location between at least two fingers and tangency between at least two fingers. ### Claim 19 (depends on 17) The software program product of claim 17, wherein said hand motion feature is a member selected from a group consisting of: motion properties and motion script, said motion script defines at least one of: hand motion and motion of at least one finger. ### Claim 20 (depends on 16) The software program product of claim 16, wherein said logical sequence which represents said at least one hand gesture is defined by a unique finite state machine model. ### Claim 21 (depends on 16) The software program product of claim 16, further comprising said code segment conforms to a gesture application user interface (API). ### Claim 22 (depends on 16) The software program product of claim 16, further comprising said GUI displays said logical sequence so that each of said at least one hand pose features record and said at least one hand motion features record is assigned with a construction area which is displayed with an associated instance of said hierarchical menu driven interface. ### Claim 23 (depends on 22) The software program product of claim 22, further comprising said associated instance is a hierarchical arc menus structure comprising a plurality of menus shaped as arcs each having at least one of a plurality of feature fields, said hierarchical arc menus structure having at least one of a plurality of levels: a first level arc menu which is displayed by said GUI adjacent to said construction area; and at least one of a plurality of next level arc menus, each of said plurality of next level arc menus is adjacent to a respective previous level arc menu and having a larger radius from a center of said construction area; wherein each of said plurality of feature fields is associated with a gesture feature which is assigned at least one of: said plurality of discrete pose values and said plurality of discrete motion values. ### Claim 24 (depends on 23) The software program product of claim 23, wherein said each next level arc menu is presented by said GUI following an indication in said at least one field of said previous level arc menu, said indication is made by said user. ### Claim 25 (depends on 23) The software program product of claim 23, further comprising said at least one feature field is actively displayed to indicate availability of said associated gesture creation operation.