# US 12,182,334 B2 · Gesture-based user interface device and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,334 B2 - Document ID: US12182334B2 - Status: Granted - Kind: B2 - Application number: 18/011,954 - Filed: 2021-06-22 - Published: 2023-06-01 - Granted: 2024-12-31 - Earliest claimed: 2020-06-26 - Assignees: InterDigital Madison Patent Holdings, SAS - Applicants: Interdigital Madison Patent Holdings, SAS - Inventors: Thomas Morin; Serge Defrance; Hassane Guermoud; Sylvain Lelievre - Web page: https://patentgrep.com/patent/US12182334B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A user interface device including memory configured to store instructions and at least one hardware processor configured to execute the instructions to obtain, from at least one sensor, sensor data corresponding to a user gesture, determine a user interface gesture that matches the sensor data, obtain parameters for the determined interface gesture, and upon obtaining the parameters for the determined interface gesture, use the sensor data and the parameters for the determined interface gesture to estimate the user gesture in a user interface. Also provided is a method. ## Claims ### Claim 1 A device comprising: memory configured to store instructions, executable by a processor, and to store parameters; and at least one hardware processor configured to execute the instructions to perform steps of a method comprising: obtaining, from at least one sensor, sensor data corresponding to a captured user gesture; and upon determining that not all the sensor data is reliable: determining a stored user interface gesture that matches the sensor data; obtaining parameters for the determined stored user interface gesture learnt from a present user, wherein: in case no parameters for the determined stored user interface gesture learnt from the present user are obtained, obtaining default parameters for the determined stored user interface gesture; in case no default parameters for the determined stored user interface gesture are obtained, using the sensor data in the user interface; and enhancing reliable parts of the sensor data with the parameters for the determined stored user interface gesture by replacing unreliable sensor data with data based on the parameters to estimate the user gesture in a user interface. ### Claim 2 (depends on 1) The device of claim 1, wherein the determining the stored user interface gesture that matches the sensor data further comprises: determining the stored user interface gesture with a highest similarity score to the sensor data. ### Claim 3 (depends on 1) The device of claim 1, wherein the data based on the parameters is further obtained from reliable sensor data. ### Claim 4 (depends on 3) The device of claim 3, wherein the sensor data comprises node coordinates and relative node coordinates of the user interface gesture and wherein the reliable sensor data comprises reliable coordinates for at least one node and wherein the data based on the parameters is obtained by adding relative node coordinates to the reliable coordinates for the at least one node. ### Claim 5 (depends on 1) The device of claim 1, wherein the estimated user gesture is used in the user interface for at least one of determining interaction with an object in the user interface and rendering of the user gesture in the user interface. ### Claim 6 (depends on 1) The device of claim 1, wherein the at least one hardware processor is configured to determine whether all the sensor data is reliable. ### Claim 7 (depends on 1) The device of claim 1, wherein the at least one hardware processor is further configured to, upon determining that all the sensor data is reliable, estimating the user gesture in the user interface based only on the sensor data. ### Claim 8 A method implemented in at least one hardware processor of a device, the method comprising: obtaining, from at least one sensor, sensor data corresponding to a captured user gesture; and upon determining that not all the sensor data is reliable: determining a stored user interface gesture that matches the sensor data; obtaining parameters for the determined stored user interface gesture learnt from a present user, wherein: in case no parameters for the determined stored user interface gesture learnt from the present user are obtained, obtaining default parameters for the determined stored user interface gesture; in case no default parameters for the determined stored user interface gesture are obtained, using the sensor data in the user interface; and enhancing reliable parts of the sensor data with the parameters for the determined stored user interface gesture by replacing unreliable sensor data with data based on the parameters to estimate the user gesture in a user interface. ### Claim 9 (depends on 8) The method of claim 8, wherein determining the stored user interface gesture that matches the sensor data comprises determining the stored user interface gesture with a highest similarity score to the sensor data. ### Claim 10 (depends on 8) The method of claim 8, further comprising upon determining that all the sensor data is reliable, estimating the user gesture in the user interface based only on the sensor data. ### Claim 11 (depends on 8) A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one hardware processor to perform a method of claim 8.