# US 2021/0333884 A1 · Method and device for adjusting the control-display gain of a gesture controlled electronic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0333884 A1 - Document ID: US20210333884A1 - Status: Published application - Kind: A1 - Application number: 17/085,866 - Filed: 2020-10-30 - Published: 2021-10-28 - Earliest claimed: 2020-04-26 - Assignees: Huawei Technologies Co., Ltd. - Inventors: Wei LI; Wei ZHOU; Sachi MIZOBUCHI; Ghazaleh SANIEE-MONFARED; Juwei LU; Taslim Arefin KHAN; Rafael VERAS GUIMARAES - Web page: https://patentgrep.com/patent/US20210333884A1 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/50: Depth or shape recovery - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/30196: Human being; Person - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Methods, devices, and processor-readable media for adjusting the control-display gain of a gesture-controlled device are described. Adjusting the control-display gain may facilitate user interaction with content or UI elements rendered on a display screen of the gesture-controlled device. The control-display gain may be adjusted based on a property of how a mid-air dragging gesture is being performed by a user's hand. The property may be the location of the gesture, the orientation of the hand performing the gesture, or the velocity of the gesture. A hand that becomes stationary for a threshold time period while performing the dragging gesture may adjust the control-display gain to a different level. Control-display gain may be set to a different value based on the current velocity of the hand performing the gesture. The control-display gain levels may be selected from a continuous range of values or a set of discrete values. Devices for performing the methods are described. ## Claims ### Claim 1 A method for adjusting the control-display gain of a gesture-controlled device, the method comprising: processing a frame to detect a location and shape of a hand in the frame; generating gesture data for the frame based on the location and shape of the hand; determining that the hand is performing a dragging hand gesture based on gesture data for the frame and gesture data for one or more previous frames; identifying a property of the hand performing the dragging hand gesture; and adjusting a control-display gain for the gesture-controlled device to a control-display gain determined based on the identified property. ### Claim 2 (depends on 1) The method of claim 1, wherein the property is the location of the hand. ### Claim 3 (depends on 2) The method of claim 2, the method further comprising: storing the location of the hand in a queue that includes one or more locations of the hand while the gesture-controlled device is in a dragging state; determining a movement of the hand with respect to a first axis based on the one or more locations of the hand stored in the queue; and applying the control-display gain to the movement to generate a drag control output value, wherein the control-display gain is determined based on the location of the hand with respect to a second axis orthogonal to the first axis. ### Claim 4 (depends on 2) The method of claim 2, wherein: identifying the location of the hand comprises determining that the hand is located within a first region of the frame; and the control-display gain is a first control-display gain corresponding to the first region. ### Claim 5 (depends on 1) The method of claim 1, wherein the property is an orientation of the hand relative to a reference orientation. ### Claim 6 (depends on 5) The method of claim 5, wherein the orientation of the hand comprises an angle of the hand relative to the reference orientation. ### Claim 7 (depends on 6) The method of claim 6, wherein the angle of the hand is measured with respect to a vertical plane. ### Claim 8 (depends on 6) The method of claim 6, wherein: identifying the orientation of the hand comprises determining that the angle of the hand is within a first angle range defined between a first angle and a second angle; and adjusting the control-display gain comprises adjusting the control-display gain to a first control-display gain corresponding to the first angle range. ### Claim 9 (depends on 1) The method of claim 1, wherein the property is a velocity of the hand. ### Claim 10 (depends on 9) The method of claim 9, wherein: identifying the velocity of the hand comprises: determining a location of the hand; storing the location of the hand in a queue that includes one or more locations of the hand while the gesture-controlled device is in a drag state; and determining the velocity of the hand based on the one or more locations of the hand stored in the queue; adjusting the control-display comprises: in response to determining that the velocity has been below a velocity threshold for at least a dwell time threshold, and that the gesture-controlled device is in a first control-display gain state of a plurality of control-display gain states: placing the gesture-controlled device into a second control-display gain state of the plurality of control-display gain states; and adjusting the control-display gain to a control-display gain corresponding to the second control-display gain state. ### Claim 11 (depends on 1) The method of claim 1, wherein: identifying the velocity of the hand comprises: determining a location of the hand; storing the location of the hand in a queue that includes one or more locations of the hand while the gesture-controlled device is in a drag state; and determining the velocity of the hand based the one or more locations of the hand stored in the queue; and the control-display gain is determined in accordance with a function dependent on the velocity of the hand. ### Claim 12 (depends on 11) The method of claim 11, wherein the function dependent on the velocity is a linear function. ### Claim 13 (depends on 11) The method of claim 11, wherein the function dependent on the velocity is a generalized logistic function. ### Claim 14 A gesture-controlled device comprising: a processor device; and a memory storing machine-executable instructions thereon which, when executed by the processing device, cause the gesture-controlled device to: processing a frame to detect a location and shape of a hand in the frame; generating gesture data for the frame based on the location and shape of the hand; determining that the hand is performing a dragging hand gesture based on gesture data for the frame and gesture data for one or more previous frames; identify a property of a hand of a user of the gesture-controlled device performing the dragging hand gesture; and adjust a control-display gain for the gesture-controlled device to a control-display gain determined based on the identified property. ### Claim 15 (depends on 14) The gesture-controlled device of claim 14, wherein the property is the location of the hand. ### Claim 16 (depends on 14) The gesture-controlled device of claim 14, wherein the property is an orientation of the hand relative to a reference orientation. ### Claim 17 (depends on 14) The gesture-controlled device of claim 14, wherein the property is a velocity of the hand. ### Claim 18 (depends on 17) The gesture-controlled device of claim 17, wherein: identifying the velocity of the hand comprises: determining a location of the hand; storing the location of the hand in a queue that includes one or more locations of the hand while the gesture-controlled device is in a drag state; and determining the velocity of the hand based the one or more locations of the hand stored in the queue; adjusting the control-display comprises: in response to determining that the velocity has been below a velocity threshold for at least a dwell time threshold, and that the gesture-controlled device is in a first control-display gain state of a plurality of control-display gain states: placing the gesture-controlled device into a second control-display gain state of the plurality of control-display gain states; and adjusting the control-display gain to a control-display gain corresponding to the second control-display gain state. ### Claim 19 (depends on 17) The gesture-controlled device of claim 17, wherein: identifying the velocity of the hand comprises: determining a location of the hand; storing the location of the hand in a queue that includes one or more locations of the hand while the gesture-controlled device is in a drag state; and determining the velocity of the hand based the one or more locations of the hand stored in the queue; and the control-display gain is determined in accordance with a function dependent on the velocity of the hand. ### Claim 20 A processor-readable medium having machine-executable instructions stored thereon which, when executed by a processor of a gesture-controlled device, cause the processor device to: process a frame to detect a location and shape of a hand mid-air hand gesture in the frame; generate gesture data for the frame based on the location and shape of the hand; determine that the mid-air hand is performing hand gesture is a dragging hand gesture based on gesture data for the frame and gesture data for one or more previous frames; identify a property of a hand of a user of the gesture-controlled device performing the dragging hand gesture; and adjust a control-display gain for the gesture-controlled device to a control-display gain determined based on the identified property.