# US 12,118,152 B2 · Method, device for gesture recognition and electronic equipment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,152 B2 - Document ID: US12118152B2 - Status: Granted - Kind: B2 - Application number: 17/881,130 - Filed: 2022-08-04 - Published: 2022-11-24 - Granted: 2024-10-15 - Earliest claimed: 2021-03-16 - Assignees: Qingdao Pico Technology Co., Ltd. - Applicants: Qingdao Pico Technology Co., Ltd. - Inventors: Tao Wu - Web page: https://patentgrep.com/patent/US12118152B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The present application discloses a method, a device for gesture recognition and an electronic equipment. The method includes: receiving a first gesture trajectory input by a user; obtaining a target scene category according to the first gesture trajectory, wherein the target scene category represents a category of an interactive scene in which the user is located; and obtaining one or more target gesture instructions according to the target scene category and the first gesture trajectory. ## Claims ### Claim 1 A gesture recognition method, comprising: receiving a first gesture trajectory input by a user at a current moment; obtaining a second gesture trajectory input by the user in a preset historical time before the current moment; obtaining a gesture trajectory behavior feature corresponding to the user according to the first gesture trajectory and the second gesture trajectory, wherein the gesture trajectory behavior feature is data that characterizes the behavior or intent of the user and is derived from a gesture trajectory state of the user within a certain duration obtained by comprehensively analyzing the first gesture trajectory of the current moment of the user and the second gesture trajectory within a historical duration; obtaining a target scene category according to the gesture trajectory behavior feature, wherein the target scene category represents a category of an interactive scene in which the user is located; obtaining one or more target gesture instructions according to the target scene category and the first gesture trajectory. ### Claim 2 (depends on 1) The method according to claim 1, wherein obtaining the target scene category according to the gesture trajectory behavior feature comprises: obtaining at least one gesture instruction to be matched according to the gesture trajectory behavior feature; and performing a query operation in a preset correspondence between scene categories and gesture instructions according to the at least one gesture instruction to be matched, to obtain a scene category that meets a preset condition as the target scene category. ### Claim 3 (depends on 2) The method according to claim 2, wherein performing the query operation in the preset correspondence between scenario types and gesture instructions according to the at least one gesture instruction to be matched, to obtain a scene category that meets a preset condition as the target scene category comprises: obtaining at least one gesture instruction set according to the correspondence between the scene categories and the gesture instructions, wherein each gesture instruction set corresponds to a scene category; separately calculating the number of instructions that are in the gesture instruction set and match the at least one gesture instruction to be matched, to obtain at least one matching number, wherein each matching number corresponds to a scene category; and selecting out a target matching number that meets the preset numerical condition from the at least one matching number, and taking the scene category corresponding to the target matching number as the target scene category. ### Claim 4 (depends on 1) The method according to claim 1, wherein obtaining the target gesture instructions according to the target scene category and the first gesture trajectory comprises: inputting the first gesture trajectory into a gesture recognition model to obtain predicted gesture instructions; and obtaining the target gesture instructions according to the predicted gesture instructions and the target scene category. ### Claim 5 (depends on 4) The method according to claim 4, wherein the number of the predicted gesture instructions is more than one, the predicted gesture instructions are output by the gesture recognition model together with a confidence condition corresponding to each gesture instruction; obtaining the target gesture instructions according to the predicted gesture instructions and the target scene category comprises: selecting, from the predicted gesture instructions, an instruction whose corresponding scenario category is the target scenario category and whose corresponding confidence condition meets a preset confidence condition as the target gesture instruction. ### Claim 6 (depends on 4) The method according to claim 4, wherein a training process of the gesture recognition model comprises: obtaining sample data, wherein the sample data is obtained by collecting preset gesture actions, and the preset gesture actions are obtained by making statistics of gesture actions under a preset scene category; and training the gesture recognition model by the sample data, to obtain a gesture recognition model that satisfies a preset convergence condition. ### Claim 7 (depends on 1) The method according to claim 1, wherein the method is applied to an electronic equipment, the number of the target gesture instructions is more than one, and after obtaining the target gesture instructions, the method further comprises: displaying the target gesture instructions; receiving a gesture instruction selected by the user from the target gesture instructions; and making a response corresponding to the selected gesture instruction. ### Claim 8 (depends on 5) The method according to claim 5, wherein the preset confidence condition comprises: a value of the confidence condition satisfies a preset ranking. ### Claim 9 A gesture recognition device, comprises a memory storing instructions and a processor in communication with the memory, wherein the processor is configured to execute the instructions to: receive a first gesture trajectory input by a user at a current moment; obtain a second gesture trajectory input by the user in a preset historical time before the current moment; obtain a gesture trajectory behavior feature corresponding to the user according to the first gesture trajectory and the second gesture trajectory, wherein the gesture trajectory behavior feature is data that characterizes the behavior or intent of the user and is derived from a gesture trajectory state of the user within a certain duration obtained by comprehensively analyzing the first gesture trajectory of the current moment of the user and the second gesture trajectory within a historical duration; obtain a target scene category according to the gesture trajectory behavior feature, wherein the target scene category represents a category of an interactive scene in which the user is located; and obtain one or more target gesture instructions according to the target scene category and the first gesture trajectory. ### Claim 10 (depends on 1) An electronic equipment, wherein the equipment comprises: A memory, which is configured to store executable instructions; and A processor, which is configured to run the electronic equipment under the control of the instruction and execute the method of claim 1. ### Claim 11 (depends on 1) A non-transitory computer-readable storage medium, which is used to store computer programs that could carry out the method of claim 1 when run by the processor. ### Claim 12 (depends on 9) The device according to claim 9, wherein the processor, when obtain the target scene category according to the gesture trajectory behavior feature, is further configured to: obtain at least one gesture instruction to be matched according to the gesture trajectory behavior feature; and perform a query operation in a preset correspondence between scene categories and gesture instructions according to the at least one gesture instruction to be matched, to obtain a scene category that meets a preset condition as the target scene category. ### Claim 13 (depends on 12) The device according to claim 12, wherein the processor, when perform the query operation in the preset correspondence between scenario types and gesture instructions according to the at least one gesture instruction to be matched, to obtain a scene category that meets a preset condition as the target scene category, is further configured to: obtain at least one gesture instruction set according to the correspondence between the scene categories and the gesture instructions, wherein each gesture instruction set corresponds to a scene category; separately calculate the number of instructions that are in the gesture instruction set and match the at least one gesture instruction to be matched, to obtain at least one matching number, wherein each matching number corresponds to a scene category; and select out a target matching number that meets the preset numerical condition from the at least one matching number, and taking the scene category corresponding to the target matching number as the target scene category. ### Claim 14 (depends on 9) The device according to claim 9, wherein the processor, when obtain target gesture instructions according to the target scene category and the first gesture trajectory, is further configured to: input the first gesture trajectory into a gesture recognition model to obtain predicted gesture instructions; and obtain the target gesture instructions according to the predicted gesture instructions and the target scene category. ### Claim 15 (depends on 14) The device according to claim 14, wherein the number of the predicted gesture instructions is more than one, the predicted gesture instructions are output by the gesture recognition model together with a corresponding confidence condition corresponding to each gesture instruction, the processor, when obtain the target gesture instructions according to the predicted gesture instructions and the target scene category, is further configured to: select, from the predicted gesture instructions, an instruction whose corresponding scenario category is the target scenario category and whose corresponding confidence condition meets a preset confidence condition as the target gesture instruction. ### Claim 16 (depends on 14) The device according to claim 14, wherein a training process of the gesture recognition model comprises: obtaining sample data, wherein the sample data is obtained by collecting preset gesture actions, and the preset gesture actions are obtained by making statistics of gesture actions under a preset scene category; and training the gesture recognition model by the sample data, to obtain a gesture recognition model that satisfies a preset convergence condition. ### Claim 17 (depends on 9) The device according to claim 9, wherein the number of the target gesture instructions is more than one, after obtaining the target gesture instructions, the processor is further configured to: display the target gesture instructions; receive a gesture instruction selected by the user from the target gesture instructions; and make a response corresponding to the selected gesture instruction. ### Claim 18 (depends on 15) The device according to claim 15, wherein the preset confidence condition comprises: a value of the confidence condition satisfies a preset ranking.