# US 12,111,973 B2 · System and method to manipulate virtual model based on physical parameters of gesture input > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,111,973 B2 - Document ID: US12111973B2 - Status: Granted - Kind: B2 - Application number: 17/902,672 - Filed: 2022-09-02 - Published: 2024-01-11 - Granted: 2024-10-08 - Earliest claimed: 2022-07-06 - Assignees: Shopify Inc. - Applicants: Shopify Inc. - Inventors: Brennan Letkeman; Bradley Joseph Aldridge - Web page: https://patentgrep.com/patent/US12111973B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06T 2219/2016: Rotation, translation, scaling - G06T 2219/2024: Style variation - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The present disclosure provides AR systems and methods. The computer-implemented method comprises displaying, on a mobile device, a rendered virtual object in an augmented reality (AR) scene, and detecting, using an input device of the mobile device, a gesture having a detected speed. The method further includes identifying the gesture, wherein the gesture is identified as a first command to implement a first function related to the virtual object responsive to the detected speed of the gesture being less than a speed threshold, and the gesture is identified as a second command to implement a second function related to the virtual object responsive to the detected speed of the gesture being greater than the speed threshold. The second command and second function are different from the first command and the first function, respectively. The identified gesture is then processed. ## Claims ### Claim 1 A computer-implemented method comprising: displaying, on a mobile device, a virtual model of a product rendered in an augmented reality (AR) scene; detecting, using an input device of the mobile device, a gesture having a detected speed; and identifying the gesture as a command affecting the virtual model within the AR scene, wherein: the gesture is identified as a first command to implement a first function that manipulates the virtual model within the AR scene responsive to the detected speed of the gesture being less than a speed threshold, and the gesture is identified as a second command to implement a second function that replaces the virtual model within the AR scene with a subsequent virtual model of another product in a defined sequence of virtual models responsive to the detected speed of the gesture being greater than the speed threshold; and processing the identified gesture. ### Claim 2 (depends on 1) The method of claim 1, wherein processing the identified gesture comprises: implementing the first function that manipulates the virtual model within the AR scene or implementing the second function that replaces the virtual model within the AR scene with the subsequent virtual model based on the gesture being identified as the first command or the second command. ### Claim 3 (depends on 1) The method of claim 1, wherein the first function includes rotating the virtual model in the AR scene on the mobile device or changing a location of the virtual model within the AR scene. ### Claim 4 (depends on 1) The method of claim 1, wherein the method is performed in a first mode, the method further comprising: receiving indication to switch to a second mode; detecting, using the input device of the mobile device, another gesture having a second detected parameter; and identifying the other gesture further comprising: the other gesture is identified as a third command to implement a third function related to the virtual model responsive to the second detected parameter of the other gesture being within a parameter range, the third command being different from the first command and the second command, and the third function being different from the first function and the second function. ### Claim 5 (depends on 4) The method of claim 4, wherein the second detected parameter is a detected direction and the preset parameter range is a preset direction range. ### Claim 6 (depends on 4) The method of claim 4, wherein the third function includes one or more of: adding the product corresponding to the virtual model to a virtual cart, saving current virtual model options, buying the product corresponding to the virtual model, deleting the virtual model from the AR scene, or exiting the AR scene. ### Claim 7 (depends on 4) The method of claim 4, wherein identifying the other gesture further comprises: the other gesture is identified as a fourth command to implement a fourth function related to the virtual model responsive to the second detected parameter of the other gesture being outside the parameter range, the fourth command being different from the first, second, and third commands, and the fourth function being different from the first, second, and third functions. ### Claim 8 (depends on 1) The method of claim 1, wherein implementing the first and the second functions depend on the detected speed of the gesture in absence of displaying the first and second functions as selectable options on a screen of the mobile device. ### Claim 9 (depends on 1) The method of claim 1, further comprising providing a visual, tactile, and/or audible notification indicating whether the first or the second function is being implemented. ### Claim 10 (depends on 1) The method of claim 1, wherein the AR scene includes the virtual model rendered in a virtual background, and wherein the gesture causes the virtual model to be manipulated or replaced in the virtual background. ### Claim 11 A system comprising: a processing unit configured to execute instructions to cause the system to: display, on a mobile device, a virtual model of a product rendered in an augmented reality (AR) scene; detect, using an input device of the mobile device, a gesture having a detected speed; and identify the gesture as a command affecting the virtual model within the AR scene, wherein: the gesture is identified as a first command to implement a first function that manipulates the virtual model within the AR scene responsive to the detected speed of the gesture being less than a speed threshold, and the gesture is identified as a second command to implement a second function that replaces the virtual model within the AR scene with a subsequent virtual model of another product in a defined sequence of virtual models responsive to the detected speed of the gesture being greater than the speed threshold; and process the identified gesture. ### Claim 12 (depends on 11) The system of claim 11, wherein the processing unit is further configured to execute instructions to cause the system to process the identified gesture by implementing the first function that manipulates the virtual model within the AR scene or implementing the second function that replaces the virtual model within the AR scene with the subsequent virtual model based on the gesture being identified as the first command or the second command. ### Claim 13 (depends on 11) The system of claim 11, wherein the first function includes rotating the virtual model in the AR scene on the mobile device or changing a location of the virtual model within the AR scene. ### Claim 14 (depends on 11) The system of claim 11, wherein the processing unit is operating in a first mode, the processing unit being further configured to execute instructions to cause the system to: receive indication to switch to a second mode; detect, using the input device of the mobile device, another gesture having a second detected parameter; and identify the other gesture wherein: the other gesture is identified as a third command to implement a third function related to the virtual model responsive to the second detected parameter of the other gesture being within a parameter range, the third command being different from the first command and the second command, and the third function being different from the first function and the second function. ### Claim 15 (depends on 14) The system of claim 14, wherein the second detected parameter is a detected direction and the preset parameter range is a preset direction range. ### Claim 16 (depends on 14) The system of claim 14, wherein the third function includes one or more of: adding the product corresponding to the virtual model to a virtual cart, saving current virtual model options, buying the product corresponding to the virtual model, deleting the virtual model from the AR scene, or exiting the AR scene. ### Claim 17 (depends on 11) The system of claim 11, wherein the processing unit is further configured to execute instructions to cause the system to provide a visual, tactile, and/or audible notification indicating whether the first or the second function is being implemented. ### Claim 18 (depends on 11) The system of claim 11, wherein implementing the first and the second functions depend on the detected speed of the gesture in absence of displaying the first and second functions as selectable options on a screen of the mobile device. ### Claim 19 A non-transitory computer-readable medium storing instructions that, when executed by a processor of a system, causes the system to: display, on a mobile device, a virtual model of a product rendered in an augmented reality (AR) scene; detect, using an input device of the mobile device, a gesture having a detected speed; and identify the gesture as a command affecting the virtual model within the AR scene, wherein: the gesture is identified as a first command to implement a first function that manipulates the virtual model within the AR scene responsive to the detected speed of the gesture being less than a speed threshold, and the gesture is identified as a second command to implement a second function that replaces the virtual model within the AR scene with a subsequent virtual model of another product in a defined sequence of virtual models responsive to the detected speed of the gesture being greater than the speed threshold; and process the identified gesture. ### Claim 20 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the first function includes rotating the virtual model in the AR scene on the mobile device or changing a location of the virtual model within the AR scene. ### Claim 21 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the method is performed in a first mode, the method further comprising: receiving indication to switch to a second mode; detecting, using the input device of the mobile device, another gesture having a second detected parameter; and identifying the other gesture further comprising: the other gesture is identified as a third command to implement a third function related to the virtual model responsive to the second detected parameter of the other gesture being within a parameter range, the third command being different from the first command and the second command, and the third function being different from the first function and the second function.