# US 12,008,154 B2 · Method and system for the detection and augmentation of tactile interactions in augmented reality > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,154 B2 - Document ID: US12008154B2 - Status: Granted - Kind: B2 - Application number: 18/212,052 - Filed: 2023-06-20 - Published: 2023-10-19 - Granted: 2024-06-11 - Earliest claimed: 2017-03-21 - Assignees: InterDigital VC Holdings, Inc. - Applicants: InterDigital VC Holdings, Inc. - Inventors: Vincent Levesque - Web page: https://patentgrep.com/patent/US12008154B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves} - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 18/24: Classification techniques - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Program actions may be initiated after detection of a predefined gesture by a user with a real-world object. Users may interact with their physical environment in an augmented reality by detecting interactions with real objects using a combination of location and motion detection, material identification using wearable sensors, or both. Based on detected sensor data from user interaction with a real-world object, a predefined gesture may be identified and a program action associated with that target interaction for the real-world object may be executed. In some cases, the user experience may be enhanced by providing haptic feedback in response to tactile gestures and resulting events. ## Claims ### Claim 1 A method comprising: operating at least one sensor in communication with a mobile device to detect vibration data; executing a first haptic feedback response comprising a simulated texture, wherein executing the first haptic feedback response comprises: detecting vibrations resulting from continuing contact between a user and a first real-world object; and controlling a first haptic feedback component to adjust the first haptic feedback response in view of the detected vibrations. ### Claim 2 (depends on 1) The method of claim 1, wherein adjusting the first haptic feedback response comprises increasing an intensity of the first haptic feedback response. ### Claim 3 (depends on 1) The method of claim 1, wherein adjusting the first haptic feedback response comprises decreasing an intensity of the first haptic feedback response. ### Claim 4 (depends on 1) The method of claim 1, wherein adjusting the first haptic feedback response corresponds to outputting a haptic effect modulating a perceived coefficient of friction of the simulated texture. ### Claim 5 (depends on 1) The method of claim 1, wherein the at least one sensor and the first haptic feedback component are parts of a wearable device of the user. ### Claim 6 (depends on 5) The method of claim 5, wherein the wearable device is a smartwatch. ### Claim 7 (depends on 1) The method of claim 1, further comprising: determining a location of the user; and using the location of the user to retrieve, from an object database, information associated with the first real world object, the information comprising haptic feedback information to simulate the simulated texture. ### Claim 8 (depends on 1) The method of claim 1, further comprising: determining a location of the user; and using the location of the user to retrieve, from an object database, information associated with the first real world object, the information comprising a description of a detected gesture. ### Claim 9 An apparatus comprising: a processor; and a non-transitory computer-readable medium storing instructions operative, when executed by the processor, to cause the apparatus to: operating at least one sensor in communication with a mobile device to detect vibration data; executing a haptic feedback response comprising a simulated texture, wherein executing the first haptic feedback response comprises: detecting vibrations resulting from continuing contact between a user and a first real-world object; and controlling a first haptic feedback component to adjust the first haptic feedback response in view of the detected vibrations. ### Claim 10 (depends on 9) The method of claim 9, wherein the sensor is an accelerometer. ### Claim 11 (depends on 9) The method of claim 9, wherein the mobile device is a smartwatch. ### Claim 12 A method comprising: detecting that a user is making a gesture against a real world object, monitoring, using a vibration sensor, first vibrations from contact of the user with the real world object as a result of the gesture; obtaining second vibrations associated with a virtual texture; subtracting the first vibrations from the second vibrations to get a haptic feedback signal; and simulating the virtual texture by rendering the haptic feedback signal using a haptic actuator. ### Claim 13 (depends on 12) The method of claim 12, further comprising overlaying the virtual texture on the real world object using an augmented reality display device. ### Claim 14 (depends on 12) The method of claim 12, further comprising: rendering the haptic feedback signal while motion of the user against the real world object is detected; and stopping the rendering of the haptic feedback signal when motion of the user against the real world object is no longer detected. ### Claim 15 (depends on 12) The method of claim 12, wherein the vibration sensor and the haptic actuator are parts of a wearable device of the user. ### Claim 16 (depends on 15) The method of claim 15, wherein the wearable device is a smartwatch in communication with an AR device of the user. ### Claim 17 (depends on 12) The method of claim 12, further comprising: determining a location of the user; and using the location of the user to retrieve, from an object database, information associated with the real world object, the information comprising haptic feedback information to simulate the virtual texture. ### Claim 18 (depends on 12) The method of claim 12, further comprising: determining a location of the user; and using the location of the user to retrieve, from an object database, information associated with the real world object, the information comprising a description of the detected gesture.