# US 10,120,445 B2 · Glove interface object with flex sensing and wrist tracking for virtual interaction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,120,445 B2 - Document ID: US10120445B2 - Status: Granted - Kind: B2 - Application number: 15/980,542 - Filed: 2018-05-15 - Published: 2018-09-13 - Granted: 2018-11-06 - Earliest claimed: 2014-08-22 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Sony Interactive Entertainment Inc. - Inventors: Shai Messingher; Richard Stenson - Web page: https://patentgrep.com/patent/US10120445B2 ## CPC - G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves} - A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/017: {Head mounted} - G02B 27/0172: {characterised by optical features} - G06F 3/012: {Head tracking input arrangements} - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06F 3/041: Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G06F 2203/013: Force feedback applied to a game - G06F 2203/04102: Flexible digitiser, i.e. constructional details for allowing the whole digitising part of a device to be flexed or rolled like a sheet of paper - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A glove interface object, including: a flex sensor configured to generate flex sensor data identifying a flex of a finger portion of the glove interface object; a trackable object configured to be illuminated during interactivity, the trackable object being positioned at a wrist portion of the glove interface object; a communications module configured to transmit the flex sensor data to a computing device for processing to determine a finger position pose of the glove interface object, applied for rendering a virtual hand in a view of a virtual environment on a head-mounted display (HMD), the virtual hand rendered based on the identified finger position pose, the computing device identifying the trackable object from captured image data to track a location of the glove interface object; wherein the virtual hand is rendered at a location in the virtual environment that is substantially defined by the location of the glove interface object. ## Claims ### Claim 1 A glove interface object, comprising: at least one flex sensor configured to generate flex sensor data identifying a flex of at least one finger portion of the glove interface object; at least one trackable object that is configured to be illuminated during interactivity, the at least one trackable object being positioned at a wrist portion of the glove interface object; a communications module configured to transmit the flex sensor data to a computing device for processing to determine a finger position pose of the glove interface object, the finger position pose being applied for rendering a virtual hand in a view of a virtual environment on a head-mounted display (HMD), the virtual hand being rendered based on the identified finger position pose, the computing device configured to identify the at least one trackable object from captured image data to enable tracking of a location of the glove interface object in an interactive environment; wherein the virtual hand is rendered at a location in the virtual environment that is substantially defined by the location of the glove interface object in the interactive environment. ### Claim 2 (depends on 1) The glove interface object of claim 1, wherein the at least one trackable object includes a plurality of trackable objects. ### Claim 3 (depends on 1) The glove interface object of claim 1, wherein the computing device is configured to identify the plurality of trackable objects from the captured image data to enable tracking of an orientation of the glove interface object in the interactive environment; wherein the virtual hand is rendered having an orientation in the virtual environment that is substantially defined by the orientation of the glove interface object in the interactive environment. ### Claim 4 (depends on 3) The glove interface object of claim 3, wherein the plurality of trackable objects are configured to be illuminated to have different colors, the different colors being identified from the captured image data to enable the tracking of the orientation of the glove interface object in the interactive environment. ### Claim 5 (depends on 2) The glove interface object of claim 2, wherein the plurality of trackable objects are configured to be illuminated according to a predefined pattern, the predefined pattern being identified from the captured image data to enable the tracking of the orientation of the glove interface object in the interactive environment. ### Claim 6 (depends on 1) The glove interface object of claim 1, wherein the at least one trackable object includes at least one light or LED. ### Claim 7 (depends on 1) The glove interface object of claim 1, wherein the wrist portion of the glove interface object is defined by a bracelet, the trackable object being attached to the bracelet. ### Claim 8 (depends on 1) The glove interface object of claim 1, further comprising: at least one inertial sensor for generating inertial sensor data; wherein the communications module is configured to transmit the inertial sensor data to the computing device for processing to enable the tracking of the location of the glove interface object in the interactive environment. ### Claim 9 (depends on 8) The glove interface object of claim 8, wherein the communications module is configured to transmit the inertial sensor data to the computing device for processing to enable tracking of an orientation of the glove interface object in the interactive environment; wherein the virtual hand is rendered having an orientation in the virtual environment that is substantially defined by the orientation of the glove interface object in the interactive environment. ### Claim 10 (depends on 1) The glove interface object of claim 1, wherein the view of the virtual environment is defined from a perspective of a virtual character in the virtual environment that is associated to the head-mounted display; wherein the virtual hand is a hand of the virtual character. ### Claim 11 (depends on 1) The glove interface object of claim 1, wherein the HMD includes an image capture device for capturing images of the interactive environment. ### Claim 12 (depends on 1) The glove interface object of claim 1, wherein the communications module is configured to receive haptic feedback data from the computing device; the glove interface object further comprising a haptic feedback mechanism that is configured to generate haptic feedback based on the haptic feedback data. ### Claim 13 A glove interface object, comprising: a plurality of flex sensors configured to generate flex sensor data identifying flexion of a plurality of finger portions of the glove interface object; a plurality of trackable objects configured to be illuminated during interactivity, the plurality of trackable objects being positioned at a wrist portion of the glove interface object; a communications module configured to transmit the flex sensor data to a computing device for processing to determine a finger position pose of the glove interface object, the finger position pose being applied for rendering a virtual hand in a view of a virtual environment on a head-mounted display (HMD), the virtual hand being rendered based on the identified finger position pose, the computing device configured to identify the plurality of trackable objects from captured image data to enable tracking of a location and orientation of the glove interface object in an interactive environment; wherein the virtual hand is rendered at a location in the virtual environment that is substantially defined by the location of the glove interface object in the interactive environment; wherein the virtual hand is rendered having an orientation in the virtual environment that is substantially defined by the orientation of the glove interface object in the interactive environment. ### Claim 14 (depends on 13) The glove interface object of claim 13, wherein the plurality of trackable objects are configured to be illuminated to have different colors, the different colors being identified from the captured image data to enable the tracking of the location and orientation of the glove interface object in the interactive environment. ### Claim 15 (depends on 13) The glove interface object of claim 13, wherein the plurality of trackable objects are configured to be illuminated according to a predefined pattern, the predefined pattern being identified from the captured image data to enable the tracking of the location and orientation of the glove interface object in the interactive environment. ### Claim 16 (depends on 13) The glove interface object of claim 13, further comprising: at least one inertial sensor for generating inertial sensor data; wherein the communications module is configured to transmit the inertial sensor data to the computing device for processing to enable the tracking of the location and orientation of the glove interface object in the interactive environment. ### Claim 17 A method, comprising: receiving flex sensor data generated from at least one flex sensor of a glove interface object, the flex sensor data identifying a flex of at least one finger portion of the glove interface object; receiving captured image data of an interactive environment in which the glove interface object is disposed; processing the flex sensor data to determine a finger position pose of the glove interface object, and applying the finger position pose to render a virtual hand in a view of a virtual environment that is displayed through a head-mounted display (HMD); processing the captured image data to identify at least one trackable object positioned at a wrist portion of the glove interface object, the trackable object configured to be illuminated during interactivity, wherein processing the captured image data enables tracking of a location of the glove interface object in the interactive environment; wherein the virtual hand is rendered at a location in the virtual environment that is substantially defined by the location of the glove interface object in the interactive environment. ### Claim 18 (depends on 17) The method of claim 17, further comprising: receiving inertial sensor data generated from at least one inertial sensor of the glove interface object; processing the inertial sensor data to enable the tracking of the location of the glove interface object in the interactive environment. ### Claim 19 (depends on 18) The method of claim 18, wherein the processing of the inertial sensor data further enables tracking of an orientation of the glove interface object in the interactive environment; wherein the virtual hand is rendered having an orientation in the virtual environment that is substantially defined by the orientation of the glove interface object in the interactive environment. ### Claim 20 (depends on 17) The method of claim 17, wherein the view of the virtual environment is defined from a perspective of a virtual character in the virtual environment that is associated to the head-mounted display; wherein the virtual hand is a hand of the virtual character.