# US 10,311,849 B2 · Interactive instruments and other striking objects > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,311,849 B2 - Document ID: US10311849B2 - Status: Granted - Kind: B2 - Application number: 15/996,825 - Filed: 2018-06-04 - Published: 2018-10-04 - Granted: 2019-06-04 - Earliest claimed: 2015-01-08 - Assignees: Muzik Inc. - Applicants: Muzik Inc. - Inventors: Jason Hardi; Eric Gregory White - Web page: https://patentgrep.com/patent/US10311849B2 ## CPC - G10H 3/146: {using a membrane, e.g. a drum; Pick-up means for vibrating surfaces, e.g. housing of an instrument} - G10D 13/02: Drums; Tambourines with drumheads - G10D 13/12: Drumsticks; Mallets (pedals for beaters G10D13/11) - G10D 13/26: {Mechanical details of electronic drums} - G10H 1/0008: {Associated control or indicating means} - G10H 1/0033: {Recording/reproducing or transmission of music for electrophonic musical instruments} - G10H 1/0083: {using wireless transmission, e.g. radio, light, infrared} - G10H 1/18: Selecting circuits - G10H 1/34: Switch arrangements, e.g. keyboards or mechanical switches specially adapted for electrophonic musical instruments - G10H 1/348: {Switches actuated by parts of the body other than fingers} - G10H 2220/021: Indicator, i.e. non-screen output user interfacing, e.g. visual or tactile instrument status or guidance information using lights, LEDs or seven segments displays - G10H 2220/061: LED, i.e. using a light-emitting diode as indicator - G10H 2220/161: with 2D or x/y surface coordinates sensing - G10H 2220/185: Stick input, e.g. drumsticks with position or contact sensors - G10H 2220/311: with controlled tactile or haptic feedback effect; output interfaces therefor - G10H 2220/391: Angle sensing for musical purposes, using data from a gyroscope, gyrometer or other angular velocity or angular movement sensing device - G10H 2220/395: Acceleration sensing or accelerometer use, e.g. 3D movement computation by integration of accelerometer data, angle sensing with respect to the vertical, i.e. gravity sensing - G10H 2230/281: Spint drum assembly, i.e. mimicking two or more drums or drumpads assembled on a common structure, e.g. drum kit - G10H 2250/435: Gensound percussion, i.e. generating or synthesising the sound of a percussion instrument; Control of specific aspects of percussion sounds, e.g. harmonics, under the influence of hitting force, hitting position, settings or striking instruments such as mallet, drumstick, brush or hand - G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE - G10D: STRINGED MUSICAL INSTRUMENTS; WIND MUSICAL INSTRUMENTS; ACCORDIONS OR CONCERTINAS; PERCUSSION MUSICAL INSTRUMENTS; AEOLIAN HARPS; SINGING-FLAME MUSICAL INSTRUMENTS; MUSICAL INSTRUMENTS NOT OTHERWISE PROVIDED FOR (organs, harmoniums or similar wind musical instruments with associated blowing apparatus G10B; pianos, harpsichords, spinets or similar stringed musical instruments with one or more keyboards G10C; automatic musical instruments G10F; electrophonic musical instruments in which tones are generated by electromechanical means G10H) ## Abstract Systems, methods, and devices for providing interactive striking objects (e.g., drumsticks) and performing actions in response to striking motions of the striking objects are disclosed. In some embodiments, the systems and methods provide an interactive drumstick, which includes a lighting display located at a tip portion of the interactive drumstick, a motion detector contained at least partially within the drumstick, a processor and memory contained at least partially within the drumstick, and an interactive system stored within the memory of the drumstick. The interactive system includes a striking motion module that determines striking motions of the drumstick with respect to a virtual percussion instrument based on accessing information measured by the motion detector, and a display module that causes the lighting display to present a certain type of illumination based on the striking motions determined by the striking motion module. ## Claims ### Claim 1 A device comprising: a motion detector contained at least partially within the device; a processor and memory contained at least partially within the device; and an interactive system stored within the memory of the device, wherein the interactive system comprises: a device state module that measures a state of motion of the device relative to a virtual strike location for a virtual strike of a virtual object to be performed by the device; a strike prediction module that determines a predicted time at which a virtual striking portion of the device arrives at the virtual strike location for the virtual strike of the virtual object based on the state of motion; and a haptic feedback module synchronized with the strike prediction module to provide haptic feedback to a user at the predicted time at which the virtual striking portion of the device arrives at the virtual strike location. ### Claim 2 (depends on 1) The device of claim 1, wherein the strike prediction module: measures, from the state of motion of the device relative to the virtual strike location, a current acceleration and trajectory of the device within three-dimensional space with respect to the virtual strike location of the virtual object; and determines the predicted time as a time at which the virtual striking portion of the device is predicted to arrive at the virtual strike location based on the current acceleration and trajectory of the device with respect to the virtual strike location. ### Claim 3 (depends on 1) The device of claim 1, wherein the strike prediction module determines the predicted time as a time at which the state of motion of the device is associated with the virtual striking portion decelerating to approximately zero proximate to the virtual strike location of the virtual object. ### Claim 4 (depends on 1) The device of claim 1, wherein the strike prediction module determines the predicted time as a time at which a trajectory of the device within three-dimensional space with respect to the virtual strike location of the virtual object is predicted to change from a first direction towards the virtual strike location of the virtual object to a second direction away from the virtual strike location of the virtual object. ### Claim 5 (depends on 1) The device of claim 1, wherein the interactive system stored within the memory of the device further comprises: a communication module that communicates a message from the strike prediction module to the haptic feedback module before the virtual striking portion of the device arrives at the virtual strike location of the virtual object, the message comprising information representing the predicted time and information representing the state of motion of the device. ### Claim 6 (depends on 1) The device of claim 1, wherein the device is in communication over a network with a mobile device, an audio presentation device, and/or a gaming system. ### Claim 7 (depends on 1) The device of claim 1, wherein the device comprises a wand, a mobile device, an audio presentation device, a gaming system, and/or is cloud based. ### Claim 8 (depends on 1) The device of claim 1, wherein the device is worn on or is attached to the user. ### Claim 9 (depends on 1) The device of claim 1, wherein the device fits in, is worn on, and/or is attached to a hand of the user. ### Claim 10 (depends on 1) The device of claim 1, wherein the virtual striking portion is a virtual extension of the device. ### Claim 11 A method, comprising: measuring a state of motion of a device relative to a virtual strike location for a virtual strike of a virtual object at the virtual strike location; determining a predicted time at which a virtual striking portion of the device arrives at the virtual strike location for the virtual strike of the virtual object based on the state of motion of the device; performing a remote action associated with a striking object striking an object at the predicted time delayed by a first time to compensate for a communications delay associated with the virtual strike; and performing a local action associated with the striking object striking the object at the predicted time delayed by a second time that is less than the first time. ### Claim 12 (depends on 11) The method of claim 11, wherein determining the predicted time at which a virtual striking portion of the device arrives at the virtual strike location for a virtual strike of the virtual object based on the state of motion of the device comprises: measuring, from the state of motion of the device relative to the virtual strike location, a current acceleration and trajectory of the device within three-dimensional space with respect to the virtual strike location of the virtual object; and determining the predicted time as a time at which a virtual strike portion of the device is estimated to arrive at the virtual strike location based on the current acceleration and trajectory of the device with respect to the virtual strike location. ### Claim 13 (depends on 11) The method of claim 11, wherein determining the predicted time at which the virtual striking portion of the device arrives at the virtual strike location for the virtual strike of the virtual object based on the state of motion of the striking object includes determining the predicted time as a time at which the state of motion of the device is associated with the device decelerating to approximately zero when proximate to the virtual strike location of the virtual object. ### Claim 14 (depends on 11) The method of claim 11, wherein determining the predicted time at which the virtual striking portion of the device arrives at the virtual strike location for the virtual strike of the virtual object based on the state of motion of the device comprises determining the predicted time as a time at which a trajectory of the device within three-dimensional space with respect to the virtual strike location of the virtual object is predicted to change from a first direction towards the virtual strike location of the virtual object to a second direction away from the virtual strike location of the virtual object. ### Claim 15 (depends on 11) The method of claim 11, wherein performing a remote action associated with the device striking an object upon commencement of the predicted time includes causing an audio and/or video presentation device to play a sound and/or video indicative of striking the object. ### Claim 16 (depends on 11) The method of claim 11, wherein the device is in communication over a network with a mobile device, an audio presentation device, and/or a gaming system. ### Claim 17 (depends on 11) The method of claim 11, wherein the device is worn on or is attached to a user of the device. ### Claim 18 (depends on 11) The method of claim 11, wherein the device fits in, is worn on, and/or is attached to a hand of a user of the device. ### Claim 19 (depends on 11) The method of claim 11, wherein the virtual striking portion is a virtual extension of the device. ### Claim 20 A non-transitory computer-readable medium whose contents, when executed by a computing system, cause the computing system to perform operations for generating an audio and/or a video sequence based on a monitored movement of a device with respect to virtual object locations, the operations comprising: monitoring movement of the device relative to the virtual object locations; determining predicted times of virtual strikes performed by the device at the virtual object locations; generating an audio and/or video sequence that comprises sounds and/or video to be played at the predicted times of virtual strikes at the virtual object locations; and generating haptic feedback at the device synchronized with the predicted times of virtual strikes at the virtual object locations.