# US 12,535,876 B2 · Method and apparatus for virtualizing a computer accessory

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,535,876 B2
- Document ID: US12535876B2
- Status: Granted
- Kind: B2
- Application number: 18/932,061
- Filed: 2024-10-30
- Published: 2025-02-13
- Granted: 2026-01-27
- Earliest claimed: 2018-01-11
- Assignees: STEELSERIES ApS
- Applicants: STEELSERIES ApS
- Inventors: Tino Soelberg
- Web page: https://patentgrep.com/patent/US12535876B2

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity
- A63F 13/22: Setup operations, e.g. calibration, key configuration or button assignment
- A63F 13/23: for interfacing with the game device, e.g. specific interfaces between game controller and console
- A63F 13/25: Output arrangements for video game devices
- A63F 13/5255: according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
- A63F 13/53: involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
- A63F 13/65: automatically by game devices or servers from real world data, e.g. measurement in live racing competition
- G06F 3/021: {Arrangements integrating additional peripherals in a keyboard, e.g. card or barcode reader, optical scanner}
- G06F 3/0219: {Special purpose keyboards}
- G06F 3/0325: {using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image}
- G06F 3/0426: {tracking fingers with respect to a virtual keyboard projected or printed on the surface (virtual keyboards on touch screens G06F3/04886)}
- A63F 13/211: using inertial sensors, e.g. accelerometers or gyroscopes
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 2300/8082: Virtual reality
- G06F 3/0227: {Cooperation and interconnection of the input arrangement with other functional units of a computer (G06F3/023 - G06F3/037 take precedence)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR

## Abstract

A system that incorporates the subject disclosure may include, for example, device may perform operations for receiving a signal from an operational space associated with an accessory device, determining hand feature information according to the signal, accessing device location information associated with the computing accessory device, generating hand-device interaction information according to the hand feature information and device location information associated with the accessory device, and transmitting the hand-device interaction information to a virtual reality system, wherein the virtual reality system generates, according to the hand-device interaction information, a virtual hand and a virtual accessory device in a virtual reality image. Additional embodiments are disclosed.

## Claims

### Claim 1

A method, comprising:
comparing, by a processing system comprising a processor, a virtual stimulation operation according to a first location of a virtual finger and a second location of a virtual keyboard device with an actual stimulation operation to detect a difference, wherein the actual stimulation operation comprises user interaction with a physical keyboard device; and
modifying, by the processing system, the first location of the virtual finger, to obtain a modified virtual finger location, responsive to detecting the difference, wherein the modified virtual finger location corresponds to an updated virtual stimulation that equates to the actual stimulation operation.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
transmitting, by the processing system, the modified virtual finger location to a virtual reality system,
wherein the modified virtual finger location is provided to a virtual reality application by the virtual reality system,
wherein the virtual reality application superimposes a first image of the first location of the virtual finger and a second image of the virtual keyboard device onto a virtual reality display of the virtual reality application according to the modified virtual finger location, and
wherein an input based on actual stimulation data generated by the physical keyboard device is provided to the virtual reality system, the input being preferred over the virtual stimulation operation.

### Claim 3 (depends on 2)

The method of claim 2, further comprising: translating, by the processing system, the virtual stimulation operation into a format compatible with the virtual reality system prior to the transmitting of the modified virtual finger location to the virtual reality system, wherein the virtual stimulation operation is converted to the format compatible with the virtual reality system by a conversion unit attached to the physical keyboard device.

### Claim 4 (depends on 1)

The method of claim 1, wherein a mapping of the first location of the virtual finger and the second location of the virtual keyboard device is obtained from a virtual reality system.

### Claim 5 (depends on 1)

The method of claim 1, further comprising: providing, by the processing system, actual stimulation data generated by the physical keyboard device to a virtual reality system.

### Claim 6 (depends on 5)

The method of claim 5, wherein the actual stimulation data is provided to a virtual reality application by way of the virtual reality system.

### Claim 7 (depends on 6)

The method of claim 6, wherein the virtual reality application comprises a virtual reality gaming application.

### Claim 8 (depends on 4)

The method of claim 4, wherein the virtual reality system comprises a virtual reality operating system.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
recording, by the processing system, an association between first stimulation data that can be generated by the physical keyboard device and a first substitute keyboard stimulation;
receiving, by the processing system, actual stimulation data from the physical keyboard device;
detecting, by the processing system, a match between the actual stimulation data that is received from the physical keyboard device and the association between the first stimulation data and the first substitute keyboard stimulation;
retrieving, by the processing system, the first substitute keyboard stimulation responsive to detecting the match; and
replacing, by the processing system, the actual stimulation data that is received from the physical keyboard device with the first substitute keyboard stimulation to generate actual substitute stimulation data generated by the physical keyboard device prior to transmission of the actual substitute stimulation data to a virtual reality system.

### Claim 10

A non-transitory, machine-readable storage medium, comprising instructions, wherein responsive to executing the instructions, a processing system including a processor performs operations comprising:
comparing a virtual stimulation operation of a virtual accessory device according to a virtual hand-device interaction of between a virtual hand and the virtual accessory device with physical stimulation data to detect a difference, wherein user interaction with a physical accessory device generates the physical stimulation data, and wherein the physical stimulation data corresponds to an actual stimulation operation; and
modifying the virtual hand-device interaction to generate modified virtual hand-device interaction responsive to detecting the difference, wherein the modified virtual hand-device interaction corrects the virtual hand-device interaction so that the virtual hand-device interaction indicated by an apparent location of the virtual hand and the virtual accessory device agrees with the user interaction with the physical accessory device.

### Claim 11 (depends on 10)

The non-transitory, machine-readable storage medium of claim 10, wherein the operations further comprise:
providing the modified virtual hand-device interaction to a virtual reality system,
wherein the modified virtual hand-device interaction is provided by the virtual reality system to a virtual reality application, and
wherein the virtual reality application superimposes the virtual hand and the virtual accessory device onto a virtual reality display of the virtual reality application according to the modified virtual hand-device interaction.

### Claim 12 (depends on 11)

The non-transitory, machine-readable storage medium of claim 11, wherein the virtual reality application comprises a virtual reality gaming application.

### Claim 13 (depends on 10)

The non-transitory, machine-readable storage medium of claim 10, wherein the physical stimulation data is transmitted to a virtual reality system.

### Claim 14 (depends on 13)

The non-transitory, machine-readable storage medium of claim 13, wherein the virtual reality system comprises a virtual reality operating system.

### Claim 15 (depends on 10)

The non-transitory, machine-readable storage medium of claim 10, wherein the operations further comprise:
recording an association between first stimulation data that can be generated by the physical accessory device and a first substitute accessory stimulation;
receiving the physical stimulation data from the physical accessory device;
detecting a match between the physical stimulation data that is received from the physical accessory device and the association between the first stimulation data and the first substitute accessory stimulation;
retrieving the first substitute accessory stimulation responsive to detecting the match; and
replacing the physical stimulation data that is received from the physical accessory device with the first substitute accessory stimulation to generate physical substitute stimulation data generated by the physical accessory device prior to transmission of the physical substitute stimulation data to a virtual reality system.

### Claim 16 (depends on 15)

The non-transitory, machine-readable storage medium of claim 15, wherein the operations further comprise:
receiving image information from an operational space associated with the physical accessory device; and
determining location information associated with the physical accessory device and a proximate hand of a user according to the image information that is received, wherein the location information corresponds to the physical substitute stimulation data.

### Claim 17

A device, comprising:
a processing system including a processor; and
a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations, the operations comprising:
comparing a virtual stimulation operation of a virtual accessory device that corresponds to virtual hand-device interaction information according to a mapping of a virtual hand and virtual accessory device location information with actual stimulation data to detect a disagreement, wherein a user interaction with a physical accessory device generates the actual stimulation data; and
modifying the virtual hand-device interaction information to generate modified virtual hand-device interaction information responsive to the disagreement, wherein the modified virtual hand-device interaction information corrects the virtual hand-device interaction information so that an apparent location of a virtual hand-device interaction indicated by virtual stimulation operation agrees with an actual location of the user interaction of the physical accessory device dictated by the actual stimulation data.

### Claim 18 (depends on 17)

The device of claim 17, wherein the operations further comprise:
providing the modified virtual hand-device interaction information to a virtual reality system, wherein the modified virtual hand-device interaction information is provided by the virtual reality system to a virtual reality application, and
wherein the virtual reality application superimposes a virtual hand and the virtual accessory device onto a virtual reality display of the virtual reality application according to the modified virtual hand-device interaction information.

### Claim 19 (depends on 18)

The device of claim 18, wherein the virtual reality system comprises a virtual reality operating system.

### Claim 20 (depends on 18)

The device of claim 18, wherein the virtual reality application comprises a virtual reality gaming application.