# US 2024/0004475 A1 · Spatially aware computing hub and environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004475 A1
- Document ID: US20240004475A1
- Status: Published application
- Kind: A1
- Application number: 18/464,933
- Filed: 2023-09-11
- Published: 2024-01-04
- Earliest claimed: 2020-03-16
- Assignees: ARKH, Inc.
- Applicants: ARKH, Inc.
- Inventors: Landon Nickerson
- Web page: https://patentgrep.com/patent/US20240004475A1

## CPC

- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves}
- G06F 3/015: {Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
- G06Q 20/3224: {Transactions dependent on location of M-devices}
- G06Q 20/40145: {Biometric identity checks}
- G06Q 30/0639: {Locating goods or services, e.g. based on physical position of the goods or services within a shopping facility}
- G09B 21/001: {Teaching or communicating with blind persons (G09B21/02 - G09B21/06 take precedence)}
- G09B 21/009: {Teaching or communicating with deaf persons (G09B21/04 takes precedence)}
- G06F 2203/0331: Finger worn pointing device
- G06N 20/00: Machine learning
- G06Q 20/321: {using wearable devices}
- G06Q 20/36: using electronic wallets or electronic money safes
- G09B 9/00: Simulators for teaching or training purposes (for the use of weapons F41; computing aspects G06; {protocols for games, networked simulations or virtual reality H04L67/131})
- G16H 40/67: for remote operation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Aspects of the disclosure provide for a sensory augmentation system. In some examples, the sensory augmentation system includes a first electro-tactile stimulator comprising at least one user interaction element driven by a driver and contactable to a human body to provide stimulation having a first stimulation waveform and a signal generation and sensing processor configured to receive data corresponding to information to be communicated to the user. The signal generation and sensing processor may generate a first instruction to the driver, the first instruction corresponding to the first stimulation waveform. The driver may generate the stimulation having the first stimulation waveform in response to the first instruction.

## Claims

### Claim 1

A sensory augmentation system, comprising:
a hub comprising a processor and memory, the hub configured to:
store data regarding a context environment;
determine and maintain an indication of a relative position of multiple items in the context environment;
determine and maintain a relative position of a computing device in the context environment;
based on the determined relative positions of the multiple items and the computing device, determine an interaction has occurred between a user of the computing device and one of the multiple items.

### Claim 2 (depends on 1)

The system of claim 1, wherein the interaction is the user removing the one of the multiple items from a product display, and wherein the hub executes a transaction to charge the user for the one of the multiple items responsive to the interaction.

### Claim 3 (depends on 1)

The system of claim 1, wherein the sensory augmentation system includes multiple sensors, and wherein the relative position of the multiple items in the context environment is determined and maintained by the multiple sensors providing data to the hub.

### Claim 4 (depends on 1)

The system of claim 1, wherein the hub maintains a spatial map of the context environment, augmented with digital assets to form an augmented reality environment, and wherein the augmented reality environment is viewable via the computing device.

### Claim 5 (depends on 1)

The system of claim 1, wherein the hub is configured to determine the relative position of the computing device according to a non-visual process.

### Claim 6 (depends on 1)

The system of claim 1, wherein the hub is configured to determine the relative position of the computing device via wireless signal triangulation including the computing device.

### Claim 7 (depends on 1)

The system of claim 1, wherein the hub is configured to determine the relative position of the computing device relative to a position of the hub according to wireless signal processing.

### Claim 8 (depends on 1)

The system of claim 1, wherein the hub is configured to track changes in location of at least some of the multiple items in the context environment with respect to a location of the hub and the computing device.

### Claim 9

A method, comprising:
storing, at a hub, data regarding a context environment;
determining and maintaining an indication of a relative position of multiple items in the context environment;
determining and maintaining a relative position of a computing device in the context environment;
based on the determined relative positions of the multiple items and the computing device, determining an interaction has occurred between a user of the computing device and one of the multiple items.

### Claim 10 (depends on 9)

The method of claim 9, wherein the interaction is the user removing the one of the multiple items from a product display, and wherein the hub executes a transaction to charge the user for the one of the multiple items responsive to the interaction.

### Claim 11 (depends on 9)

The method of claim 9, wherein the relative position of the multiple items in the context environment is determined and maintained by sensors providing data to the hub.

### Claim 12 (depends on 9)

The method of claim 9, further comprising maintaining, at the hub, a spatial map of the context environment, augmented with digital assets to form an augmented reality environment, and wherein the augmented reality environment is viewable via the computing device.

### Claim 13 (depends on 9)

The method of claim 9, further comprising determining the relative position of the computing device relative to a position of the hub via wireless signal processing.

### Claim 14 (depends on 9)

The method of claim 9, further comprising determining the relative position of the computing device via wireless signal triangulation including the computing device.

### Claim 15 (depends on 9)

The method of claim 9, further comprising determining the relative position of the computing device via non-visual information.

### Claim 16

A sensory augmentation system, comprising:
a wearable device; and
a hub comprising a processor and memory, the hub configured to:
store data regarding a context environment in which the hub is located;
determine and maintain an indication of a relative position of multiple items in the context environment;
determine and maintain a relative position of a computing device in the context environment;
based on the determined relative positions of the multiple items and the computing device, determine an interaction has occurred between a user of the computing device and one of the multiple items.

### Claim 17 (depends on 16)

The system of claim 16, wherein the hub is configured to determine the relative position of the computing device via wireless signal triangulation including the computing device and the wearable device.

### Claim 18 (depends on 16)

The system of claim 16, wherein the hub is configured to determine the relative position of the computing device relative to a position of the hub according to wireless signal processing.

### Claim 19 (depends on 16)

The system of claim 16, wherein the hub is further configured to determine the interaction has occurred between the user of the computing device and one of the multiple items based on a determined position of the wearable device.

### Claim 20 (depends on 16)

The system of claim 16, wherein the hub is configured to track changes in location of at least some of the multiple items in the context environment with respect to a location of the hub and the computing device.
