# US 2018/0024362 A1 · Data processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0024362 A1
- Document ID: US20180024362A1
- Status: Published application
- Kind: A1
- Application number: 15/656,244
- Filed: 2017-07-21
- Published: 2018-01-25
- Earliest claimed: 2016-07-22
- Assignees: Arm Limited
- Applicants: ARM Ltd
- Inventors: Paul WILLIAMSON
- Web page: https://patentgrep.com/patent/US20180024362A1

## CPC

- G02B 27/017: {Head mounted}
- G01S 1/20: using a comparison of transit time of synchronised signals transmitted from non-directional antennas or antenna systems spaced apart, i.e. path-difference systems
- G01S 5/10: Position of receiver fixed by co-ordinating a plurality of position lines defined by path-difference measurements {, e.g. omega or decca systems}(G01S5/12 takes precedence {; beacons and receivers cooperating therewith G01S1/306, G01S1/308})
- G01S 5/12: by co-ordinating position lines of different shape, e.g. hyperbolic, circular, elliptical or radial
- G01S 5/14: Determining absolute distances from a plurality of spaced points of known location
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G01S 5/02: using radio waves (using satellite radio beacon systems for determining position G01S19/00)
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0147: {comprising a device modifying the resolution of the displayed image}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Measures, including methods, systems, and non-transitory computer-readable storage media, for processing data for an augmented reality environment. An augmented reality user device receives at least one signal from a radio frequency beacon at at least one radio frequency receiver of the augmented reality user device. A spatial resolution operation is performed in relation to the at least one received signal to determine a location of the radio frequency beacon. A virtual object is rendered in an augmented reality environment on the augmented reality user device at least on the basis of the determined location.

## Claims

### Claim 1

A method of processing data for an augmented reality environment, the method comprising, at an augmented reality user device:
receiving at least one signal from a radio frequency beacon at at least one radio frequency receiver of the augmented reality user device;
performing a spatial resolution operation in relation to the at least one received signal to determine a location of the radio frequency beacon; and
rendering a virtual object in an augmented reality environment on the augmented reality user device at least on the basis of the determined location,
wherein the rendering comprises rendering one or more audio features of the virtual object in relation to the determined location in the augmented reality environment.

### Claim 2 (depends on 1)

The method of claim 1, wherein the augmented reality user device comprises a plurality of audio channel loudspeakers and rendering the one or more audio features comprises outputting a plurality of audio signals comprising differing spatial information via the plurality of audio channel loudspeakers,
whereby a user of the augmented reality user device is able to determine a relative orientation of the virtual object by spatial audio perception.

### Claim 3 (depends on 1)

The method of claim 1, comprising transmitting a connection request to the radio frequency beacon.

### Claim 4 (depends on 3)

The method of claim 3, wherein the connection request comprises a request to connect to the radio frequency beacon over a different type of connection from a connection over which the at least one signal is received from the radio frequency beacon and/or over which the connection request is transmitted to the radio frequency beacon.

### Claim 5 (depends on 4)

The method of claim 4, wherein the different type of connection has a higher bandwidth than the connection over which the at least one signal is received from the radio frequency beacon and/or over which the connection request is transmitted to the radio frequency beacon.

### Claim 6 (depends on 1)

The method of claim 1, wherein one or more visual features of the virtual object are rendered in association with the one or more audio features.

### Claim 7 (depends on 6)

The method of claim 6, wherein the one or more audio features and the one or more visual features of the virtual object are rendered in association with a real object.

### Claim 8 (depends on 1)

The method of claim 1, wherein the augmented reality device comprises a user interface for interacting with a real object via user input.

### Claim 9 (depends on 8)

The method of claim 8, wherein the user interface is operable to change one or more settings of the real object via user input.

### Claim 10 (depends on 9)

The method of claim 9, comprising:
obtaining a key based on data comprised in the received at least one signal; and
using the key to enable the one or more settings of the real object to be changed.

### Claim 11 (depends on 8)

The method of claim 8, wherein the user interface comprises a voice user interface.

### Claim 12 (depends on 8)

The method of claim 8, wherein the user interface comprises a graphical user interface and wherein the augmented reality user device comprises a headset configurable to provide user input to the graphical user interface via eye-tracking functionality of the headset.

### Claim 13 (depends on 1)

The method of claim 1, wherein the received at least one signal comprises real-time information from the local environment of a real object, the method comprising rendering the virtual object in the augmented reality environment further on the basis of the real-time information from the local environment of the real object.

### Claim 14 (depends on 1)

The method of claim 1, comprising using a network address comprised in the received at least one signal to access the rendering information at the network address,
wherein the rendering is performed on the basis of the accessed rendering information.

### Claim 15 (depends on 1)

The method of claim 1, wherein the augmented reality user device comprises an array of at least two radio frequency receivers and the receiving comprises receiving a plurality of signals at a plurality of radio frequency receivers in the array and wherein performing the spatial resolution operation comprises:
identifying angles of arrival and received signal strengths of the received plurality of signals;
identifying angles of departure and received signal strengths of the received plurality of signals; and/or
identifying times of flight and received signal strengths of the received plurality of signals.

### Claim 16 (depends on 1)

The method of claim 1, comprising:
maintaining a blacklist of one or more virtual objects which are not to be rendered in an augmented reality environment on the augmented reality user device; and
inhibiting rendering of a virtual object in an augmented reality environment on the augmented reality user device if the virtual object is on the maintained blacklist.

### Claim 17 (depends on 1)

The method of claim 1, comprising:
maintaining a whitelist of one or more virtual objects which are can be rendered in an augmented reality environment on the augmented reality user device; and
inhibiting rendering of a virtual object in an augmented reality environment on the augmented reality user device unless the virtual object is on the maintained whitelist.

### Claim 18 (depends on 1)

The method of claim 1, comprising:
receiving signals from a plurality of radio frequency beacons at at least one radio frequency receiver of the augmented reality user device; and
performing spatial resolution operations and rendering of virtual objects in relation to a subset of the received signals and not performing spatial resolution operations and rendering of virtual objects in relation to received signals not in the subset.

### Claim 19

A system for use in processing data for an augmented reality environment, the system comprising:
at least one memory including computer program code; and
at least one processor in data communication with the at least one memory, wherein the at least one processor is configured to:
receive at least one signal from a radio frequency beacon at at least one radio frequency receiver of the augmented reality user device;
perform a spatial resolution operation in relation to the at least one received signal to determine a location of the radio frequency beacon; and
render one or more audio features of a virtual object in an augmented reality environment on the augmented reality user device in relation to the determined location in the augmented reality environment.

### Claim 20

A non-transitory computer-readable storage medium comprising computer-executable instructions which, when executed by a processor, cause an augmented reality device to perform a method, the method comprising:
receiving at least one signal from a radio frequency beacon at at least one radio frequency receiver of the augmented reality user device;
performing a spatial resolution operation in relation to the at least one received signal to determine a location of the radio frequency beacon; and
rendering a virtual object in an augmented reality environment on the augmented reality user device at least on the basis of the determined location,
wherein the rendering comprises rendering one or more audio features of the virtual object in relation to the determined location in the augmented reality environment.
