# US 2018/0321776 A1 · Method for acting on augmented reality virtual objects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0321776 A1
- Document ID: US20180321776A1
- Status: Published application
- Kind: A1
- Application number: 15/773,248
- Filed: 2016-11-17
- Published: 2018-11-08
- Earliest claimed: 2015-11-18
- Assignees: Devar Entertainment Limited
- Applicants: Devar Entertainment Limited
- Inventors: Vitaly Vitalyevich AVERYANOV; Andrey KOMISSAROV
- Web page: https://patentgrep.com/patent/US20180321776A1

## CPC

- G06F 3/048: Interaction techniques based on graphical user interfaces [GUI]
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/277: involving stochastic approaches, e.g. using Kalman filters
- G06T 7/73: using feature-based methods
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30204: Marker
- G06T 2207/30241: Trajectory
- G06T 2207/30244: Camera pose
- G06T 2219/2016: Rotation, translation, scaling
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The invention relates to methods for acting on augmented reality virtual objects. According to the invention, the coordinates of a device for creating and viewing augmented reality are determined in relation to a real-world physical marker by means of analysis of an image from a camera of the device; a virtual camera is positioned in calculated coordinates of the device in relation to a physical base coordinate system in such a way that the marker, which is visible to the virtual camera, is positioned in the field of vision thereof, just as the physical marker is positioned in the field of vision of the device camera; a vector is calculated, which corresponds to a direction from the marker to the virtual camera in real-time mode; information is generated relating to all of the movements of the camera in relation to the marker, i.e. rotation, zoom, tilt.

## Claims

### Claim at position 1

-6. (canceled)

### Claim 7

A method for influencing virtual objects of augmented reality, said method comprising the following steps:
obtaining images of a real three-dimensional space by a camera of a device adapted to create and view augmented reality,
identifying one or more markers of the real three-dimensional space based upon said images obtained by the camera,
forming a base coordinate system tied to a spatial position of the markers of the real three-dimensional space,
determining coordinates of the device adapted to create and view augmented reality relative to the base coordinate system,
specifying coordinates of virtual objects of augmented reality in the base coordinate system,
modifying at least some of the virtual objects of augmented reality based upon an identified motion of a user,
setting a virtual camera in the determined coordinates of the device adapted to create and view augmented reality relative to the base coordinate system so that a virtual marker visible by the virtual camera is located in a field of view of the virtual camera in the same way that a marker of the real three-dimensional space is located in a field of view of the camera of the device adapted to create and view augmented reality,
calculating a vector corresponding to a direction from the virtual marker to the virtual camera in real time using quaternions, and
generating information about all movements of the virtual camera relative to the markers of the real three-dimensional space, said movements comprising rotation, approximation, and tilt, by sequential iteration and in real time.

### Claim 8 (depends on 7)

The method of claim 7, wherein the quaternions of the vector corresponding to the direction from the virtual marker to the virtual camera are calculated using at least one or more coordinates and one or more angles

### Claim 9 (depends on 7)

The method of claim 7, wherein the quaternions of the vector corresponding to the direction from the virtual marker to the virtual camera are calculated using polar coordinates.

### Claim 10 (depends on 7)

The method of claim 7, wherein the quaternions of the vector corresponding to the direction from the virtual marker to the virtual camera are calculated using a Euler angle method.

### Claim 11 (depends on 7)

The method of claim 7, further comprising generating information regarding all movements of the virtual camera relative to the markers of the real three-dimensional space by analyzing a video stream received from the camera of the device adapted to create and view augmented reality.

### Claim 12 (depends on 11)

The method of claim 11, wherein an analysis of an image from the camera of the device adapted to create and view augmented reality is performed by an algorithm for searching for points of interest.

### Claim 13 (depends on 11)

The method of claim 11, wherein an analysis of an image from the camera of the device adapted to create and view augmented reality is performed by an image classifier algorithm.

### Claim 14 (depends on 11)

The method of claim 11, wherein an analysis of an image from the camera of the device adapted to create and view augmented reality is performed by a Kalman Filter algorithm.

### Claim 15 (depends on 11)

The method of claim 11, wherein an analysis of an image from the camera of the device adapted to create and view augmented reality is performed by an Image Pyramid algorithm.
