# US 2021/0343040 A1 · Object tracking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0343040 A1
- Document ID: US20210343040A1
- Status: Published application
- Kind: A1
- Application number: 17/271,241
- Filed: 2019-08-29
- Published: 2021-11-04
- Earliest claimed: 2018-09-03
- Assignees: University of Northumbria at Newcastle
- Applicants: University of Northumbria At Newcastle
- Inventors: Lars Erik Holmquist
- Web page: https://patentgrep.com/patent/US20210343040A1

## CPC

- G06T 7/74: {involving reference images or patches}
- G06T 7/75: {involving models}
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30204: Marker
- G06T 2207/30208: Marker matrix
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method of tracking an object. The method comprises imaging a marker on an object surface using a depth imaging camera, said marker comprising a three-dimensional pattern; generating a representation of the marker; matching the representation of the marker to a representation of a reference three-dimensional pattern in a reference orientation; comparing the representation of the marker to the representation of the reference three-dimensional pattern, and thereby determining a position and orientation of the object relative to the camera.

## Claims

### Claim 1

A method of tracking an object comprising:
imaging a marker on an object surface using a depth imaging camera, said marker comprising a three-dimensional pattern;
generating a representation of the marker;
matching the representation of the marker to a representation of a reference three-dimensional pattern in a reference orientation;
comparing the representation of the marker to the representation of the reference three-dimensional pattern, and thereby
determining a position and orientation of the object relative to the camera.

### Claim 2 (depends on 1)

A method of tracking an object according to claim 1, further comprising
generating and displaying on a display a graphical element in dependence on the determined position and orientation of the object.

### Claim 3 (depends on 2)

A method according to claim 2, wherein generating and displaying on the display the graphical element in dependence on the determined position and orientation of the object, comprises displaying one or more of a position, size and orientation of the graphical element on the display in dependence on the determined position and orientation of the object.

### Claim 4 (depends on 2)

A method according to claim 2, comprising
capturing images of a scene including the object with a two-dimensional camera, and
displaying the images of the scene on the display.

### Claim 5 (depends on 4)

A method according to claim 4, comprising displaying the graphical element on the display in a position corresponding to the position of the object in the scene, thereby superimposing the graphical element over the object in the image of the scene.

### Claim 6 (depends on 2)

A method according to claim 2, wherein the graphical element is a representation of a three-dimensional object.

### Claim 7 (depends on 2)

A method according to claim 2, wherein the marker is one of a plurality of markers, said method comprising
identifying to which marker from the plurality of markers the marker on the object surface corresponds.

### Claim 8 (depends on 7)

A method according to claim 7, wherein each of the plurality of markers is associated with an object, said method further comprising
identifying the object based on the identification of the marker from the plurality of markers.

### Claim 9 (depends on 7)

A method according to claim 7, wherein the graphical element is one of a plurality of graphical elements, said method further comprising, responsive to identifying which marker from the plurality of markers the marker on the object surface corresponds,
selecting the graphical element from the plurality of graphical elements.

### Claim 10 (depends on 1)

A method according to claim 1, comprising imaging one or more further markers on the object surface using the depth imaging camera, said further markers comprising a three-dimensional pattern;
generating a representation of each of the further markers;
matching the representations of the further markers to a representation of a reference three-dimensional pattern in a reference orientation;
comparing the representations of the further markers to the representation of the reference three-dimensional pattern, and thereby
further determining a position and orientation of the object relative to the camera.

### Claim 11 (depends on 10)

A method according to claim 10, comprising
identifying on the marker and the one or more further markers an identifying mark uniquely identifying that marker, and
determining a position of the marker and one or more further markers on the surface of the object based on the identifying mark on each of the marker and one or more further markers.

### Claim 12 (depends on 1)

A method according to claim 1, wherein the marker is a three-dimensional grid comprising grid elements, wherein one or more of the grid elements are raised grid elements.

### Claim 13 (depends on 2)

A method according to claim 2, wherein the display is a display of a user device.

### Claim 14 (depends on 1)

An object tracking system comprising a depth imaging camera, a processor and a memory adapted to perform a method according to claim 1.

### Claim 15 (depends on 14)

An object tracking system according to claim 14, wherein the object tracking device is provided by a user device.

### Claim 16 (depends on 15)

An object tracking system according to claim 15, wherein the user device is a smartphone.

### Claim 17 (depends on 1)

A computer program comprising computer implementable instructions which when implemented on a processor of an object tracking device controls the processor to perform a method according to claim 1.
