US 12,039,749 B2Granted

Low power visual tracking systems

AssigneeQUALCOMM Incorporated

InventorsRavishankar Sivalingam, Donald William Kidwell, Russell Gruhlke, Edwin Chongwoo Park, Khurshid Syed Alam

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems, methods, and non-transitory media are provided for low-power visual tracking systems. An example method can include receiving one or more images captured by each image sensor system from a set of image sensor systems on a first device, the one or more images capturing a set of patterns on a second device, wherein the first device has lower power requirements than the second device, the set of patterns having a predetermined configuration on the second device; determining, from the one or more images captured by each image sensor system, a set of pixels corresponding to the set of patterns on the second device; determining, based on the set of pixels corresponding to the set of patterns, a location and relative pose in space of each pattern; and determining, based on the location and relative pose of each pattern, a pose of the first device relative to the second device.

Claims30

Dependent claims

A method comprising:

receiving one or more images captured by at least one image sensor system on a first device, the one or more images depicting a set of patterns having a predetermined configuration on a second device and at least one reference line between the set of patterns, the predetermined configuration allowing disambiguation of at least one position of at least one pattern of the set of patterns relative to at least one other position of at least one other pattern of the set of patterns, wherein the first device has lower power requirements than the second device;
determining, from the one or more images captured by the at least one image sensor system, a set of pixels corresponding to the set of patterns on the second device;
determining a reference location for the at least one reference line between the set of patterns based on a reference coordinate system;
determining an observed location of the at least one reference line depicted in the one or more images with respect to an image coordinate system associated with the one or more images;
determining three-dimensional (3D) locations of the set of patterns at least in part by translating 3D coordinates of the set of pixels corresponding to the set of patterns based on a comparison between the reference location and the observed location of the at least one reference line, wherein a 3D location of each pattern of the set of patterns comprises a respective 3D location from the 3D locations of the set of patterns;
determining 3D orientations of the set of patterns at least in part by rotating the 3D coordinates of the set of pixels corresponding to the set of patterns based on the comparison between the reference location and the observed location of the at least one reference line, wherein a relative 3D pose in space of each pattern of the set of patterns is based on a respective 3D orientation from the 3D orientations of the set of patterns;
determining, based on a first orientation of a first pattern from the set of patterns in the predetermined configuration relative to a second orientation of a second pattern from the set of patterns in the predetermined configuration, that the first pattern is at a left, right, top, or bottom position in the predetermined configuration relative to the second pattern;
and
determining, based on the 3D location and the relative 3D pose in space of each pattern from the set of patterns and further based on the determining that the first pattern is at a left, right, top, or bottom position in the predetermined configuration relative to the second pattern, a pose of the first device relative to the second device.

6 more elements

An apparatus comprising:

one or more processors operably configured to:
receive one or more images captured by at least one image sensor system on the apparatus, the one or more images depicting a set of patterns having a predetermined configuration on a device and at least one reference line between the set of patterns, the predetermined configuration allowing disambiguation of at least one position of at least one pattern of the set of patterns relative to at least one other position of at least one other pattern of the set of patterns, wherein the apparatus has lower power requirements than the device;
determine, from the one or more images captured by the at least one image sensor system, a set of pixels corresponding to the set of patterns on the device;
determine a reference location for the at least one reference line between the set of patterns based on a reference coordinate system;
determine an observed location of the at least one reference line depicted in the one or more images with respect to an image coordinate system associated with the one or more images;
determine 3D locations of the set of patterns at least in part by translating 3D coordinates of the set of pixels corresponding to the set of patterns based on a comparison between the reference location and the observed location of the at least one reference line, wherein a 3D location of each pattern of the set of patterns comprises a respective 3D location from the 3D locations of the set of patterns;
determine 3D orientations of the set of patterns at least in part by rotating the 3D coordinates of the set of pixels corresponding to the set of patterns based on the comparison between the reference location and the observed location of the at least one reference line, wherein a relative 3D pose in space of each pattern of the set of patterns is based on a respective 3D orientation from the 3D orientations of the set of patterns;
determine, based on a first orientation of a first pattern from the set of patterns in the predetermined configuration relative to a second orientation of a second pattern from the set of patterns in the predetermined configuration, that the first pattern is at a left, right, top, or bottom position in the predetermined configuration relative to the second pattern;
and
determine, based on the 3D location and the relative 3D pose in space of each pattern from the set of patterns and further based on the determining that the first pattern is at a left, right, top, or bottom position in the predetermined configuration relative to the second pattern, a pose of the apparatus relative to the device.

7 more elements

A non-transitory computer-readable storage medium of a first device comprising:

instructions stored thereon which, when executed by one or more processors, cause the one or more processors to:
receive one or more images captured by at least one image sensor system on the first device, the one or more images depicting a set of patterns having a predetermined configuration on a second device and at least one reference line between the set of patterns, the predetermined configuration allowing disambiguation of at least one position of at least one pattern of the set of patterns relative to at least one other position of at least one other pattern of the set of patterns, wherein the first device has lower power requirements than the second device;
determine, from the one or more images captured by the at least one image sensor system, a set of pixels corresponding to the set of patterns on the second device;
determine a reference location for the at least one reference line between the set of patterns based on a reference coordinate system;
determine an observed location of the at least one reference line depicted in the one or more images with respect to an image coordinate system associated with the one or more images;
determine 3D locations of the set of patterns at least in part by translating 3D coordinates of the set of pixels corresponding to the set of patterns based on a comparison between the reference location and the observed location of the at least one reference line, wherein a 3D location of each pattern of the set of patterns comprises a respective 3D location from the 3D locations of the set of patterns;
determine 3D orientations of the set of patterns at least in part by rotating the 3D coordinates of the set of pixels corresponding to the set of patterns based on the comparison between the reference location and the observed location of the at least one reference line, wherein a relative 3D pose in space of each pattern of the set of patterns is based on a respective 3D orientation from the 3D orientations of the set of patterns;
determine, based on a first orientation of a first pattern from the set of patterns in the predetermined configuration relative to a second orientation of a second pattern from the set of patterns in the predetermined configuration, that the first pattern is at a left, right, top, or bottom position in the predetermined configuration relative to the second pattern;
and
determine, based on the 3D location and the relative 3D pose in space of each pattern from the set of patterns and further based on the determining that the first pattern is at a left, right, top, or bottom position in the predetermined configuration relative to the second pattern, a pose of the first device relative to the second device.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/904,466Filed 17 Jun 2020
Published: US 2021/0398314 A123 Dec 2021
Granted: 16 Jul 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification51 codes

G06T 7/73
Determining position or orientation of objects or cameras: using feature-based methods
G06F 3/012
Arrangements for interaction with the human body, e.g. for user immersion in virtual reality: head tracking input arrangements
G06F 3/0346
Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor: with detection of the…
H04L 12/4633
Interconnection of networks: interconnection of networks using encapsulation techniques, e.g. tunneling