# US 2013/0156264 A1 · Minimizing drift using depth camera images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2013/0156264 A1 - Document ID: US20130156264A1 - Status: Published application - Kind: A1 - Application number: 13/688,741 - Filed: 2012-11-29 - Published: 2013-06-20 - Earliest claimed: 2011-12-15 - Assignees: Sony Corporation; Sony Mobile Communications Inc. - Inventors: Linus Mårtensson - Web page: https://patentgrep.com/patent/US20130156264A1 ## CPC - G06T 7/73: using feature-based methods - G06T 2207/10012: Stereo images - G06T 2207/30244: Camera pose - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A device may obtain, from a camera associated with a reference object, depth image data including objects in a first frame and a second frame; identify features of the objects in the first frame and the second frame; and track movements of the features between the first frame and the second frame. The device may also identify independently moving features in the second frame, based on the tracking movements; remove the independently moving features from the depth image data to obtain a static feature set; and process the depth image data corresponding to the static feature set to detect changes in the relative position of objects in the first frame and the second frame. The processor may further translate the changes in relative position into corresponding movement data of the camera and provide the corresponding movement data to an inertial navigation system. ## Claims ### Claim 1 A method, comprising: receiving, by a computing device and from a depth camera associated with a reference object, depth image data including objects in a first frame and in a second frame; processing, by the computing device, the depth image data to detect changes in relative positions of the objects in the first frame and in the second frame; translating, by the computing device, the changes in relative positions into corresponding movement data of the depth camera; obtaining, by the computing device, sensor data associated with the reference object; filtering, by the computing device, the corresponding movement data and the sensor data; and calculating, by the computing device, a geographic position of the reference object based on the corresponding movement data and the sensor data. ### Claim 2 (depends on 1) The method of claim 1, wherein processing the depth image data comprises: detecting edges of the objects in the first frame, defining triangles between portions of edges within the first frame, detecting edges of the objects in the second frame, defining triangles between the portions of the edges within the second frame, and calculating a mean transformation over the triangles of the first frame and the triangles of the second frame to determine the changes in the relative position. ### Claim 3 (depends on 1) The method of claim 1, wherein processing the depth image data comprises: defining a feature set based on the first frame and the second frame, identify common features in the feature set that are in both the first frame and the second frame, tracking relative movement, of the common features, between the first frame and the second frame, and calculating a mean transformation of the relative movement to determine the changes in the relative position. ### Claim 4 (depends on 3) The method of claim 3, wherein processing the depth image data further comprises: removing, from the feature set, features associated with non-static objects. ### Claim 5 (depends on 1) The method of claim 1, wherein processing the depth image data comprises: detecting features of the objects in the first frame, defining a feature set based on the features of the objects in the first frame, detecting features of the objects in the second frame, and updating the feature set based on the features of the objects in the second frame. ### Claim 6 (depends on 5) The method of claim 5, wherein processing the depth image data further comprises: updating the feature set based on the calculated geographic position of the reference object. ### Claim 7 (depends on 1) The method of claim 1, wherein the computing device includes an inertial navigation system (INS). ### Claim 8 (depends on 1) The method of claim 1, wherein the depth camera associated with the reference object is included in an augmented reality (AR) visor. ### Claim 9 (depends on 1) The method of claim 1, wherein the sensor data associated with the reference object includes: accelerometer data, gyroscope data, or magnetometer data. ### Claim 10 (depends on 1) The method of claim 1, wherein processing the depth image data comprises: extracting, from the depth image data, objects that are moving relative to the reference object and objects that are occluded in the second frame. ### Claim 11 A device comprising: a memory to store a plurality of instructions; and a processor configured to execute the instructions in the memory to: obtain, from a camera associated with a reference object, depth image data including objects in a first frame and a second frame, identify features of the objects in the first frame and in the second frame, track movements of the features between the objects in the first frame and in the second frame; identify independently moving features in the second frame, based on the tracking movements, remove the independently moving features from the depth image data to obtain a static feature set, process the depth image data corresponding to the static feature set to detect changes in the relative position of objects in the first frame and the second frame, translate the changes in relative position into corresponding movement data of the camera, and provide, by the computing device, the corresponding movement data of the camera to an inertial navigation system. ### Claim 12 (depends on 11) The device of claim 11, wherein the device comprises: a smart phone, a tablet computer, or an augmented reality (AR) visor. ### Claim 13 (depends on 11) The device of claim 11, wherein the processor is further configured to execute instructions in the memory to: obtain sensor data associated with the reference object, and calculate a geographic position of the reference object based on the corresponding movement data and the sensor data. ### Claim 14 (depends on 13) The device of claim 13, wherein the processor is further configured to execute instructions in the memory to: filter the corresponding movement data and the sensor data using a Kalman filter. ### Claim 15 (depends on 13) The device of claim 13, wherein the sensor data associated with the reference object comprises at least one of: accelerometer data, gyroscope data, and magnetometer data. ### Claim 16 (depends on 13) The device of claim 13, wherein the processor is further configured to execute instructions in the memory to: translate the static feature set to reflect geographic positions consistent with the calculated geographic position of the reference object. ### Claim 17 (depends on 11) The device of claim 11, wherein the processor is further configured to execute instructions in the memory to: identify occluded features in the second frame, based on the tracking movements, and remove the occluded features from the depth image data. ### Claim 18 (depends on 11) The device of claim 11, further comprising one of: p1 a Bluetooth interface, a near-field communications (NFC) interface, a WiFi interface, or a wired interface for communicating with the camera. ### Claim 19 A position-determining device, comprising: a depth camera to generate depth images relative to a reference object; a sensor to generate sensor data, wherein the sensor data includes one or more of motion data, rotation data, and magnetic field data relative to the reference object; a transmitter to send the depth images and the sensor data to a processing component; a receiver to receive the depth images and the sensor data; and the processing component being configured to: detect changes in relative positions of objects in the depth images, translate the changes in relative positions into corresponding movement data of the camera, and calculate a position of the reference object based on the sensor data and the calculated corresponding movement data of the camera. ### Claim 20 (depends on 19) The position-determining device of claim 19, wherein the processing component includes an inertial navigation system (INS).