# US 10,373,339 B2 · Hyperspectral scene analysis via structure from motion > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,373,339 B2 - Document ID: US10373339B2 - Status: Granted - Kind: B2 - Application number: 15/379,166 - Filed: 2016-12-14 - Published: 2017-06-15 - Granted: 2019-08-06 - Earliest claimed: 2015-12-14 - Assignees: The Government of the United States of America, as represented by the Secretary of the Navy - Applicants: The United States of America, as represented by the Secretary of the Navy - Inventors: Corey A. Miller; Thomas J. Walls - Web page: https://patentgrep.com/patent/US10373339B2 ## CPC - G06T 7/90: Determination of colour characteristics - G01J 3/00: Spectrometry; Spectrophotometry; Monochromators; Measuring colours - G06T 7/579: from motion - G06V 20/176: {Urban or other man-made structures} - G06T 2207/10036: Multispectral image; Hyperspectral image - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method for determining structure from motion in hyperspectral imaging includes acquiring hyperspectral data cubes containing intensity data, the intensity data being stored in dimensions of the hyperspectral data cube including a first spatial dimension, a second spatial dimension, and a spectrum dimension; establishing a set of baseline spectral features from a data cube for tracking between data cubes; establishing a set of standard features from a data cube for tacking between data cubes; matching, between data cubes, respective baseline features and standard features; and extracting imaging device motion information based on relative positions of matched baseline and standard features. ## Claims ### Claim 1 A method for determining structure from motion in hyperspectral imaging, the method including the steps of: acquiring a sequence of hyperspectral data cubes taken by an imaging device, each hyperspectral data cube containing intensity data, the intensity data being stored in dimensions of the hyperspectral data cube including a first spatial dimension, a second spatial dimension, and a spectrum dimension; establishing a set of baseline spectral features from a data cube for tracking between data cubes; establishing a set of standard features from a data cube for tracking between data cubes; matching, between data cubes, respective baseline features and standard features; extracting imaging device motion information based on relative positions of matched baseline and standard features; determining a baseline pair of data cubes; determining positions of respective baseline and standard features relative to the imaging device based on triangulation of imaging device motion between the baseline pair of data cubes; iteratively building a sparse structural reconstruction of the scene with a 3D point cloud of features by adding positions of baseline and standard features from additional data cubes. ### Claim 2 (depends on 1) The method of claim 1, wherein the step of establishing a set of baseline features includes detecting anomalies in the intensity data by measuring spectral differences between spatial regions of the data cube. ### Claim 3 (depends on 2) The method of claim 2, wherein detecting anomalies in the intensity data includes performing a Reed-Xiaoli Detector (RXD) algorithm on each data cube, and comparing the output of the RXD algorithm to a predetermined threshold to select a subset of most spectrally unique points in the data cube. ### Claim 4 (depends on 1) The method of claim 1, wherein the step of establishing a set of standard features includes: flattening each data cube into a false-color RGB representation; performing on the false-color RGB representation and on three monochromatic images from the data cube a feature detection and extraction routine to determine the set of standard features. ### Claim 5 (depends on 4) The method of claim 4, wherein flattening each data cube includes selecting three individual bands to represent each of three RGB channels. ### Claim 6 (depends on 4) The method of claim 4, wherein flattening each data cube includes averaging over a range of bands to generate each of three RGB channels. ### Claim 7 (depends on 1) The method of claim 1, wherein the step of matching includes using both a brute-force feature matcher routine with additional spectral verification, providing a two-factor verification that reduces mismatched features between data cubes. ### Claim 8 (depends on 1) The method of claim 1, wherein the step of matching includes matching baseline spectral features using a nearest-neighbor routine and computing spectral similarity of the baseline features via a cosine similarity measure. ### Claim 9 (depends on 1) The method of claim 1, further comprising the step of: building a set of feature tracks that each follow a respective baseline feature across several data cubes in the sequence of data cubes. ### Claim 10 (depends on 9) The method of claim 9, wherein the step of extracting imaging device motion includes relating motion of the feature tracks across data cubes to motion of the imaging device. ### Claim 11 (depends on 1) The method of claim 1, wherein the step of determining a base pair of data cubes comprises: stepping through each pair of data cubes and determining which pair has the highest percentage of accurately-projected features. ### Claim 12 (depends on 11) The method of claim 11, wherein the step of determining a baseline pair of data cubes includes measuring accuracy based on percentage of points with positive z-axis projections relative to a focal plane of the imaging device combined with average error of projected features as the projected features are reprojected back onto the data cubes.