# US 11,047,737 B2 · Temporal-spectral multiplexing sensor and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,047,737 B2 - Document ID: US11047737B2 - Status: Granted - Kind: B2 - Application number: 16/661,254 - Filed: 2019-10-23 - Published: 2021-04-29 - Granted: 2021-06-29 - Earliest claimed: 2019-10-23 - Assignees: Spectral Sciences, Inc. - Applicants: Spectral Sciences, Inc. - Inventors: Robert M. Shroll; Pajo Vujkovic-Cvijin; Jamine Lee; Neil Goldstein; Marsha Fox; Michael Kogan - Web page: https://patentgrep.com/patent/US11047737B2 ## CPC - G01J 3/2823: {Imaging spectrometer} - G01J 3/0229: {using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters} - G01J 3/027: {Control of working procedures of a spectrometer; Failure detection; Bandwidth calculation} - G01J 3/0294: {Multi-channel spectroscopy} - G01J 3/14: using refracting elements, e.g. prisms (G01J3/18, G01J3/26 take precedence {prisms per se G02B5/04}) - G01J 3/18: using diffraction elements, e.g. grating (gratings per se G02B) - G01J 3/2803: {using photoelectric array detector} - G01J 3/2846: {using modulation grid; Grid spectrometers} - G01J 3/32: Investigating bands of a spectrum in sequence by a single detector - G01J 3/36: Investigating two or more bands of a spectrum by separate detectors - G01J 2003/1204: {Grating and filter} - G01J 2003/1213: {Filters in general, e.g. dichroic, band} - G01J 2003/1278: {Mask with spectral selection} - G01J 2003/2813: {2D-array} - G01J 2003/2826: {Multispectral imaging, e.g. filter imaging} - G01J 2003/2843: {Processing for eliminating interfering spectra} - G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N) ## Abstract A temporal-spectral multiplexing sensor for simultaneous or near simultaneous spatial-temporal-spectral analysis of an incoming optical radiation field. A spectral encoder produces a time series of spectrally encoded optical images at a high sampling rate. A series of full panchromatic spectrally encoded optical images are collected at a rate similar to the sampling rate. A detector records at least one spatial region of the spectrally encoded optical image. A processor is configured to process two series of spectrally encoded optical images to produce an artifact-free spectral image. The processing includes using the panchromatic images to normalize the spectrally encoded images, and decoding the normalized encoded images to produce high fidelity spectral signatures, free of temporal artifacts due to fluctuations at frequencies slower than the sampling rate for polychromatic images. ## Claims ### Claim 1 A temporal-spectral multiplexing sensor for simultaneous or near simultaneous spatial-temporal-spectral analysis of an incoming optical radiation field, comprising: a spectral encoder that produces a time series of spectrally encoded optical images at a high sampling rate; means of collecting a series of full panchromatic spectrally encoded optical images at a rate similar to the sampling rate, a detector that records at least one spatial region of the spectrally encoded optical image; and a processor that is configured to process two series of spectrally encoded optical images to produce an artifact-free spectral image, wherein the processor is configured to: use the panchromatic images to normalize the spectrally encoded images, and decode the normalized encoded images to produce high fidelity spectral signatures, free of temporal artifacts due to fluctuations at frequencies slower than the sampling rate for polychromatic images. ### Claim 2 (depends on 1) The temporal-spectral multiplexing sensor of claim 1, where simultaneous spatial-temporal analysis comprises the co-processing of near-simultaneous and co-registered hypertemporal and hyperspectral imagery to monitor, detect, identify, or characterize objects or events based on the combination of spectral and temporal signatures. ### Claim 3 (depends on 2) The temporal-spectral multiplexing sensor of claim 2, wherein the panchromatic signal is analyzed for its temporal content in order to reveal the frequency components of temporal signal oscillations due to changes in the object scene, such as mechanical vibrations or source emission changes, or to suppress line-of-sight motion induced signal variance. ### Claim 4 (depends on 2) The temporal-spectral multiplexing sensor of claim 2, wherein the image-to-image correlation between panchromatic images are used to generate a clutter mitigation projection operator that corrects the panchromatic and spectral imagery by projecting the images into a clutter-free subspace. ### Claim 5 (depends on 2) The temporal-spectral multiplexing sensor of claim 2, wherein object detection and/or characterization is based on combined HTI/HSI signatures, and where the analysis is performed by calculating power spectral density. ### Claim 6 (depends on 2) The temporal-spectral multiplexing sensor of claim 2, wherein at least one of object detection or characterization is based on combined HTI/HSI signatures, and where the analysis is performed by principal component analysis. ### Claim 7 (depends on 2) The temporal-spectral multiplexing sensor of claim 2, wherein at least one of object detection or characterization is based HTI analysis of a specific spectral band or combination of spectral bands. ### Claim 8 (depends on 7) The temporal-spectral multiplexing sensor of claim 7, wherein at least one of object detection or characterization comprises atmospheric absorption compensation for intensity-varying backgrounds. ### Claim 9 (depends on 2) The temporal-spectral multiplexing sensor of claim 2, wherein at least one of object detection or characterization comprises whitening of the spectral content prior to application of a detection operator, such as the ACE (Adaptive Cosine Estimator), or other projection operators. ### Claim 10 (depends on 2) The temporal-spectral multiplexing sensor of claim 2, wherein at least one of object detection or characterization comprises whitening of the temporal content prior to application of a detection operator. ### Claim 11 (depends on 2) The temporal-spectral multiplexing sensor of claim 2 wherein the panchromatic image stream is analyzed for its temporal content in order to reveal temporal frequency components of the signal, to suppress line-of-sight motion induced signal variance effect on HSI data fidelity, to capture high fidelity spectral signatures from targets with temporally unstable radiance or to perform object detection and/or characterization based on combined HTI/HSI signatures. ### Claim 12 (depends on 1) The temporal-spectral multiplexing sensor of claim 1, where the spectral encoder comprises: a system of alternately applying a selected encoding transform and the complement of the selected encoding transform simultaneously or sequentially; and the panchromatic images are produced by summation of the signals generated by the encoding transform with the signal generated by the complement of the encoding transform. ### Claim 13 (depends on 12) The temporal-spectral multiplexing sensor of claim 12, where the encoded image is captured by one detector and the complement of the encoded image is captured by another detector. ### Claim 14 (depends on 1) The temporal-spectral multiplexing sensor of claim 1, where the means of collecting the panchromatic image includes alternating spectrally encoded images with un-encoded, panchromatic images. ### Claim 15 (depends on 14) The temporal-spectral multiplexing sensor of claim 14, where the spectral encoder applies a series of spectral band pass filters. ### Claim 16 (depends on 1) The temporal-spectral multiplexing sensor of claim 1, wherein using the panchromatic images to normalize the spectrally encoded images comprises applying a clutter mitigation projection operator, generated from the panchromatic images to the spectrally encoded images prior to decoding. ### Claim 17 (depends on 1) The temporal-spectral multiplexing sensor of claim 1 that allows capture high fidelity spectral signatures from targets with temporally unstable radiance. ### Claim 18 (depends on 1) The temporal-spectral multiplexing sensor of claim 1 where the spectral encoder comprises: a first polychromator that disperses the light to form a dispersed spectral image on a spatial light modulator, encoding a set of spectral bands by actuating specific areas of the spatial light modulator, a second polychromator that is configured to recombine the light; and a detector that is configured to form a spectrally encoded image. ### Claim 19 (depends on 1) The temporal-spectral multiplexing sensor of claim 1 where the detector comprises a single-element detector and captures a single spatial element. ### Claim 20 (depends on 1) The temporal-spectral multiplexing sensor of claim 1 where the detector comprises a linear array of optical detectors and captures a linear array of spatial elements. ### Claim 21 (depends on 1) The temporal-spectral multiplexing sensor of claim 1 where the detector comprises a two-dimensional array of optical detectors which capture a two-dimensional image.