# US 2019/0170660 A1 · Method and system of detecting foreign materials within an agricultural product stream > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0170660 A1 - Document ID: US20190170660A1 - Status: Published application - Kind: A1 - Application number: 16/266,530 - Filed: 2019-02-04 - Published: 2019-06-06 - Earliest claimed: 2016-10-10 - Assignees: Altria Client Services LLC - Applicants: Altria Client Services LLC - Inventors: Amrita Sahu; Henry M. Dante; Jerry W. Morris - Web page: https://patentgrep.com/patent/US20190170660A1 ## CPC - G01N 21/94: Investigating contamination, e.g. dust (G01N21/85 takes precedence) - B07C 5/3422: {using video scanning devices, e.g. TV-cameras} - B07C 5/3427: {by changing or intensifying the optical properties prior to scanning, e.g. by inducing fluorescence under UV or x-radiation, subjecting the material to a chemical reaction} - G01N 21/3563: for analysing solids; Preparation of samples therefor - G01N 21/359: using near infrared light - G01N 21/85: Investigating moving fluids or granular solids - B07C 2501/0081: Sorting of food items - B07C 2501/009: Sorting of fruit - G01N 2021/845: {Objects on a conveyor} - G01N 2021/8455: {and using position detectors} - G01N 2021/8466: {Investigation of vegetal material, e.g. leaves, plants, fruits} - G01N 2021/8592: {Grain or other flowing solid samples} - G01N 2201/10: Scanning - G01N 2201/127: Calibration; base line adjustment; drift compensation - G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q) - B07C: POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING (specially adapted for a specific purpose covered by another class, see the relevant place, e.g. A43D33/06, B23Q7/12) ## Abstract Methods and systems for detecting foreign material within a product stream in real-time, involve illuminating a portion of the agricultural product stream with light spanning a wavelength range including or within near-infrared and/or shortwave infrared wavelengths; scanning a line of the illuminated agricultural product stream to acquire a hyperspectral image of the line, the hyperspectral image of the line having a width of a single pixel; processing the hyperspectral image of the scanned line to obtain spectrum data for one or more pixels of the hyperspectral image of the scanned line; comparing the obtained spectrum data of the one or more pixels to predetermined spectrum data to determine whether the obtained spectrum data is indicative of foreign material within the scanned line of the agricultural product stream. ## Claims ### Claim 1 A method of detecting foreign material within an agricultural product stream in real-time, the method comprising: illuminating a portion of the agricultural product stream with light spanning a wavelength range including or within near-infrared and/or shortwave infrared wavelengths; scanning a line of the illuminated agricultural product stream to acquire a hyperspectral image of the line, the hyperspectral image of the line having a width of a single pixel; processing the hyperspectral image of the scanned line to obtain spectrum data for one or more pixels of the hyperspectral image of the scanned line; and comparing the obtained spectrum data of the one or more pixels to predetermined spectrum data to determine whether the obtained spectrum data is indicative of foreign material within the scanned line of the agricultural product stream without constructing a hyperspectral image cube. ### Claim 2 (depends on 1) The method of claim 1, wherein the scanned line of agricultural product stream is generally perpendicular to a direction in which the agricultural product stream is being moved. ### Claim 3 (depends on 1) The method of claim 1, wherein the predetermined spectrum data is a calculated value based upon spectral data associated with tobacco, foam, cardboard, plastic, foil, lubricant, paper, a sub-combination thereof, or a combination thereof. ### Claim 4 (depends on 1) The method of claim 1, further comprising a step of: determining, based upon the comparing step, the location of foreign material in the product stream; or determining, based upon the comparing step, that the scanned line of the agricultural product stream does not contain foreign material. ### Claim 5 (depends on 1) The method of claim 1, wherein the processing step comprises calculating the Euclidean distance associated with one or more pixels of the hyperspectral image of the scanned line, wherein the Euclidean distance is calculated by the formula d  ( p , q ) = ∑ i = 1 n   ( q i - p i ) 2 , wherein p=(p1,p2 . . . , pn) and q=(q1,q2 . . . , qn) and p and q are two points in Euclidean n-space. ### Claim 6 (depends on 1) The method of claim 1, wherein the scanning step includes sensing a spectrum of light reflected, scattered, and/or emitted from the agricultural product stream with at least one sensor; wherein: (a) the scanning is performed by a line-scan hyperspectral imaging device that includes the at least one sensor; (b) the at least one sensor is an Indium Gallium Arsenide (InGaAs) sensor; and/or (c) the at least one sensor is operable to sense light in a range of about 900 nm to 1700 nm or about 900 nm to 2500 nm. ### Claim 7 (depends on 1) The method of claim 1, wherein: (a) the scanning is performed by a line-scan hyperspectral imaging device wherein the hyperspectral imaging device operates in the range of about 900 nm to 1700 nm or about 900 nm to 2500 nm; and/or (b) the illuminating is performed by a tungsten halogen light source. ### Claim 8 (depends on 1) The method of claim 1, wherein the agricultural product stream is a continuously moving agricultural product stream and is conveyed on a conveyer belt operating at a speed of about 80-100 ft/min. ### Claim 9 (depends on 1) The method of claim 1, wherein multiple lines of agricultural product stream are individually scanned consecutively, and the consecutively scanned lines are separately processed in-real time. ### Claim 10 (depends on 1) The method of claim 1, wherein the processing step comprises: calibrating the hyperspectral image of the scanned line by removing dark values of the hyperspectral image; normalizing the hyperspectral image of the scanned line; and smoothing the hyperspectral image of the scanned line to eliminate noise using median filtering. ### Claim 11 (depends on 1) The method of claim 1, wherein the predetermined spectrum data is a threshold value associated with a single product material, a class of product materials, a single known foreign material, a class of known foreign materials, a sub-combination thereof, or a combination thereof. ### Claim 12 (depends on 1) The method of claim 1 further comprising: generating an activation signal in response to determining that the scanned line of agricultural product stream comprises foreign material; transmitting the activation signal to a removal system; and removing the foreign material from the scanned line. ### Claim 13 (depends on 12) The method of claim 12, wherein the foreign material is removed by a vacuum device and/or a forced fluid stream. ### Claim 14 A system for detecting foreign material within an agricultural product stream in real-time, the system comprising: an illumination device configured to illuminate a portion of the agricultural product stream at a wavelength range including or within near-infrared and/or shortwave infrared wavelengths; a hyperspectral imaging device configured to scan a line of the illuminated agricultural product stream to acquire a hyperspectral image of the line, the hyperspectral image of the line having a width of a single pixel; and a processor configured to: process the hyperspectral image of the scanned line to obtain spectrum data for one or more pixels of the hyperspectral image of the scanned line, and compare the obtained spectrum data of the one or more pixels to predetermined spectrum data to determine whether the obtained spectrum data is indicative of foreign material within the scanned line of the agricultural product stream without constructing a hyperspectral image cube. ### Claim 15 (depends on 14) The system of claim 14, wherein the processor is further configured to: determine, based upon the compared spectrum data, the location of foreign material in the product stream; or determine, based upon the compared spectrum data, that the scanned line of the agricultural product stream does not contain foreign material. ### Claim 16 (depends on 1) A non-transitory computer readable medium having instructions stored thereon, the instructions being configured to cause a processor to execute the steps of the method of claim 1. ### Claim 17 A method of detecting foreign material within stationary agricultural product in real-time, the method comprising: illuminating a portion of the stationary agricultural product with light spanning a wavelength range including or within near-infrared and/or shortwave infrared wavelengths; moving a hyperspectral imaging device over a portion of the stationary agricultural product while simultaneously scanning a line of the illuminated stationary agricultural product to acquire a hyperspectral image of the line, the hyperspectral image of the line having a width of a single pixel; processing the hyperspectral image of the scanned line to obtain spectrum data for one or more pixels of the hyperspectral image of the scanned line; and comparing the obtained spectrum data of the one or more pixels to predetermined spectrum data to determine whether the obtained spectrum data is indicative of foreign material within the scanned line of the stationary agricultural product without constructing a hyperspectral image cube.