Integrated hyperspectral water quality analysis method

An integrated hyperspectral water quality analysis method, wherein the integrated hyperspectral water quality analysis method comprises the following steps:

• step 1:
• water quality data collection and water quality image collection in early stage;
• water quality data and water quality images required for the establishment of hyperspectral water quality inversion models are prepared in early stage;
• multiple points at different positions and depths in a collection area are selected as required, and a hyperspectral camera is used to collect spectral data of different points;
• at the same time, a sensor and a chemical method are used to acquire water quality parameters of the multiple points at different positions and depths in the collection area;
• step 2:
• data preprocessing;
• the water quality images and spectral data collected at different points are preprocessed, wherein the preprocessing includes reflectance correction, noise reduction and preprocessing operation;
• step 3:
• data dimensionality reduction methods and fused dimensionality reduction;
• three dimensionality reduction methods are adopted, which are:
• Principal Component Analysis (PCA), t-distributed Stochastic Neighbor Embedding (t-SNE), and Uniform Manifold Approximation and Projection (UMAP), and fused dimensionality reduction is conducted by parameter trade-off;
• specifically:
• step 3.1:
• for the data preprocessed in step 2, using PCA, t-SNE and UMAP to conduct independent dimensionality reduction to the data respectively;
• dimensionality reduction by PCA:
• eigenvalues and eigenvectors of a covariance matrix are calculated, the eigenvectors corresponding to the top k largest eigenvalues are selected, and the data is projected into a subspace composed of the eigenvectors;
• dimensionality reduction by t-SNE:
• the t-SNE algorithm is used to map high-dimensional data to a two-dimensional or three-dimensional space, and maintain a local structure among data points;
• dimensionality reduction by UMAP:
• the UMAP algorithm is used to map high-dimensional data to a low-dimensional space, and retain both global and local structures at the same time;
• step 3.2:
• adopting a cross validation method to determine a weight of each method in fusion;
• a data set is divided into several subsets, one subset is reserved as a test set each time, and the rest are reserved as training sets;
• this process is repeated for multiple times, and performance indexes of each dimensionality reduction method in different compromises are calculated;
• for each method, the average value of the performance indexes thereof in cross validation is calculated;
• a weight is assigned to each method based on the average value of the performance indexes;
• step 3.3:
• conducting fused dimensionality reduction:
• a weighted average of the dimensionality reduction results of the three methods is calculated according to the determined weights to obtain a final fused dimensionality reduction result;
• a formula for the weighted average is:
• Xfused=wpcaXpca+wt-SNEXt-SNE+wUMAPXUMAP, wherein Xfused is a dimensionality reduction result after fusion, Xpca, Xt-SNE and XUMAP are the dimensionality reduction results of PCA, t-SNE and UMAP respectively, and wPCA, wt-SNE and wUMAP are the corresponding weights;
• step 4:
• training of the hyperspectral water quality inversion models;
• three machine learning algorithms, i.e. Support Vector Machine (SVM), Random Forest and Decision Tree, are adopted to train and test the hyperspectral water quality inversion models on the spectral data after dimensionality reduction;
• step 5:
• selection and optimization of the hyperspectral water quality inversion models;
• a determination coefficient R2 and a root mean square error (RMSE) are selected as fitting accuracy test indexes of the hyperspectral water quality inversion models;
• through the training of the hyperspectral water quality inversion models, predicted results of concentrations for water quality parameters of the points at different positions and depths in the collection area can be obtained, including the concentration values of seven water quality parameters at each point;
• accurate and intuitive information on various water quality parameters are acquired.