# US 2022/0214322 A1 · Air pollutants concentration forecasting method and apparatus and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0214322 A1
- Document ID: US20220214322A1
- Status: Published application
- Kind: A1
- Application number: 17/562,109
- Filed: 2021-12-27
- Published: 2022-07-07
- Earliest claimed: 2021-01-07
- Assignees: Tsinghua University
- Applicants: Tsinghua University
- Inventors: Shiji SONG; Gao HUANG; Zhuofan XIA
- Web page: https://patentgrep.com/patent/US20220214322A1

## CPC

- G01N 33/0034: {comprising neural networks or related mathematical techniques}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06M 3/04: with an additional counter train operating in the reverse direction
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06M: COUNTING MECHANISMS; COUNTING OF OBJECTS NOT OTHERWISE PROVIDED FOR

## Abstract

A method, apparatus and storage medium for forecasting air pollutant concentration, including: constructing a training set, a validation set and a test set based on a data set; the data set is obtained by collecting pollutant concentration data and meteorological data in a predetermined length of time in a target area; constructing an adjacent matrix A of a graph structure based on the spatial distribution of monitoring stations in the target area; establishing a neural network model F(x;Θ|A), where x is the input data of it, including pollutant concentration data and meteorological data within predetermined time period, training the neural network model using the data of the training set, adjusting the parameters Θ of the neural network model using the data of the validation set and the data of the test set, and obtaining the modified neural network model; using the modified neural network model for air pollutant concentration forecasting.

## Claims

### Claim 1

A method for forecasting air pollutant concentrations, wherein the method comprises:
constructing a training set, a validation set, and a test set based on a data set; wherein the data set is obtained by collecting pollutant concentration data and meteorological data in a target area for a predetermined length of time;
constructing an adjacent matrix A of a graph structure based on a spatial distribution of monitoring stations in the target area;
building a neural network model F(x;Θ|A), wherein x is input data of the neural network model, the input data comprises the pollutant concentration data and the meteorological data during a set time period,
training the neural network model by using data of the training set,
adjusting parameters Θ of the neural network model by using data of the validation set and data of the test set to obtain a modified neural network model;
forecasting the air pollutant concentrations, by using the modified neural network model.

### Claim 2 (depends on 1)

The method for forecasting the air pollutant concentrations according to claim 1, wherein the step of constructing the training set, the validation set, and the test set based on the data set, comprises:
reading the pollutant concentration data and the meteorological data from the data set, wherein the pollutant concentration data comprises values for individual pollutant concentrations and the meteorological data comprises values for various meteorological conditions;
according to an input time length tin and an output time length tout needed for forecasting, obtaining a plurality of data segments after sliding window slicing operation of the data set in a time dimension, wherein lengths of the plurality of data segments in the time dimension are t, characteristic dimensions of the plurality of data segments are the values for the various meteorological conditions and the values for the individual pollutant concentrations, wherein t=tin+tout;
constructing the training set, the validation set, and the test set based on the plurality of data segments.

### Claim 3 (depends on 1)

The method for forecasting the air pollutant concentrations according to claim 1, wherein the step of training the neural network model by using the data of the training set, comprises:
feeding data (x,y) in the training set into the neural network model in a batch for training to obtain a predicted output ŷ,
calculating a loss function (ŷ, y), and
optimizing the parameters Θ of the model based on a gradient descent algorithm;
wherein y is the pollutant concentration data; the neural network model comprises: an input layer, a hidden layer, and an output layer, the hidden layer comprises a plurality of self-attentive modules based on a graph convolution layer and a one-dimensional convolution layer.

### Claim 4 (depends on 1)

The method for forecasting the air pollutant concentrations according to claim 1, wherein the step of constructing the adjacent matrix A of the graph structure based on the spatial distribution of the monitoring stations in the target area, comprises:
reading latitude and longitude coordinates of the monitoring stations from the data set, converting the latitude and longitude coordinates of the monitoring stations to relative coordinates in a Cartesian coordinate system, and constructing the adjacent matrix A based on a distance between every two of the monitoring stations:
A
ij
=
{
exp
(
-
d
ij
2
σ
2
)
⁢
,
d
ij
≥
κ
,
0
,
d
ij
<
κ
,
where Aij is an element of the i-th row and j-th column of the adjacent matrix A, dij denotes a distance between a monitoring station i and a monitoring station j, σ is a standard deviation of a distance between all of the monitoring stations, and κ is a preset hyperparameter for ensuring a sparsity of the adjacent matrix A.

### Claim 5 (depends on 1)

The method for forecasting the air pollutant concentrations according to claim 1, wherein the input data x of the neural network model is z-score normalized in a characteristic dimension to obtain z-score normalized input data x, the z-score normalized input data x has a zero mean and a unit standard deviation:
x
norm
(
c
)
=
x
(
c
)
-
μ
(
c
)
σ
(
c
)
,
where superscript (c) denotes a cth characteristic dimension, xnorm(c) denotes the z-score normalized input data x on the cth characteristic dimension, μ denotes a mean of data points, and σ denotes a standard deviation of a distance between all of the monitoring stations;
a dimension of a composition tensor of the input data x of the neural network model is N×T×G×C;
where N denotes a batch size, T denotes time, G denotes a graph node, and C denotes the characteristic dimension.

### Claim 6 (depends on 3)

The method for forecasting the air pollutant concentrations according to claim 3, wherein the data (x,y) are computed in the hidden layer in the following order:
a temporal self-attention operation, followed by a one-dimensional convolution operation with a gated linear activation in a time dimension, a graph node self-attention operation, a first-order Chebyshev graph convolution operation with a residual connectivity in a graph node dimension, and a one-dimensional convolution operation with the residual connectivity in the time dimension;
wherein, result data of an operation, coming first in order, is configured as input data for a next operation.

### Claim 7 (depends on 6)

The method for forecasting the air pollutant concentrations according to claim 6, wherein the temporal self-attention operation comprises:
when a dimension of first input data x1 of the temporal self-attention operation is N×T×G×C, operating the first input data x1 by two graph convolution layers φG and θG respectively, to obtain z1=φG(x1|A) and z2=θG (x1|A) respectively;
reshaping z1, z2 in a dimension N×T×GC and doing a softmax operation by a row after multiplying by a batching matrix in the last dimension to obtain a self-attentive relation a1=softmax(bmm(z1T, z2)), wherein the self-attentive relation a1 is in a dimension N×T×T;
multiplying the self-attentive relation a1 right on a linear mapping of the first input data x1 by a batching and multiplying the self-attentive relation a1 by a predetermined scaling factor γ, to deform the self-attentive relation a1 into N×T×G×C, and then adding the self-attentive relation a1 with the first input data x1 to obtain a result of the temporal self-attention operation y1 and outputting the result.

### Claim 8 (depends on 6)

The method for forecasting the air pollutant concentrations according to claim 6, wherein the one-dimensional convolution operation with the gated linear activation in the time dimension comprises:
when a dimension of second input data x2 of the one-dimensional convolution operation with the gated linear activation in the time dimension is N×L×Ci, activating a result of an one-dimensional convolution operation on the second input data x2 in a dimension L to obtain an output z having a dimension of N×L×2C2, where L is a length of a single sample, C1 is a characteristic dimension of the second input data x2, and C2 is a dimension of a set output;
splitting the output z into z3, z4 according to the characteristic dimension in half, wherein dimensions of z3, z4 are both N×L×C2; a result of multiplying z3 by elements by z4 after activating z3 by a sigmoid function to obtain an operation y2, and outputting y2=sigmoid(z3)⊗z4, where ⊗ denotes an element-wise multiplication.

### Claim 9 (depends on 6)

The method for forecasting the air pollutant concentrations according to claim 6, wherein the graph node self-attention operation, comprising:
when a dimension of third input data x3 of the graph node self-attention operation is N×T×G×C, operating the third input data x3 by two one-dimensional convolution layers φT and θT respectively, to obtain z5=φT(x3) and z6=θT(x3) respectively;
reshaping z5, z6 in a dimension N×G×TC and doing a softmax operation by row after multiplication by a batching matrix in the last dimension to obtain a self-attentive relation a2=softmax(bmm(z5T, z6)), wherein a dimension of the self-attentive relation a2 is N×G×G;
multiplying the self-attentive relation a2 right on a linear mapping of the third input data x3 by batching and reshaping the self-attentive relation a2 to N×T×G×C after multiplying by a predetermined scaling factor γ; summing the self-attentive relation a2 with the third input data x3 to obtain a result of the graph node self-attention operation y3 and outputting the result.

### Claim 10 (depends on 6)

The method for forecasting the air pollutant concentrations according to claim 6, wherein the first-order Chebyshev graph convolution operation with the residual connection in the graph node dimension comprises:
for each node, calculating separately and normalizing symmetrically features of each node and other nodes in a one-hop neighborhood according to a degree matrix:
g
θ
⁡
(
x
4
)
=
θ
0
+
θ
1
⁢
x
4
+
θ
2
⁢
D
-
1
2
⁢
AD
-
1
2
⁢
x
4
,
where θ is a parameter, x4 is fourth input data for carrying out the first-order Chebyshev graph convolution operation with the residual connection in the graph node dimension, D is a symmetry matrix, Dii=ΣjAij is a degree matrix of a graph;
after performing the first-order Chebyshev graph convolution operation, obtaining and outputting a result y4, wherein y4=x4+LReLU(gθ(x4)), and LReLU is a LeakyReLU activation function.

### Claim 11 (depends on 3)

The method for forecasting the air pollutant concentrations according to claim 3, wherein the loss function is a smoothed L1 loss function,
for each predicted output ŷi calculating a value of an error ei, ei=|ŷi−yi|/s,
where s is a hyperparameter controlling a size of the error, and the loss function is li=0.5ei2 when ei<1, i=ei−0.5 when ei≥1, and a total loss function is
ℓ
=
1
N
⁢
Σ
i
⁢
ℓ
i
.

### Claim 12 (depends on 1)

An air pollutant concentration forecasting device, comprising a memory and a processor, wherein the memory is configured to store a program for performing an air pollutant concentration forecasting; the processor is configured to read the program for performing the air pollutant concentration forecasting and to execute the method for forecasting the air pollutant concentrations as claimed in claim 1.

### Claim 13 (depends on 1)

A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to perform the method of claim 1.

### Claim 14 (depends on 2)

The method for forecasting the air pollutant concentrations according to claim 2, wherein the input data x of the neural network model is z-score normalized in a characteristic dimension to obtain z-score normalized input data x, the z-score normalized input data x has a zero mean and a unit standard deviation:
x
n
⁢
o
⁢
r
⁢
m
(
c
)
=
x
(
c
)
-
μ
(
c
)
σ
(
c
)
,
(
c
)
where superscript (c) denotes a cth characteristic dimension, xnorm(c) denotes the z-score normalized input data x on the cth characteristic dimension, μ denotes a mean of data points, and σ denotes a standard deviation of a distance between all of the monitoring stations;
a dimension of a composition tensor of the input data x of the neural network model is N×T×G×C;
where N denotes a batch size, T denotes time, G denotes a graph node, and C denotes the characteristic dimension.

### Claim 15 (depends on 3)

The method for forecasting the air pollutant concentrations according to claim 3, wherein the input data x of the neural network model is z-score normalized in a characteristic dimension to obtain z-score normalized input data x, the z-score normalized input data x has a zero mean and a unit standard deviation:
x
norm
(
c
)
=
x
(
c
)
-
μ
(
c
)
σ
(
c
)
,
where superscript (c) denotes a cth characteristic dimension, xnorm(c) denotes the z-score normalized input data x on the cth characteristic dimension, μ denotes a mean of data points, and σ denotes a standard deviation of a distance between all of the monitoring stations;
a dimension of a composition tensor of the input data x of the neural network model is N×T×G×C;
where N denotes a batch size, T denotes time, G denotes a graph node, and C denotes the characteristic dimension.

### Claim 16 (depends on 4)

The method for forecasting the air pollutant concentrations according to claim 4, wherein the input data x of the neural network model is z-score normalized in a characteristic dimension to obtain z-score normalized input data x, the z-score normalized input data x has a zero mean and a unit standard deviation:
x
norm
(
c
)
=
x
(
c
)
-
μ
(
c
)
σ
(
c
)
,
where superscript (c) denotes a cth characteristic dimension, xnorm(c) denotes the z-score normalized input data x on the cth characteristic dimension, μ denotes a mean of data points, and σ denotes the standard deviation of the distance between all of the monitoring stations;
a dimension of a composition tensor of the input data x of the neural network model is N×T×G×C;
where N denotes a batch size, T denotes time, G denotes a graph node, and C denotes the characteristic dimension.

### Claim 17 (depends on 12)

The air pollutant concentration forecasting device according to claim 12, wherein the step of constructing the training set, the validation set, and the test set based on the data set, comprises:
reading the pollutant concentration data and the meteorological data from the data set, wherein the pollutant concentration data comprises values for individual pollutant concentrations and the meteorological data comprises values for various meteorological conditions;
according to an input time length tin and an output time length tout needed for forecasting, obtaining a plurality of data segments after sliding window slicing operation of the data set in a time dimension, wherein lengths of the plurality of data segments in the time dimension are t, characteristic dimensions of the plurality of data segments are the values for the various meteorological conditions and the values for the individual pollutant concentrations, wherein t=tin±tout;
constructing the training set, the validation set, and the test set based on the plurality of data segments.

### Claim 18 (depends on 12)

The air pollutant concentration forecasting device according to claim 12, wherein the step of training the neural network model by using the data of the training set, comprises:
feeding data (x,y) in the training set into the neural network model in a batch for training to obtain a predicted output ŷ,
calculating a loss function (ŷ,y), and
optimizing the parameters Θ of the model based on a gradient descent algorithm;
wherein y is the pollutant concentration data; the neural network model comprises: an input layer, a hidden layer, and an output layer, the hidden layer comprises a plurality of self-attentive modules based on a graph convolution layer and a one-dimensional convolution layer.

### Claim 19 (depends on 12)

The air pollutant concentration forecasting device according to claim 12, wherein the step of constructing the adjacent matrix A of the graph structure based on the spatial distribution of the monitoring stations in the target area, comprises:
reading latitude and longitude coordinates of the monitoring stations from the data set, converting the latitude and longitude coordinates of the monitoring stations to relative coordinates in a Cartesian coordinate system, and constructing the adjacent matrix A based on a distance between every two of the monitoring stations:
A
ij
=
{
exp
(
-
d
ij
2
σ
2
)
⁢
,
d
ij
≥
κ
,
0
,
d
ij
<
κ
,
where Aij is an element of the i-th row and j-th column of the adjacent matrix A, dij denotes a distance between a monitoring station i and a monitoring station j, σ is a standard deviation of a distance between all of the monitoring stations, and κ is a preset hyperparameter for ensuring a sparsity of the adjacent matrix A.

### Claim 20 (depends on 12)

The air pollutant concentration forecasting device according to claim 12, wherein the input data x of the neural network model is z-score normalized in a characteristic dimension to obtain z-score normalized input data x, the z-score normalized input data x has a zero mean and a unit standard deviation:
x
norm
(
c
)
=
x
(
c
)
-
μ
(
c
)
σ
(
c
)
,
where superscript (c) denotes a cth characteristic dimension, xnorm(c) denotes the z-score normalized input data x on the cth characteristic dimension, μ denotes a mean of data points, and σ denotes a standard deviation of a distance between all of the monitoring stations;
a dimension of a composition tensor of the input data x of the neural network model is N×T×G×C;
where N denotes a batch size, T denotes time, G denotes a graph node, and C denotes the characteristic dimension.
