HTML

ePub

PDF

Introduction

Spatial supervised classification is a problem of labeling observations based on fea- ture information and information about spatial adjacency relationships with training sample. This problem has been studied by numerous authors (see e.g. Atkinson and Lewis [1]). However, these studies are usually based on the assumption of con- ditional independence of feature observations. Comprehensive overview of methods for statistical classification and discrimination of Gaussian spatial data is provided by Berrett and Calder [3]. The novel approach to classification of Gaussian Random Importar imagen Importar tabla Field (GRF) observation by avoiding the assumption of conditional independence is developed by Dučinskas and Dreižienė [5]. However, statistical discriminant analy- sis of spatio-temporal data has been rarely considered previously. Šaltytė-Benth and Dučinskas [8] considered classification of spatio-temporal data modeled by GRF in particular case when observation of feature at focal location is uncorrelated with the training sample.

In the present paper, avoiding independence restrictions, we focus on the classi- fication of data modeled by random fields with separable spatio-temporal covariance structures specified by geostatistical spatial margins and discrete temporal margins (see e.g. [4]). Separability of covariances was assumed for the sake of reduction of complexity due to interdependencies between features.

The main distinctive feature of the proposed approach is allowing the class label to depend on spatial location as well as on time moment. That essentially widens the application area of presented investigations.

For the performance of classifiers, the values of derived in Local Bayes error rates and empirical error rates are used. For numerical illustrations, the exponential isotropic models for spatial covariance and AR(p) model of temporal covariance are considered. This is the extension of AR(1) case explored in Karaliutė and Dučinskas [7]. Performance of proposed classification rule is compared for various parameters of pure spatial and temporal covariances and prior class probabilities models.

This paper is organized as follows: proposed spatio-temporal data models and conditional distributions are delivered in the next section; in Section 2 conditional Bayes classification rule and its error rates are presented; in Section 3 the numerical illustrations and simulations for various separable stationary spatio-temporal covari- ance and prior probabilities models are displayed, and finally, the conclusions are in the last section.

1 Spatio-temporal data models and conditional distributions

The main objective of this paper is to classify feature observations of $\text{GRF}\{Z(s;t):s\in D\subset {\text{D}}_{\text{T}},t\in {\text{D}}_{\text{T}}=[0,\infty ]\},$ , where s and t define spatial and temporal coordinates, respectively. Let $\textcolor{red}{}\{Y(s;t):s\in D\subset {\text{R}}^2,t\in {\text{D}}_{\text{T}}\}$ be a random field that represents class label and takes only the value 0 or 1.

In this study, we assume that for $\iota =0,1$ the model of observation $\text{Z}(s;t)$con- ditional on$\text{Y}(s;t)=\iota$ is $\text{Z}(s;t)=\mu \iota (s;t)+\epsilon (s;t)$ , where$\mu \iota (s;t)-$ deterministic spatio-temporal drift. The error term is assumed to be generated by the univariate zero-mean GRF $\left\{\epsilon \right(s;t):s\in D\subset {\text{R}}^{\text{2}},t\in T\}$ , with the separable spatio-temporal covariance model $cov\left(\epsilon \right(s;t),\epsilon (u;r\left)\right)=C(s,u;t,r)$ for all$s,u$ $\in D$and$t,r\in T.$ .

Here ${\text{C}}_s(s,u)$ denotes pure spatial covariance between observations in locations $s$ and $u$ and ${\text{C}}_{\text{T}}(t,r)$, denotes pure temporal covariance between observations at time points $t$ and$r$. Under this assumption, the spatio-temporal covariance structure factors into a purely spatial and a purely temporal component, which allows for computationally efficient estimation and inference.

In this study we follow the popular tradition in environmental and agricultural research when the data are recorded at regular time intervals (time lags) and at irregular stations (locations) in compact area (see e.g. [2, 6]).

Let ${\text{S}}_n=\{s_i\in D;i=1,...,n\}$ be a set of locations where observations are taken at time $t\in {\text{D}}_m=\{1,2,...,m,m+1\}.$ At every moment of time$t\in {\text{D}}_m$ the set $s_n$ is split into two classes, ${\text{S}}_t^{\left(0\right)}$ and ${\text{S}}_t^{\left(1\right)}$ $(i.e.,{\text{S}}_n={\text{S}}_t^{\left(0\right)}\cup {\text{S}}_t^{\left(1\right)})$ $:{\text{S}}_t^{\left(l\right)}\{=s\in {\text{S}}_n:Y(s,t)=l\},l=0,1:$. Denote nlt the number of locations (of n) at time $t$ that belong to class $l;$thus nlt is the number of points in the set ${\text{S}}_t^{\left(l\right)}$, and $n=n_{0t}+n_{1t}$ for every$t\in {\text{D}}_m$. Hence a set of class labels at any time moment can differ in composition.

Joint training sample$\text{Z}$ is stratified training sample, specified by $n\times m$ ma- trix $\text{Z}=({\text{Z}}_1,...,{\text{Z}}_m)$, where ${\text{Z}}_t=(Z(s_n,t),...,Z(s_n,t))\prime .$This structure of data presentation is motivated by a model that assumes multivariate (in space) time se- ries. Denote by $z_t=(z_t^1,...,z_t^n)$and $y_t=(y_t^1,...,y_t^n)$ the realized value of ${\text{Z}}_t$ and ${\text{Y}}_t=(Y(s_1,t),...,Y(s_n,t))\prime ,$ respectively

In this study we consider the linear parametric drift ${\mu }_1(s;t)={\beta }_l^{\prime }x\left(s\right),$where $x_i=\left(x_1\right(s_i),...,x_q(s_i\left)\right)\prime$ is the vector of a spatial covariates, and${\beta }_l$ is a q dimensional vector of parameters,$i=1,...,n,l=0,1$ , and$\Delta \beta ={\beta }_1-{\beta }_0$.

Denote by $\text{X}$ the $n\times 2qm$ matrix $\text{X}=({\text{X}}_{\left(1\right)},{\text{X}}_{\left(2\right)},...,{\text{X}}_{\left(m\right)})$ where

Then the matrix model for $\text{Z}$ conditional on $\{{\text{Y}}_t=y_t,t=,...,m\}$is $\text{Z}=XB\times E,$, where $\text{B=I}m⨂\beta$with $\beta =({\beta }_0^{\text{'}},{\beta }_1^{\text{'}})\text{'}$ and $n\times m$ matrix of Gaussian errors $\text{E}=\left(\epsilon \right(s_i;t):i=1,...,n;t=1,...m).$ Here ${\text{I}}_m$ is $m\times m$ identity matrix, and $\beta is2q\times 1$vector of parameters. Under covariance separability assumption $vec(E)~{\text{N}}_{nm}({\text{C}}_{\text{T}}⨂{\text{C}}_s)$ with ${\text{C}}_{\text{T}}=(c_{\text{T}}^{tr}={\text{C}}_{\text{T}}(t,r);t,r=1,...,m)$ denoting the $m\times m$ matrix of pure temporal covariances and ${\text{C}}_s=(c_s^{ij}={\text{C}}_s(s_i,s_j);i,j=1,...,n)$ denoting the$n\times n$ matrix of pure spatial covariances.

In present paper we are dealing with the problem of classification of the observa- tions $\text{Z}(s_i,m+1),i=1,...,n$ into one of two classes with given joint training sample $\text{Z}$ or, in other words, based on training sample information we want to predict label at an observed location at the time moment$t=m+1.Set{c}_{\text{T}}^{m+1,r}={\text{C}}_{\text{T}}(m+1,r);r=1,...,m,$ $c_{\text{T}}^{m+1,1},...,c_{\text{T}}^{m+1,p})\text{'}$and $e_i^{\text{'}}$ the ith row of identity matrix ${\text{I}}_n$.

Under spatio-temporal data model specification, we can conclude that in $l=0,1,$, the conditional distribution of $\text{Z}(s_i,m+1)$given $\text{Z}=z$ and $\text{Y}(s_i;m+1)=l,$ is Gaussian, $i.e.,$

where ${\sum }_{m+1,i\left(z\right)}=var(Z(s_i,m+1))-c_s^{ii}(c_{\text{T}}^{m+1})\text{'}{\text{C}}_{\text{T}}^{-1}{c}_{\text{T}}^m$ $=c_s^{ii},{\rho }_{m+1}^{\textcolor{red}{}},{\mu }_{li}^{m+1}={\beta \text{'}}_l{x}_i+\left(\right(c_{\text{T}}^{m+1})\text{'}{\text{C}}_{\text{T}}^{-1}⨂e_i^{\text{'}})vec(E)$with ${\rho }_{m+1}=c_{\text{T}}^{m+1,m+1}-\left(c_{\text{T}}^{m+1}\right)\text{'}{\text{C}}_{\text{T}}^{-1}{c}_{\text{T}}^{m+1}.$.

In this study, we assume that the conditional distribution of label $\text{Y}(s_i,m+1),i=1,...,n,$, given joint training sample $\text{Z}$ depends only on class labels values, $i.e.$ conditional distribution of $(Y(s_i,m+1=l|Z=z)$ is identical to conditional distribution of$(Y(s_{i}m+1=l|\{{\text{Y}}_t=y_t,t=1,...,m\})$. Set P $\text{P}(Y(s_i,m+1=l|Z=z={\pi }_l(s_i,m+1)l=0,1,$, and, for simplicity, call them prior class probabilities.

2 Conditional Bayes discriminant functions and its error rates

Under the assumption that the classes are completely specified, the conditional Bayes discriminant function (CBDF) minimizing the probability of misclassification is formed by the log-ratio of conditional likelihood of distribution specified in (1), that is

where $\gamma i(m+1)=$ ln$\left({\pi }_1\right(s_i,m+1)/{\pi }_2(s_i,m+1\left)\right).$.

Let us call the probability of misclassification for ${\text{W}}_{\text{Z}}(Z(s_i,m+1))$ as Local Bayes error rate and denote it by ${\text{P}}_i$. Also let us denote squared Mahalanobis distance between conditional distributions by

Specific attention is given to the Gaussian spatio-temporal model with pure spatial covariances belonging to the family of powered-exponential isotropic models and with pure temporal covariance of stationary AR(p) model.

Lemma 1. The Local Bayes error rate is

where $\Phi \left(x\right)$ is the standard normal cumulative distribution function and

Proof. It is known that for AR(p) model parameters quantify the temporal depen-dency and for$t=1,2,...,m+1,c_{\text{T}}^{tt}={\text{C}}_{\text{T}}\left(0\right)={\sigma }_{\text{T}}^2+{\sum }_{j=1}^p{a}_j\left(j\right)$ (j), where ${\sigma }_{\text{T}}^2$ is the variance of the temporal white noise.

Then ${\mu }_{li\left(z\right)}^{m+1}={\beta }_{lxi}^{\text{'}}+\left(\right(0,...,0,0\mathrm{\alpha }p,...,{\mathrm{\alpha }}_1)\text{'}⨂e_i^{\text{'}})$ $vec(E)$ and${\sum }_{m+1,i\left(z\right)}=c_{\text{S}}^{ii}{\mathrm{\sigma }}_{\text{T}}^2.$

By using the properties of multivariate Gaussian distribution and inserting the above expressions in formula (3) we complete the proof of Lemma 1. H

Error estimation is critical to classification because the validity of the resulting classifier model, composed of the classifier and its error estimate, is based on the accuracy of the error estimation procedure. Given a set of sample data, the data can be split between training and test data, with a classifier being designed on the training data and its error being validated on the test data. In this paper our focus is on using p temporal observations for training and the observations at m + 1th time moment is using for testing.

3 Numerical illustrations and simulations

For numerical illustrations of the proposed classifier performance, we considered the Gaussian spatio-temporal model with pure spatial exponential covariances and with

pure temporal covariance stationary AR(1) model. Then it is easy to derive that$\left(c_{\text{T}}^{m+1}\right)\text{'}{\text{C}}_{\text{T}}^{-1}=(0,...,0,{\mathrm{\alpha }}_1).$

In the study, the exponential isotropic nugetless spatial covariance is considered.

So ${\text{C}}_s={\sigma }_s^2\text{R,}$, where $\text{R}=\left(r_{ij}\right)$ denotes the spatial correlation matrix with$r_{ij}=r\left(\right|s_i-s_j\left|\right)=e^{-|s_j-s_j|/\mathrm{\phi }}.$Here $\textcolor{red}{}\mathrm{\phi }$is the so called range parameter that represents the spatial dependence.

This choice is based on the smoothness level of sample paths. Sample paths of a GRF with the exponential covariance function are not smooth, when the squared exponential covariance model has smooth sample paths.

Two methods for prior class probabilities is proposed.

First one is based on Temporal Weighted Moving Average (TWMA) method

Second one adds spatial correlations for weighting

where$i_o$ denotes the index of the nearest neighbor to $\textcolor{red}{}{s}_i$. Denote this method by (STWMA). We have compared these four particular cases by calculating the $\textcolor{red}{}{\text{P}}_i$ for$\textcolor{red}{}i=1,...,n.$. Numerical illustrations performed on 20 locations in two dimensional Euclidean area with$\textcolor{red}{}°$ and $\textcolor{red}{}∎$for label indicators are depicted in Fig. 1.

The averages $\text{AP}={\sum }_{i=1}^{20}{\text{P}}_i/20$ for two models of prior probabilities and for$\textcolor{red}{}\phi =$ 2.5 and various $\textcolor{red}{}{\mathrm{\alpha }}_1$ at moment$\textcolor{red}{}t=5$are presented in Fig. 2. As it might be seen from Fig. 2, incorporation of spatial correlation in class prior probabilities ($\textcolor{red}{}i.e.$ method STWMA) does not have advantage against method TWMA for almost all values of parameter${\mathrm{\alpha }}_1$.

4 Conclusions

In this paper we propose approach to classification of spatio-temporal data in the framework of Bayes discriminant analysis for separable spatio-temporal covariances.

Several simulation studies were conducted to estimate and compare empirically the classifiers for particular separable stationary spatio-temporal covariance and various prior class probabilities models. Numerical analysis showed that there is no reason to incorporate spatial correlation in the prior probabilities since it does not improve the performance of the proposed classifier.

References

[1] P.M. Atkinson, P. Lewis. Geostatistical classification for remote sensing: an introduction. Comput. Geosc., 26(4):361–371, 2000. https://doi.org/10.1016/S0098-3004(99)00117- X.

[2] G. Atluri, A. Karpatne, V. Kumar. Spatio-temporal data mining: a sur- vey of problems and methods. ACM Comput Surv: CSUR, 51(4):83, 2018. https://doi.org/10.1145/3161602.

[3] C. Berrett, C.A. Calder. Bayesian spatial binary classification. Spatial Stat., 16:72–102, 2016. https://doi.org/10.1016/j.spasta.2016.01.004.

[4] S.S. Demel, J. Du. Spatio-temporal models for some data sets in continuous space and discrete time. Stat. Sin., 25:81–98, 2015. https://doi.org/10.5705/ss.2013.223w.

[5] K. Dučinskas, L. Dreižienė. Risks of classification of the Gaussian Markov random field observations. J. Class., 35:422–436, 2018. https://doi.org/10.1007/s00357-018-9269-7.

[6] J. Haslett, A.E. Raftery. Space-time modelling with long memory depen- dence: Assessing Ireland’s wind power resourse. Appl. Stat., 38:1–50, 1989. https://doi.org/10.2307/2347679.

[7] M. Karaliutė, K. Dučinskas. Classification of gaussian spatio-temporal data with sta- tionary separable covariances. Nonlinear Anal. Model. Control, 26(2):363–374, 2021. https://doi.org/10.15388/namc.2021.26.22359.

[8] J. Šaltytė-Benth, K. Dučinskas. Linear discriminant analysis of multivariate spatial- temporal regressions. Scand. J. Stat., 32:281–294, 2005.

IR