# Stable Distributions: Generalised CLT and Series Representation Theorem

This is a continuation of the previous post that can be found here. We discuss the Generalised CLT in this post. We give a sloppy proof of the theorem. Then we end with a series representation theorem for the Stable distributions.

The Generalised Central Limit Theorem

Theorem 9: ${X_1,X_2,\ldots}$ are iid symmetric random variables with ${P(X_1>\lambda) \sim k\lambda^{-\alpha}}$ as ${\lambda \rightarrow\infty}$ for some ${k>0}$ and ${\alpha \in (0,2)}$. Then

$\displaystyle \frac{X_1+X_2+\cdots+X_n}{n^{1/\alpha}} \stackrel{d}{\rightarrow} Y$

where ${Y\sim S\alpha S}$ with appropiate constants.`

Motivation for proof: Recall the proof of usual CLT. We assume ${X_i}$ are iid random variables mean ${0}$ and variance ${\sigma^2}$ and characteristic function ${\phi}$. We use the fact that under finite variance assumption we have

$\displaystyle \frac{1-\phi(t)}{t^2} \rightarrow \frac{\sigma^2}{2}$

and hence using this we get

\displaystyle \begin{aligned} E\left[e^{i\frac{t}{\sqrt{n}}(X_1+X_2+\cdots+X_n)}\right] = \left[E\left(e^{i\frac{t}{\sqrt{n}}X_1}\right)\right]^n & = \left[\phi\left(\frac{t}{\sqrt{n}}\right)\right]^n \\ & = \left[1-\left(1-\phi\left(\frac{t}{\sqrt{n}}\right)\right)\right]^n \\ & \rightarrow \exp(-t^2\sigma^2/2) \end{aligned}

We will this idea in our proof. We will leave some of the technical details of the proof. The proof presented here is not rigourous. We primarily focus on the methodology and tricks used in the proof.

Proof: Let ${F}$ be the cdf of ${X_1}$. ${F}$ is symmetric and ${F(x)\sim 1-kx^{-\alpha}}$ as ${x\rightarrow \infty}$. Let ${\psi}$ be the characteristic function of ${X_1}$. ${\psi}$ is real and even. We are interested in the behaviour of ${1-\psi(t)}$ near ${0}$. Note that

$\displaystyle 1-\psi(t)=E(1-\cos tX_1)=2\int_0^{\infty} (1-\cos tx) \,dF(x)$

First get a large enough ${M}$ such that the tail behaviour is nice enough after ${M}$. Then separate the integral into two parts

$\displaystyle \int_0^{\infty} (1-\cos tx) \,dF(x)=\int_0^{M} (1-\cos tx) \,dF(x)+\int_M^{\infty} (1-\cos tx) \,dF(x)$

Now since we are interest in the integral for small enough ${t}$, observe that for small enough ${t}$, we have for ${0\le x \le M}$

$\displaystyle (1-\cos (tx)) \le Ct^2x^2 \le Ct^2M^2$

Thus for small enough positive ${t}$ we get

$\displaystyle \displaystyle 0 \le \int_0^{M} (1-\cos tx) \,dF(x) \le Ct^2M^2\int_0^M \,dF(x) \le Ct^2M^2$

For the second integral, note that since ${M}$ is large enough

$\displaystyle F(x) \approx 1-kx^{-\alpha} \implies dF(x) \approx \frac{\alpha k}{x^{\alpha+1}}\,dx$

Therefore

\displaystyle \begin{aligned} \int_M^{\infty} (1-\cos tx)\,dF(x) & \approx \int_M^\infty (1-\cos tx)\frac{\alpha k}{x^{\alpha+1}}\,dx \\ & = \alpha k t^{\alpha} \int_{Mt}^{\infty}\frac{1-\cos u}{u^{\alpha+1}}\,du \end{aligned}

Hence for ${t>0}$

$\displaystyle \frac{1-\psi(t)}{t^\alpha}=\frac1{t^{\alpha}}\mathcal{O}(t^2)+2\alpha k \int_{Mt}^{\infty} \frac{1-\cos u}{u^{\alpha+1}}\,du$

Since ${\alpha<2}$, first term vanishes as ${t\rightarrow 0^+}$ and the second term gives us in the limit

$\displaystyle 2\alpha k \int_{0}^{\infty} \frac{1-\cos u}{u^{\alpha+1}}\,du$

Note that as ${\alpha>0}$, the above integral converges near ${\infty}$. Near ${0}$, ${1-\cos u \sim u^2/2}$, and hence the integral is asymptotically like the integral of ${\frac1{2u^{\alpha-1}}}$ near ${0}$. Since ${\alpha <2}$, we have ${\alpha-1 <1}$ and hence the integral converges near ${0}$. Thus

$\displaystyle \int_{0}^{\infty} \frac{1-\cos u}{u^{\alpha+1}}\,du< \infty$

Hence ${\displaystyle \lim_{t\rightarrow 0^+} \frac{1-\psi(t)}{t^\alpha}= C}$ where ${C}$ is some constant. Since ${\psi}$ is even, we get that ${\displaystyle \lim_{t\rightarrow 0} \frac{1-\psi(t)}{|t|^\alpha}= C}$. Thus

$\displaystyle E\left[e^{i\frac{t}{n^{1/\alpha}}(X_1+X_2+\cdots+X_n)}\right]=\left[1-\left(1-\psi\left(\frac{t}{n^{1/\alpha}}\right)\right)\right]^n \rightarrow e^{-C|t|^\alpha}$

This completes the proof. ${\square}$

Remark: One can even use asymmetric iid random variables with similar tail conditions on both sider. For example,

$\displaystyle P(X>x) \sim px^{-\alpha} \ \ \mbox{and} \ \ P(X<-x) \sim qx^{-\alpha}$

where ${p,q>0}$. Then also the limit converges to the (general) stable distribution. For the exact statement of the general result one may look at Gut[3].

Application: ${X_1,X_2,\ldots}$ iid with density that is symmetric about ${0}$ and continuous and positive around ${0}$. Then

$\displaystyle \frac1n\left(\frac1{X_1}+\frac1{X_2}+\cdots+\frac1{X_n}\right) \stackrel{d}{\rightarrow} \mbox{Cauchy} \ \ \ \ \ \mbox{(with appropriate parameters)}$

Proof: Observe that

$\displaystyle P(\frac1{X_1}> x) =P(0

Thus the tail condition is satisfied with ${\alpha=1}$. Hence

$\displaystyle \frac1n\left(\frac1{X_1}+\frac1{X_2}+\cdots+\frac1{X_n}\right) \stackrel{d}{\rightarrow} Y$

where ${Y}$ is symmetric stable with ${\alpha=1}$. Thus ${Y}$ is Cauchy with appropiate parameters.

Series representation

Theorem 10: Let ${\{\epsilon_i\}_{i\ge1}}$, ${\{\Gamma_i\}_{i\ge1}}$ and ${\{W_i\}_{i\ge1}}$ be three independent sequences of random variables where ${\epsilon_1,\epsilon_2,\ldots \stackrel{iid}{\sim} \pm 1}$ with probability ${\frac12}$ each, ${\Gamma_1\le \Gamma_2 \le \Gamma_3 \le \cdots}$ are the arrival times of a homogenous poisson process with unit arrival rate, and ${W_1,W_2,\cdots}$ are iid random variables satisfying ${E|W|^{\alpha}<\infty}$. Then

$\displaystyle \sum_{i=1}^n \epsilon_i\Gamma_i^{-1/\alpha}W_i \stackrel{a.s.}{\rightarrow} X$

where ${X \sim S\alpha S}$ with characteristic function ${\exp(-c_\alpha E|W|^{\alpha}|t|^\alpha)}$ where ${c_\alpha}$ is a computable constant depending only on ${\alpha}$.

Proof: We divide the proof into following three 3 steps

{Step-1:} (Almost sure convergence) We wish to apply Kolmogorov three series theorem. But ${\Gamma_i}$‘s are not independent. Note that ${\Gamma_1, \Gamma_2-\Gamma_1, \Gamma_3-\Gamma_2 , \cdots \stackrel{iid}{\sim} \mbox{Exp}(1)}$. By strong law we have ${\frac1n\Gamma_n \stackrel{a.s.}{\rightarrow} 1}$. Let ${\displaystyle A=\left(\lim_{n\rightarrow\infty} \frac1n\Gamma_n=1\right) \cap \left(\Gamma_1>0\right)}$. Then ${P(A)=1}$. We will show ${\displaystyle \sum_{i=1}^n \epsilon_i\Gamma_i^{-1/\alpha}W_i}$ converges almost surely for each fixed sequence of ${\{\Gamma_i\}_{i\ge 1}}$ belonging to the event ${A}$. Fix such a sequence. We can get ${C_1,C_2 >0}$ such that

$\displaystyle C_1i \le \Gamma_i \le C_2i \ \ \forall \ i\ge 1$

Now let us apply Kolmogorov three series theorem. Let ${F}$ be the distribution of ${|W_1|}$. For each ${\lambda >0}$,

\displaystyle \begin{aligned} \sum_{i=1}^{\infty} P\left( |\epsilon_i\Gamma_i^{-1/\alpha}W_i| > \lambda\right) & = \sum_{i=1}^{\infty} P\left(|W_i|^{\alpha} > \lambda^{\alpha}\Gamma_i\right) \\ & \le \sum_{i=1}^{\infty} P\left(|W_i|^\alpha>\lambda^\alpha C_1i\right) < \infty \end{aligned}

The last inequality is true as ${E|W_1|^{\alpha}<\infty}$.

$\displaystyle \sum_{i=1}^n E\epsilon_i\Gamma_i^{-1/\alpha}W_i\mathbb{I}_{|\epsilon_i\Gamma_i^{-1/\alpha}W_i|\le \lambda} =0$

since the symmetric random variables is truncated expectation exists and hence zero.

\displaystyle \begin{aligned}\sum_{i=1}^n E\Gamma_i^{-2/\alpha}W_i^2\mathbb{I}_{|\epsilon_i\Gamma_i^{-1/\alpha}W_i|\le \lambda} & \le C_1^{-2/\alpha} \sum_{i=1}^{\infty} i^{-2/\alpha} \int_{0}^{\infty} w^2\mathbb{I}_{w\le \lambda C_2^{1/\alpha}i^{1/\alpha}} dF(w) \\ & \le C\int_{0}^\infty x^{-2/\alpha} \int_0^{\lambda C_2^{1/\alpha}x^{1/\alpha}} w^2\,dF(w) \,dx \\ & = C\int_0^\infty w^2\int_{\lambda^{-\alpha}C_2^{-1}w^{\alpha}}^{\infty} x^{-2/\alpha}\,dx \,dF(w) \\ & = C'\int_0^{\infty} w^2\cdot (w^{\alpha})^{-2/\alpha+1} dF(w) \\ & = C'\int_{0}^{\infty} w^\alpha \,dF(w) <\infty \end{aligned}

Hence the series converges almost surely.

{Step-2:} Since the almost sure convergence is established. We will now show only that the distributional limit of the series is indeed a ${S\alpha S}$ random variable. Since the ${\Gamma_i}$‘s are dependent, we will eliminate ${\Gamma_i}$‘s in this step. We will use a well known result from Poisson process. Recall that ${\Gamma_1\le \Gamma_2 \le \Gamma_3 \le \cdots \le \Gamma_n}$ conditional on ${\Gamma_{n+1}}$ are distributed as ordered statistics from ${\mbox{Unif}(0,\Gamma_{n+1})}$. Hence

$\displaystyle \left(\frac{\Gamma_1}{\Gamma_{n+1}},\frac{\Gamma_2}{\Gamma_{n+1}},\ldots,\frac{\Gamma_n}{\Gamma_{n+1}}\right) \mid \Gamma_{n+1}$

is conditionally distributed as ordered statistics from ${\mbox{Unif}(0,1)}$. Now the conditional distribution does not depend on ${\Gamma_{n+1}}$. Thus

$\displaystyle \left(\frac{\Gamma_1}{\Gamma_{n+1}},\frac{\Gamma_2}{\Gamma_{n+1}},\ldots,\frac{\Gamma_n}{\Gamma_{n+1}}\right) \stackrel{d}{=} (U_{(1)},U_{(2)},\ldots,U_{(n)})$

and it is independent of ${\Gamma_{n+1}}$. Therefore

\displaystyle \begin{aligned} \sum_{i=1}^n \epsilon_i\Gamma_i^{-1/\alpha}W_i & = \frac1{\Gamma_{n+1}^{1/\alpha}}\sum_{i=1}^{n} \epsilon_i\left(\frac{\Gamma_i}{\Gamma_{n+1}}\right)^{-1/\alpha} W_i \\ & \stackrel{d}{=} \frac1{\Gamma_{n+1}^{1/\alpha}}\sum_{i=1}^{n} \epsilon_iU_{(i)}^{-1/\alpha} W_i \\ & = \frac1{\Gamma_{n+1}^{1/\alpha}}\sum_{i=1}^{n} \epsilon_iU_{i}^{-1/\alpha} W_i \\ & = \left(\frac{n}{\Gamma_{n+1}}\right)^{1/\alpha}\cdot \frac{1}{n^{1/\alpha}} \sum_{i=1}^{n} \epsilon_iU_{i}^{-1/\alpha} W_i \end{aligned}

Since by SLLN ${\displaystyle \frac{n}{\Gamma_{n+1}} \stackrel{a.s.}{\rightarrow} 1}$, it is enough to consider the series

$\displaystyle \frac{1}{n^{1/\alpha}}\sum_{i=1}^{n} \epsilon_iU_{i}^{-1/\alpha} W_i$

{Step-3:} Since we know that the original series converges almost surely it is enough to find the distributional limit of ${\displaystyle\frac{1}{n^{1/\alpha}}\sum_{i=1}^{n} \epsilon_iU_{i}^{-1/\alpha} W_i}$. Clearly we should apply the Generalised CLT. So it is enough to verify whether the random variables are symmetric and satisfy tail conditions. Since ${\epsilon_i}$ takes values ${\pm 1}$ with probability ${\frac12}$ each. It forces ${\epsilon_iU_i^{-1/\alpha}W_i}$ to be symmetric. Note that

\displaystyle \begin{aligned} P(|\epsilon_1U_1^{-1/\alpha}W_1|>\lambda) & = P(U_1^{-1/\alpha}|W_1|>\lambda) \\ & = P(U_1 < \lambda^{-\alpha}|W_1|^{\alpha}) \\ & = \int_0^\infty P(U_1<\lambda^{-\alpha}w^\alpha) \,dF(w) \\ & = \int_0^{\lambda} \lambda^{-\alpha}w^{\alpha}\,dF(w)+\int_{\lambda}^{\infty} dF(w) \\ & = \lambda^{-\alpha}\int_0^{\lambda} w^{\alpha}\,dF(w)+P(|W_1|>\lambda) \end{aligned}

Observe that

$\displaystyle 0\le \lambda^{\alpha}P(|W_1|>\lambda) \le E|W_1|^{\alpha}\mathbb{I}_{|W_1|>\lambda}$

The right side goes to zero as ${\lambda \rightarrow \infty}$ by DCT. Thus

$\displaystyle \lim_{\lambda\rightarrow \infty} \lambda^{\alpha}P(|\epsilon_1U_1^{-1/\alpha}W_1|>\lambda) = E|W_1|^{\alpha}$

Hence invoking Generalised CLT we get that distributional limit is a ${S\alpha S}$ random variable. This completes the proof. ${\square}$

Concluding Remarks

Let us summarize what we did so far. We introduce stable distributions, discuss some of its properties like characteristic functions, tail behaviour and moment properties and then proved the generalised CLT. However our last result, series representation is quite different. Just like multivariate Gaussian distribution, multivariate ${\chi^2}$, multivariate ${F}$, we have the notion of multivariate stable distributions. Multivariate distributions are needed to model multivariate data, In general we have the notion of stochastic process ${\{X_t\}_{t\in I}}$ which takes into account of infinite collection of random variables. We call a process to be a random field when the index set ${I}$ is multidimensional. A popular process / random field where the random variables are continuous is the Gaussian process, where any finite dimensional marginal is multivariate Gaussian. But one of the drawbacks of Gaussian process is that we need to assume finiteness of second moments. So in the case of heavy tailed data it is incorrect to use Gaussian process to model them. That is where Stable process / Stable random field comes into picture. Stable process have all finite dimensional marginals as multivariate stable. They are useful to model heavy tailed data and hence they have been well studied in literature. The series representation theorem is the starting point of ${S\alpha S}$ random field literature. It helps us to relate ${S\alpha S}$ with the well known Poisson process and hence helps us to derive some of the important results in ${S\alpha S}$ random field theory.

References

1. Durrett, Rick. Probability: theory and examples. Cambridge university press, 2010.
2. Feller, William. Introduction to Probability Theory and Its Appl. II. New York (1971).
3. Gut, Allan. Probability: a graduate course. Vol. 75. Springer Science & Business Media, 2012.
4. Roy, Parthanil. Maxima of stable random fields, nonsingular actions and finitely generated abelian groups: A survey. arXiv preprint arXiv:1702.00393 (2017).
5. Samorodnitsky, Gennady, and Murad S. Taqqu. Stable non-Gaussian random processes: stochastic models with infinite variance. Vol. 1. CRC press, 1994.