Something that has been bothering me for a long time is the diagrammatic connection between moments and the connected moments. Or the moments of a distribution and the cumulants:
\begin{eqnarray}
\langle x \rangle \sim \langle x \rangle_c
\end{eqnarray}
The simplest way to deal with this is to write the moment generating function and cumulant generating function in a Taylor series. For the moment generating function, or should I say characteristic function we have:
\begin{eqnarray}
\phi(k) &=& \int P(x) e^{ikx}dx \\
&=& \sum_{n=0}^\infty \frac{(-ik)^n}{n!}\langle x^n \rangle
\end{eqnarray}
and for the cumulant generating function:
\begin{eqnarray}
\psi(k) &=& \log \phi(k) \\
&=& \sum_{n=0}^\infty \frac{(-ik)^n}{n!}\langle x^n \rangle_c
\end{eqnarray}
Equating these two expressions, we have
\begin{eqnarray*}
\sum_{n=0}^\infty \frac{(-ik)^n}{n!}\langle x \rangle &=& e^{\sum_{n=0}^\infty \frac{(-ik)^n}{n!}\langle x^n \rangle_c}
\end{eqnarray*}
The sum in the exponential on the right hand side is really just a product, which I can write as:
\begin{eqnarray*}
\sum_{m=0}^\infty \frac{(-ik)^m}{m!}\langle x^m \rangle &=& \prod_{n=1}^\infty \left( e^{\frac{(-ik)^n}{n!}\langle x^n \rangle_c }\right) \\
&=&\prod_{n=1}^\infty \left( \sum_p \frac{(-ik)^{np}}{p!(n!)^p}\langle x^n \rangle_c^p \right)
\end{eqnarray*}
So this is an interesting comparison. Since both expansions rely on certain powers -- or should I say a certain number of derivatives -- of $k$. If we equate the two sides under the condition
\begin{eqnarray}
\sum p n = m
\end{eqnarray}
where, $p$ is different for every $n$, so we had better write:
\begin{eqnarray}
\sum p_n n = m
\end{eqnarray}
we get:
\begin{eqnarray*}
\langle x^m \rangle &=& \sum_{\lbrace p_n \rbrace}m! \prod_{n=1}^\infty \frac{1}{p_n! (n!)^{p_n}}\langle x^n \rangle_c^{p_n}
\end{eqnarray*}
Where we sum over the set $p_n \epsilon \lbrace p \rbrace $ that satisfies $\sum p_n n = m$. Kardar says that this combinatoric factor is just the number of ways we can break up $m$ objects into $p_n$ clusters, each of $n$ components. When you think about it, this works. Take four points and split them into one cluster of two points, $p_2=1$ and two ``clusters'' of one point, $p_1=2$. We satisfy $\sum p_n n = m$ since $p_2+p_1=4$. Now, the points, as a whole, can be ordered in $4!$ ways, but once we bunch them into 1 cluster of $2$ points, we find that those remaining two points are indistinguishable ``inside the cluster''. So we get a combinatoric factor of 6. Similarly, there are 3 ways of breaking four points into two clusters of two points, one way of breaking four points into four clusters of four points, one way of breaking four points into one cluster of four points, etc.
You could go on and on like this until your head falls off. But this finally makes explicit -- and, kind of intuitive -- the connection between cumulants and moments.
So, the question becomes, \textbf{why define cumulants anyway}? \textbf{What are their properties}? One of the things I've learned is that cumulants add under convolution, so when we add two random variables, and thereby convolve their probability densities, we get a multipliciation of the characteristic functions (by the convolution theorem). And this means that
\begin{eqnarray}
P(z=x_1+x_2) &=& P(x_1) \star P(x_2) = \int P(\tau)P(z-\tau)d \tau \\
\phi(k_1,k_2) &=& \phi(k_1)\phi(k_2) = e^{\psi(k_1)+\psi(k_2)}
\end{eqnarray}
So cumulants, as I'm beginning to learn in Thermodynamics are \textbf{extensive}. The cumulant generating function itself is \textbf{extensive}. Just like the partition function $Z$ is \textbf{extensive}. This extensivity property has very deep implications because when we take the average of a sum of independently identically distributed random variables:
\begin{eqnarray}
Z &=& \frac{x_1+x_2+\dots x_N}{N}
\end{eqnarray}
The cumulant generating function of that sum goes like the sum as well:
\begin{eqnarray}
\psi(k) &=& \sum_{i=1}^N \psi_i(k)
\end{eqnarray}
and therefore, the variance of the mean is not extensive but inversely extensive, as many many mathematicians know because of the law of large numbers:
\begin{eqnarray}
\sigma^2_{x_1+x_2+\dots} &=& N \sigma_{x} \\
\sigma^2_Z &=& \frac{\sigma^2}{N}
\end{eqnarray}
Higher order cumulants, as $N$ gets large will vanish even faster:
\begin{eqnarray}
\langle Z^n \rangle_c \sim \frac{1}{N^{n-1}}
\end{eqnarray}
And this is another way to prove the the law of large numbers. All higher order cumulants will diminish faster than $\frac{1}{N}$, so the leading order contribution comes from variance. This convergence of averages is a very powerful tool in Statistical mechanics, where $N$ is absurdly large, like $10^{23}$.
------------------------------------------
The central limit theorem, I am still convinced, relies on something else, which is the boundedness above of the characteristic function $\phi(k)$ by unity. Under many many multiplications, this multivariate $\phi(k)$ collapses about the origin, and one is well justified in Taylor expanding in $k$ only to second order, which is, a Gaussian.
-----------------------------------------
Now back to this cluster expansion. This formula, the combinatoric factor for breaking $m$ balls into $p_n$ clusters of $n$ constituents is a very important concept, because it is, in the multivariate case, what underlies Wick's theorem in field theory. Field Theory is so heavily based on Gaussian statistics because all free fields have Gaussian properties, so it is often easy to forget how to generally break down a multi-variate cumulant, but the key lies in this cluster expansion.
Let's see if we can do the cluster expansion in the multi-variate case. The characteristic function can be expanded as
\begin{eqnarray}
\phi(\vec{k}) &=& \sum_{m=0}^\infty \frac{(-i)^m}{m!} \prod_{i=1}^N \left(k_i^{m_i}\right)\langle x_1^{m_1}x_2^{m_2}\dots x_N^{m_N} \rangle
\end{eqnarray}
equating this with the cumulant generating function, we get something quite nasty:
\begin{eqnarray*}
\phi(\vec{k}) &=& \prod_{n=1}^\infty \left(\sum_{p_n=0}^\infty \frac{(-i)^{np_n}}{(n!)^{p_n}p_n!} \left(\prod_{i=1}^N k_i^{n_i}\right)^{p_n} \langle x_1^{n_1}x_2^{n_2}\dots x_N^{n_N} \rangle_c \right)
\end{eqnarray*}
Equating like powers of $k_1,k_2,k_3,\dots$ separately, I get the condition
\begin{eqnarray}
\sum_n p_n n_i &=& m_i \ \forall i \epsilon [1,N] \\
\sum_n p_n n &=& m
\end{eqnarray}
Which is equivalent of saying, in each expectation value, we require the same total number of random variables and the same number of each type of random variable $x_i$. The combinatoric factors in this case might be tricky, since we are equating:
\begin{eqnarray*}
\sum_{m=0}^\infty \frac{(-i)^m}{m!} \prod_{i=1}^N \left(k_i^{m_i}\right)\langle x_1^{m_1}x_2^{m_2}\dots x_N^{m_N} \rangle &=& \prod_{n=1}^\infty \left(\sum_{p_n=0}^\infty \frac{(-i)^{np_n}}{(n!)^{p_n}p_n!} \left(\prod_{i=1}^N k_i^{n_i}\right)^{p_n} \langle x_1^{n_1}x_2^{n_2}\dots x_N^{n_N} \rangle_c^{p_n} \right)
\end{eqnarray*}
When we look at the requirement $\sum_i n_i = n$, we realize that there are $\frac{n!}{n_1! n_2! \cdots n_N!}$ ways -- a multinomial combinatoric -- to do this. So each term on the right should be weighted by this factor:
\begin{eqnarray*}
\langle x_1^{m_1}x_2^{m_2}\dots x_N^{m_N} \rangle &=& \sum_{\lbrace p \rbrace} m! \prod_{n=1}^\infty \left(\frac{1}{(n!)^{p_n}p_n!} \left(\frac{n!}{n_1! n_2! \cdots n_N!} \right) \langle x_1^{n_1}x_2^{n_2}\dots x_N^{n_N} \rangle_c^{p_n} \right)
\end{eqnarray*}
This is an intimidating formula, and may not be correct.
No comments:
Post a Comment