\n", "Pearson's cumulative test statistic $\\chi^2$ is a measure of the error between observations, $O_i$, and expected values, $E_i$:\n", "$$\\chi^2 = \\sum_{i} \\frac{\\left( O_i - E_i \\right)^2}{E_i}$$\n", "The numerator is a squared distance between observations, $O_i$, and expected values, $E_i$, and thus should be compared (at the denominator) with the expected squared fluctuations (variance) of the relative distribution. Why variance = $E_i$? \n", "The probability of getting exactly $k$ (independent) successes, each one with probability $p$, in $n$ trials is given by the Binomial distribution (see this Wikipedia link):\n", "$$Pr(X=k) = {{n}\\choose{k}} p^k (1-p)^{n-k} = \\frac{n!}{k! (n-k)!} p^k (1-p)^{n-k}$$\n", "The average of the Binomial distribution is $\\langle X \\rangle = np$, the variance is: $\\sigma^2 = np (1-p)$. Thus if $p$ is small we have that $\\sigma^2 \\simeq np$, that is $\\sigma^2 \\simeq \\langle X \\rangle$ and this explains the denominator $E_i$ in $\\chi^2$\n", "
\n", "\n", "3. Divide $[0,1]$ into $M$ identical sub-intervals and implement the $\\chi^2$ test. Obviously, the number of expected events observed in each sub-interval after $n$ *throws*, according to a uniform distribution, is $np = n\\times 1/M= n/M$. Fix $M=10^2$ and use for $n$ the first $10^4$ pseudo-random numbers, then the successive $10^4$ pseudo-random numbers, and so on ... 100 times. Plot $\\chi^2_j$ for $j=1, ..., 100$. In this case the chi-square statistic is:\n", "$$\\chi^2 = \\sum_{i=1}^M \\frac{\\left( n_i - n/M \\right)^2}{n/M}$$\n", "We should expect on average that $(n_i - n/M)^2 \\simeq n/M$ and thus $\\chi^2 \\simeq 100$, i.e. the number of sub-intervals.\n", "A larger value of $\\chi^2$ indicates that the hypothesis ($n_i$ are drawn from a uniform distribution) is rather unlikely ... but before judging, you could compare the histogram of the values obtained for $\\chi^2_j$ with the true probability distribution (see: this Wikipedia link)." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[\n",
" In computing the statistical uncertainties, i.e. the standard deviation of the mean, you need an estimation of the variance $\\sigma^2$, the second central moment. First of all note that the variance can be computed from\n",
" $$\\sigma^2_A := \\langle (A-\\langle A\\rangle)^2\\rangle = \\langle A^2 \\rangle -\\langle A\\rangle^2 $$\n",
" What is the best way to estimate $\\sigma^2_A$? Imagine that your Monte Carlo estimation of $A$ is obtained from a calculation which uses $M$ Monte Carlo \"steps\" (intentionally, here I am generic because what is a single \"step\" in a Monte Carlo calculation strictly depends on the specific calculation); you can always divide such $M$ Monte Carlo \"steps\" in $N$ blocks, with $N