Wasserman's AoS, Chapter 8, Question 5

We solve question 5 from chapter 8 os Wasserman’s “All of Statistics” making the implicit assumption that \(X_1, \dotsc, X_n\) are iid. Other computation solutions can be found in the corresponding GitHub reepo.

Given \(X_1, \dotsc, X_n\), the random variable \(X_i^*\) can take any of those \(\le n\) values. With the assumption that there are no ties (i.e. there are n distinct values), \(X_i^*\) has a discrete uniform distribution over \(X_1, \dotsc, X_n\). (If there are ties, then the distribution is not uniform). We use this fact multiple times to obtain the result.

\[ \begin{align} \mathbb E (\bar X_n^* \vert X_1, \dotsc, X_n) &= \frac{1}{n} \sum_1^n \mathbb E (X_i^* \vert \dotsc) \\ &= \frac{1}{n} \sum_1^n \sum_1^n X_j \frac{1}{n} \\ &= \frac{1}{n} \sum_1^n \bar X_n \\ &= \bar X_n , \end{align} \]

where we used The Fact for the 2nd equality.

Moreover,

\[ \begin{align} \mathbb E (\bar X_n^*) &= \mathbb E \mathbb E (\bar X_n^* \vert X_1, \dotsc, X_n) \\ &= \mathbb E \bar X_n \\ &= \mu , \end{align} \]

where \(\mu := \mathbb E X_1\), assuming it exists.

For the conditional variance,

\[ \begin{align} \mathbb V (\bar X_n^* \vert X_1, \dotsc, X_n) &= \frac{1}{n^2} \sum_1^n \mathbb V (X_i^* \vert \dotsc) \\ &= \frac{1}{n^2} \sum_1^n \sum_1^n \frac{(X_j - \bar X_n)^2}{n} \\ &= \frac{1}{n} \sum_1^n \frac{(X_i - \bar X_n)^2}{n} \\ &= \frac{S_n}{n} . \end{align} \]

This has expectation \(\frac{\sigma^2}{n}\), where \(\sigma^2 := \mathbb V X_1\), assuming it exists.

Before calculating the variance, we require one more identity - the variance of the conditional expectation.

\[ \begin{align} \mathbb V \mathbb E (\bar X_n^* \vert X_1, \dotsc, X_n) &= \mathbb V \bar X_n \\ &= \frac{\sigma^2}{n} . \end{align} \]

The expression for the variance now follows from Theorem 3.27.

\[ \begin{align} \mathbb V (\bar X_n^*) &= \mathbb V \mathbb E (\bar X_n^* \vert X_1, \dotsc, X_n) + \mathbb E \mathbb V (\bar X_n^* \vert X_1, \dotsc, X_n) \\ &= \frac{\sigma^2}{n} + \frac{\sigma^2}{n} \\ &= 2 \frac{\sigma^2}{n} . \end{align} \]