Wasserman's AoS, Chapter 8, Question 5

We solve question 5 from chapter 8 os Wasserman’s “All of Statistics” making the implicit assumption that X1,…,Xn are iid. Other computation solutions can be found in the corresponding GitHub reepo.

Given X1,…,Xn, the random variable X∗i can take any of those ≤n values. With the assumption that there are no ties (i.e. there are n distinct values), X∗i has a discrete uniform distribution over X1,…,Xn. (If there are ties, then the distribution is not uniform). We use this fact multiple times to obtain the result.

E(ˉX∗n|X1,…,Xn)=1nn∑1E(X∗i|…)=1nn∑1n∑1Xj1n=1nn∑1ˉXn=ˉXn,

where we used The Fact for the 2nd equality.

Moreover,

E(ˉX∗n)=EE(ˉX∗n|X1,…,Xn)=EˉXn=μ,

where μ:=EX1, assuming it exists.

For the conditional variance,

V(ˉX∗n|X1,…,Xn)=1n2n∑1V(X∗i|…)=1n2n∑1n∑1(Xj−ˉXn)2n=1nn∑1(Xi−ˉXn)2n=Snn.

This has expectation σ2n, where σ2:=VX1, assuming it exists.

Before calculating the variance, we require one more identity - the variance of the conditional expectation.

VE(ˉX∗n|X1,…,Xn)=VˉXn=σ2n.

The expression for the variance now follows from Theorem 3.27.

V(ˉX∗n)=VE(ˉX∗n|X1,…,Xn)+EV(ˉX∗n|X1,…,Xn)=σ2n+σ2n=2σ2n.