Wasserman's AoS, Chapter 7, Question 5

We show that the covariance Cov(ˆFn(x),ˆFn(y)) is 1nF(x)(1+F(y)),

for x≤y. For x≥y, we have 1nF(y)(1+F(x)).

Using linearity of expectation and the fact that EˆFn(y)=F(y), we obtain

E(ˆFn(x)−F(x))(ˆFn(y)−F(y))=E(ˆFn(x)ˆFn(y))−F(x)F(y).

By definition of the empirical cumulative distribution function, the first term is equal to

1n2∑i,jE(I(Xi≤x)I(Xj≤y))=1n2(∑i=jE(I(Xi≤x)I(Xi≤y))+∑i≠jE(I(Xi≤x)I(Xj≤y))).

We now simplify the two summations:

1. Without loss of generality, we may assume that x≤y. With this assumption, I(Xi≤x)I(Xi≤y)=I(Xi≤x).

2. Note that the Bernoulli variables I(Xi≤x) and I(Xj≤y) are independent for i≠j since Xi and Xj are independent. This implies that the expectation of the product is the product of the expectations.

These two facts allow us to rewrite the first term as

1n2(∑i=jE(I(Xi≤x))+∑i≠jE(I(Xi≤x))E(I(Xj≤y)))=1n2(∑i=jF(x)+∑i≠jF(x)F(y))=1nF(x)(1+(n−1)F(y)).

Putting everything together, we see that the covariance is

1nF(x)(1+(n−1)F(y))−F(x)F(y)=1nF(x)(1−F(y))

for x≤y.