Finger exercises for Week 13 — To be discussed Monday Nov 25, 2002

1. We continue with the data Y

1.2 2.4 1.3 1.3 0.0 1.0 1.8 0.8 4.6 1.4 ,

modeled as iid observations y from a Cauchy likelihood

f(y|θ, s) = \frac{1}{πs} \frac{1}{1 + (\frac{y−θ}{s} )^{2}}

with s fixed. Assume now that s = 0.1 and that θ has a prior distribution, θ ∼ N(1, 1).

(a) Graph the log-posterior density log p(θ|Y) (or something log-proportional to it) for the Cauchy data above.

(b) Compute an approximate 95% credible interval for θ, using the posterior mode θ̃ and −\frac{d^{2}}{dθ^{2}} log p(θ̃|Y), using whatever tools are convenient (optimization functions in R or Splus, numerical or symbolic differentiation, etc.)

(c) Use the R/Splus function integrate to calculate the coverage probability for the interval calculated in part 1b. In other words, calculate P (θ ∈ [a, b]|Y), where [a, b] is the interval you calculated in part 1b.

(d) Try calculating E[θ|Y] and \sqrt{Var(θ|Y)} using SPLUS’s integrate function. Is the

approximate 95% credible interval based on E[θ|Y] and \sqrt{Var(θ|Y)} much different from the interval in part 1b? Is the coverage probability much different?

(e) To explore what happens to numerical integrals when the likelihood becomes more and more peaked, recalculate E[θ|Y] and \sqrt{Var(θ|Y)}, when the dataset Y is replaced with

• The same dataset, with each observation repeated 2 times.

• The same dataset, with each observation repeated 3 times.

• The same dataset, with each observation repeated 5 times.

• The same dataset, with each observation repeated 10 times.

• The same dataset, with each observation repeated 20 times.

• The same dataset, with each observation repeated 30 times.

and describe/explain your findings.

2. The inverse-CDF method says that if U ∼ Unif[0, 1] then X = F^{−1}(U) will have CDF P [X ≤ x] = F (x). If X is a discrete random variable taking on the values 0, 1, 2, … then F (x) is a step function: F (x) ≡ F (k) for all x ∈ [k, k +1). This suggests a “search” method of generating X: First generate U ∼ Unif[0, 1]; then find the k such that F (k) ≤ U < F (k + 1), and return X = k.

(a) Show that this “search” method produces a random draw from X ∼ F (x).

(b) Implement the method in an Splus or R function to generate random draws from a Poisson(λ) random variable. Show that it works.

(c) Discuss at least two drawbacks of the method.

3. In the rejection methods, on average, how many pairs (Y, U) have to be drawn, before Y is accepted as an observation from the “target” density f(x)? What does this say about what the relationship between the “target” density f(x) and the “convenient” density g(x) should be?

4. Show that you can generate X ∼ N(0, 1) by applying the rejection method to Cauchy observations Y = tan(π(U − 1/2)). What is (the \underline{best}) a? Show (analytically) that it does \underline{not} work to generate X ∼ Cauchy by applying the rejection method to N(0, 1) observations Y. What goes wrong???

5. Can you develop a rejection method for discrete random variables (the method for continuous rv’s was discussed in class). Does the “convenient” density g(x) have to be for a discrete random variable?

6. In the notes and in class we discussed two versions of importance sampling, one involving

Ĩ_{n} = \frac{1}{n} ∑_{1}^{n} h(Y_{i})f(Y_{i})/g(Y_{i}) → E_{f}[h(X)] , as n → ∞

and the other involving Î_{n} = Ĩ_{n}/Õ_{n} where

Õ_{n} = \frac{1}{n} ∑_{1}^{n} 1 · f(Y_{i})/g(Y_{i}) → 1 , as n → ∞

Apply the delta method to the function g(x, y) = x/y, where x = \frac{1}{n} Ĩ_{n} and y = \frac{1}{n} Õ_{n}, to show that

Var(Î_{n}) ≈ Var(Ĩ_{n}) − 2E_{f}[h(X)]Cov(Ĩ_{n}, Õ_{n}) + (E_{f}[h(X)])^{2}Var(Õ_{n})

Under what conditions could we expect Var(Î_{n}) ≤ Var(Ĩ_{n})?