Econ 551: Note on A/R Sampling
Professor John Rust, Department of Economics, Yale University

A/R sampling (short for acceptance-rejection sampling) is a way of generating random draws from a density that is hard to sample from by drawing from another envelope or majorizing density that is easy to generate random draws from. The majorizing density g must satisfy the following condition:

where . Note that to do A/R method we must know some constant a satisfying inequality (1), since the A/R procedure works as follows

1.

Draw a uniform random draw .

2.

Draw a random draw , i.e. a random draw from the majorizing density.

3.

if reject the draw and go back to step 1. Otherwise stop, accept the draw, and set as a random draw from .

Claim: is a random draw from .

Proof: We need to show that

where the probability on the left hand side of equation (2) is the conditional probability that given that the A/R simulation procedure has stopped. However that probability can be written as:

The probability in the numerator of (3) can be written as

But the probability of stopping given a draw is just

since is uniformly distributed and independent of . Substituting equation (5) into equation (4) we get:

Letting we get

Substituting equations (6) and (7) into equation (3) we get the result in equation (2).

Note that does not have to integrate to 1 for the A/R method to work: the normalizing factor necessary for to be a valid probability density, , can be absorbed into the constant a. However since we need to know an actual value of a for the A/R method to work, we must at least have an estimate of the upper bound on this normalizing factor to determine an appropriate constant a satisfying inequality (1). Note how the normalizing factor ends up multiplying a:

So that even if we don't know the normalizing factor but can guess an upper bound for it, then we can still do A/R sampling to draw from the density computed by dividing p by its normalizing factor. We can always use a very large value of a, but the cost is a low probability of acceptance (probability of stopping) so the A/R sampling method can be very inefficient and time consuming. This is why Markov Chain Monte Carlo methods such as Gibbs sampling and the Metropolis-Hastings methods are typically used instead of A/R in order to simulate draws from a posterior density in Bayesian econometrics.

• About this document ...