Text: Dynamic Economics By Jerome Adda and Russell Cooper (2003) MIT Press.

Handbook Chapter: Structural Estimation of Markov Decision Processes by John Rust (1994) in R. Engle and D. McFadden Handbook of Econometrics volume 4, Elsevier, North Holland.

Practice Problems: Sample problems on dynamic programming and computation

Lectures 1-2: Theory

Readings:

• Rust, section 2
• Adda and Cooper, chapter 2
• John Rust (2000) Intro to decision theory and stochastic decision processes (slides, ps format) (slides, pdf format)
• Dimitri Bertsekas (1995) Dynamic Programming and Optimal Control Athena Scientific.
• Martin L. Puterman (1994) Markov Decision Processes: Discrete Stochastic Dynamic Programming Wiley.
• Larry Epstein and Stan Zin (1989) "Substitution, Risk Aversion, and the Temporal Behavior of Asset Prices" Econometrica 58 937-970.

Lectures 3-6: Computation

Readings:

• Adda and Cooper, chapter 3
• John Rust (1996) "Numerical Dynamic Programming in Economics" in H. Amman, D. Kendrick and J. Rust (eds.) Handbook of Computational Economics Elsevier, North Holland.
• John Rust (1996) Discrete DP algorithms from John Rust's Handbook of Econometrics chapter
• John Rust (1996) Continuous DP algorithms from John Rust's Handbook of Econometrics chapter
• John Rust (2000) lecture notes on Parametric Policy iteration Gauss code for computational experiments
• Hugo Benitez-Silva, George Hall, Gunter Hitsch, Giorgio Pauletto and John Rust (2000) ``A Comparison of Discrete and Parametric Approximation Methods for Continuous-State Dynamic Programming Problems''

Further readings:

• Larry Christiano and Jonas Fisher (1994) Algorithms for Solving Dynamic Models with Occasionally Binding Constraints working paper, Federal Reserve Bank of Minneapolis
• John Rust (2000) lecture notes on numerical dynamic programming
• John Rust (2000) lecture notes on learning algorithms and "neuro dynamic programming"
• John Rust (2000) lecture notes on minimum weighted residual and projection methods for solving DP problems
• John Rust (2000) lecture notes on lagrange multiplier methods for solving DP problems
• John Rust (2000) lecture notes on using randomization to break the curse of dimensionality
• John Rust (1997) lecture notes on numerical comparisons of random, quasi random and deterministic solution methods for continuous state DP problems
• Kenneth Judd (1998) "Numerical Methods in Economics MIT Press.
• Dimitri Bertsekas and John Tsitsiklis (1996) Neuro-Dynamic Programming Athena Scientific.
• John Rust (1997) "Using Randomization to Break the Curse of Dimensionality" Econometrica 65-3 487-516.
• John Rust (1997) "A Comparison of Policy Iteration Methods for Solving Continuous-State, Infinite-Horizon Markovian Decision Problems Using Random, Quasi-random, and Deterministic Discretizations" manuscript, umd University.
• John Rust, Joseph Traub, and Henryk Wozniakowski (1999) "Is There a Curse of Dimesionality for Contraction Fixed Points in the Worst Case?" manuscript, umd University.

Lectures 7-8: Empirical Applications: Discrete Decision Processes

Discrete Decision Processes are Problems where the choice variable is restricted to a finite set of alternatives. I describe parametric econometric methods for inferring the unknown parameters of these processes, and survey some of the numerous applications of these models in many different parts of economics.

Readings:

• John Rust (1994) Structural Estimation of Markov Decision Processes in R. Engle and D. McFadden (eds.) Handbook of Econometrics, North Holland: Amsterdam. (sections 3 and 4)
• John Rust (1987) "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher" Econometrica 55-5 999-1033.
• Ariel Pakes (1986) "Patents as Options: Some Estimates of the Value of Holding European Patent Stocks Econometrica 54-4 755-784.
• Kenneth Wolpin (1987) "Estimating a Structural Search Model: The Transition from School to Work" Econometrica 55-4 801-817.
• Mita Das (1992) "A Micro-Econometric Model of Capital Utilization and Retirement: The Case of the U.S. Cement Industry" Review of Economic Studies 59-2 277-297.
• Michael Keane and Kenneth Wolpin (1997) "The Career Decisions of Young Men" Journal of Political Economy 105-3 473-522.
• John Rust and Geoffrey Rothwell (1997) On the Optimal Lifetime of Nuclear Power Plants Journal of Business and Economic Statistics 15-2 195-208.
• John Rust and Christopher Phelan (1997) "How Social Security and Medicare Affect Retirement Behavior in a World of Incomplete Markets" Econometrica 65-4 781-832.
• Victor Aguirregabiria and Pedro Mira (2002) Swapping the Nested Fixed Point Algorithm: A Class of Estimators for Discrete Markov Decision Models Econometrica 70-4 1519-1543.

Lectures 9-10: Empirical Applications: Continuous Decision Processes

Readings:

• Adda and Cooper, chapters 5-10
• George Hall and John Rust (2003) Simulated Minimum Distance Estimation of a Model of Optimal Commodity Price Speculation with Endogenously Sampled Prices
• John Rust (2003) Inverse Problems and Nonparametric Identification of Dynamic Structural Models: The Case of Commodity Price Speculation
• George J Hall (2000) Non-Convex Costs and Capital Utilization: A Study of Production Scheduling at Automobile Assembly Plants"" Journal of Monetary Economics 45 681-716.
  
• Lars Hansen and Kenneth Singleton (1982) "Generalized Instrumental Variables Estimation of Nonlinear Rational Expectations Models" Econometrica 50-5 1269-1286.
• Darrell Duffie and K.J. Singleton (1993) "Simulated Moments Estimation of Markov Models of Asset Prices" Econometrica 61-4 929-952.
• Gourieroux, C. and A. Monfort (1995) Simulation Based Econometric Methods Core Lecture Series, Core Foundation, Universite Catholique de Louvain.
• Ariel Pakes and David Pollard (1989) "Simulation and the Asymptotics of Optimization Estimators" Econometrica 57 1027-1057.
• A. Ronald Gallant and George Tauchen (1996) "Which Moments to Match?" Econometric Theory 12-4 657-681.
• A. Ronald Gallant and George Tauchen (1995) "EMM: A Program for Efficient Method of Moments Estimation" manuscript, Duke University.
• Haiyan Shui and Laurence Ausubel (2003) Time Inconsistency in the Credit Card Market

Lectures 11-12: Estimation of Dynamic Games

I extend the single agent decision framework to multi-agent dynamic games. This is much harder and is at the current frontier of research in this area. We will discuss several recent papers that make substantial headway on these topics.

Readings:

• Martin Pesendorfer and Philip Schmidt-Dengler (2003) Identification and Estimation of Dynamic Games
• Victor Aguirregabiria and Pedro Mira (2002) Sequential Simulated-Based Estimation of Dynamic Discrete Games
• Ariel Pakes and Paul McGuire (2000) Stochastic Approximation for Dynamic Analysis: Markov Perfect Equilibrium and the Curse of Dimensionality
• Audra Bowlus and Shannon Seitz (2002) Domestic Violence, Employment and Divorce Queen's University, Canada
• Eugenio Giolito (2003) A Search Model of Marriage with Differential Fecundity University of Maryland.
• Natasha Zhang Foutz and Vrinda Kadiyali (2003) Competitive Dynamics in the Release Date Pre-announcements of Motion Pictures Cornell University, Johnson School of Management