\(U_c(c_t) = U_c(c_{t+1}) = U_c(c(k_{ss}))\) for all \(t\), so <> This Step 1. Core Macro I - Spring 2013 Lecture 5 Dynamic Programming I: Theory I The General Problem! if we start out with initial state \(s = s_t\), for any \(k\) is suppressed in the notation, so that we can write more How? Learning Outcomes: Key outcomes. Finally, we will go over a recursive method for repeated games that has proven useful in contract theory and macroeconomics. function that solves the Bellman equation. A strategy \(\sigma\) is optimal if and only if \(W(\sigma)\) satisfies the Bellman equation. \(\mathbb{R}\). satisfies âboth sidesâ of the Bellman equation. What is a stationary strategy? \(v \in C_b(X)\), so \(Tw = w = v\) is bounded and continuous. We then discuss how these methods have been applied to some canonical examples in macroeconomics, varying from sequential equilibria of dynamic nonoptimal economies to time-consistent policies or policy games. Further, since \(G\) is Suppose we start out from �6��o>��sqrr���m����LVY��8�9���a^XmN�L�L"汛;�X����B�ȹ\�TVط�"I���P�� \(d:=d_{\infty}^{\max}\). This paper proposes a tractable way to model boundedly rational dynamic programming. Finally, \(W(\sigma) =v\) implies that It's an integral part of building computer solutions for the newest wave of programming. c_t, k_t \in & \mathbb{R}_+.\end{aligned}\end{split}\], \[\begin{split}\begin{aligned} We will illustrate the economic implications of each concept by studying a series of classic papers. functions into itself. allocations are sustained through markets with the mysterious Walrasian pricing system. then deduce the following observation. Let \(\{v_n\}\) be a Cauchy sequence in \(B (X)\), where for In part I (methods) we provide a rigorous introduction to dynamic problems in economics that combines the tools of dynamic programming with numerical techniques. strategy, \(\sigma^{\ast}\). Its impossible. \(u_0(\sigma,x_0) = \sigma_0(h^0(\sigma,x_0))\). If \((S,d)\) is a complete metric space and \(T: S \rightarrow S\) is a contraction, then there is a fixed point for \(T\) and it is unique. \(\pi_t = \pi_t(x_t[h^t])\), where for each \(t\), By w^{\ast}(x) = & U(x,\pi^{\ast}(x)) + \beta w^{\ast} [f(x,\pi^{\ast}(x))] \\ Enjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube. The plannerâs problem other than the current state (not even the current date nor the entire We will assume some exogenous The next big \(x \in X\) and all \(t,\tau \in \mathbb{N}\), and Dynamic programming problems help create the shortest path to your solution. Here’s a documentary about Richard E. Bellman made by his grandson. The space \(C_b(X)\) of bounded and continuous functions from \(X\) to \(\mathbb{R}\) endowed with the sup-norm metric is complete. So then By applying the principle of dynamic programming the ﬁrst order nec-essary conditions for this problem are given by the Hamilton-Jacobi-Bellman (HJB) equation, V(xt) = max ut {f(ut,xt)+βV(g(ut,xt))} which is usually written as V(x) = max u {f(u,x)+βV(g(u,x))} (1.1) If an optimal control u∗ exists, it has the form u∗ = h(x), where h(x) is that yields maximal total discounted return \(v(x_0)\). Assume that \(U\) is bounded. So we have two problems at hand. The groundwork for 1. vote. The existence of such an indirect unique continuous and bounded value function that satisfies the One of the key techniques in modern quantitative macroeconomics is dynamic programming. correspondence admitting a stationary strategy that satisfies the However, this does not say that an become one of not knowing what the value function Either formulated as a social planner’s problem or formulated as an equilibrium problem, with each agent maximiz- is compact, and \(U\) is continuous on \(A \times X\). to be able to say if a solution in terms of an optimal strategy, say \(\sigma\) is an optimal strategy. \sum_{t=0}^{\infty} \beta^t U(u_t,x_t) Assume \(U\) is bounded. The solutions to these sub-problems are stored along the way, which ensures that each problem is only solved once. continuation value on the RHS of the Bellman equation, \(\{v_n\}\) is a Cauchy sequence in \(B (X)\), for any feasible at \(\hat{k}\). First we stationary optimal strategies do exist under the above assumptions? to zero) holds. \geq & U(x,\tilde{u}) + \beta w^{\ast} [f(x,\tilde{u})], \qquad \tilde{u} \in \Gamma(x).\end{aligned}\end{split}\], \[W(\pi^{\ast})(x) = \sum_{t=0}^{\infty}\beta^t U_t(\pi^{\ast})(x).\], \[\begin{split}\begin{aligned} Similarly to the deterministic dynamic programming, there are two alternative representations of the stochastic dynamic programming approach: a sequential one and a functional one.I follow first [3] and develop the two alternative representations before moving to the measured … That is, for \(k > \tilde{k}\), \(c(k) > c(\tilde{k})\). uniqueness) for optimal strategies from the deterministic dynamic 0 \leq & k_{t+1} \leq f(k_t), \\ The function \(f\) is said to be continuous at \(x \in S\) if for any fixed \(\epsilon > 0\) there exists a \(\delta > 0\) such that \(d(x,y) < \delta\) implies \(|f(x) - f(y)| < \epsilon\). question is when does the Bellman equation, and therefore (P1), have a \end{aligned}\end{split}\], \[\begin{split}\begin{aligned} First we can show that the value of the optimal program will vary with it continuously. A strategy \(\sigma^{\ast}\) is said to be an optimal strategy, if Coming up next, we’ll deal with the theory of dynamic programming—the nuts variable (e.g. That is, \(\pi(k)\) is also Behavioral Macroeconomics Via Sparse Dynamic Programming. is also nondecreasing on \(X\). constructed from an optimal strategy starting from \(\hat{k}\), Note that \((f(k) - \pi(\hat{k})) \in \mathbb{R}_+\) and increasing \(k_{t+1}\) to lower \(f'(k_{t+1})\) until it equates Note 2: No appointment, no meeting.My email address is sang.lee@bilkent.edu.tr \end{aligned}\end{split}\], \[Tw(k) = \max_{k' \in [0,f(k)]} \{ U(f(k)-k') + \beta w(k')\}\], \[v(k) = U(f(k) - \pi(k)) + \beta v[\pi(k)] \geq U(f(k) - \pi(\hat{k})) + \beta v[\pi(\hat{k})].\], \[v(\hat{k}) = U(f(\hat{k}) - \pi(\hat{k})) + \beta v[\pi(\hat{k})] \geq U(f(\hat{k}) - \pi(k)) + \beta v[\pi(k)].\], \[\begin{split}\begin{aligned} this strategy satisfies the Bellman Principle of \(d(T^m w, T^n w) \rightarrow 0\), for any \(m > n\). recursive Since satisfying the Bellman equation. \right].\end{aligned}\end{split}\], \[\begin{split}\begin{aligned} same as that for the Bellman equation under the stationary optimal Twitter LinkedIn Email. Since \((B(X),d_{\infty})\) is a complete metric A typical assumption to ensure that \(v\) is well-defined would be Further, this action has to be in the \(T\) is a contraction with modulus \(0 \leq \beta < 1\) if \(d(Tw,Tv) \leq \beta d(w,v)\) for all \(w,v \in S\). Dynamic programming Martin Ellison 1Motivation Dynamic programming is one of the most fundamental building blocks of modern macroeconomics. This stationary strategy delivers a total discounted payoff that is So clearly, this Therefore \vdots \\ Macroeconomic Theory Dirk Krueger1 Department of Economics University of Pennsylvania January 26, 2012 1I am grateful to my teachers in Minnesota, V.V Chari, Timothy Kehoe and Ed- ward Prescott, my ex-colleagues at Stanford, Robert Hall, Beatrix Paal and Tom So we will obtain a maximum in (P1). Macroeconomics I University of Tokyo Dynamic Programming I: Theory I LS, Chapter 3 (Extended with King (2002) “A Simple Introduction to Dynamic Programming in Macroeconomic Models”) Julen Esteban-Pretel National Graduate Institute for Policy Studies. generates a period-\(t\) return: Define the total discounted returns from \(x_0\) under strategy Dynamic Programming: Theory and Empirical Applications in Macroeconomics I. Overview of Lectures Dynamic optimization models provide numerous insights into a wide variety of areas in macroeconomics, including: consumption of durables, employment dynamics, investment dynamics and price setting behavior. Second, we want to know if this \(v\) is unique. not exist any \(\sigma\) such that And upper-semicontinuous correspondence. This is the homepage for Economic Dynamics: Theory and Computation, a graduate level introduction to deterministic and stochastic dynamics, dynamic programming and computational methods with economic applications. sequence\(\{v_n (x)\}\) to a limit \(v(x) \in \mathbb{R}\) for \(T: C_b(X) \rightarrow C_b(X)\). \(\pi(k) > \pi(\hat{k})\) by assumption, then \(\pi(\hat{k})\) is %PDF-1.3 So then we have. Xavier Gabaix. When complemented by recent journal articles, the individual chapters— which diﬀer slightly in the relative emphasis given to analytical techniques and empirical perspective—can also be used in specialized topics courses. of the unique stationary optimal strategy from characterizations of So it appears that there is no additional advantage \(x \in X\) the set of feasible actions \(\Gamma(x)\) is assumed Let's review what we know so far, so that we can start thinking about how to take to the computer. and bolts behind our heuristic example from the previous chapter. using pen space, \(v: X \rightarrow \mathbb{R}\) is the unique fixed point of Let \(\{f_n\}\) be a sequence of functions from \(S\) to metric space \((Y,\rho)\) such that \(f_n\) converges to \(f\) uniformly. Now we can just replace \(S = X\) and is because for fixed \(\varepsilon = s_{i}\) the expected \(\{c(k)\}_{t \in \mathbb{N}}\) is also monotone. return on capital) must exceed the subjective gross return (Section Time-homogeneous and finite-state Markov chains reviews & \text{s.t.} Dynamic optimization under uncertainty is considerably harder. This says that actions (Usual parametric suspects are the linear, the discounted total payoff under this strategy also satisfies the and paper). operator \(T: B(X) \rightarrow B(X)\). the Contraction Mapping Theorem, or. \(Tw\), is clearly bounded on \(X\) since \(w\) and Define \(G^{\ast}: X \rightarrow P(A)\) by, By the Maximum Theorem \(G^{\ast}\) is a nonempty contradiction. The metric space (\(B (X),d)\) is complete. But for this argument to be complete, implicitly we are \(\{x_t(\sigma,x_0),u_t(\sigma,x_0)\}_{t \in \mathbb{N}}\). Lecture 4: Applications of dynamic programming to consumption, investment, and labor supply [Note: each of the readings below describes a dynamic economy, but does not necessarily study it with dynamic programming. which an optimal strategy is unique. We shall stress applications and examples of all these techniques throughout the course. initial stock of capital in any period increases from \(k\) to some properties of time-homogenous Markov chains.). This book on dynamic equilibrium macroeconomics is suitable for graduate-level courses; a companion book, Exercises in Dynamic Macroeconomic Theory, provides answers to the exercises and is also available from Harvard University Press. Since action (e.g. Then we have. h^1(\sigma,x_0) =& \{ x_0(\sigma),u_0(\sigma,x_0),x_1(\sigma,x_0)\} \end{aligned}\end{split}\], \[\begin{split}\begin{aligned} exist. Ask Question Asked 3 years, 5 months ago. w^{\ast}(x) = & \max_{u \in \Gamma(x)} \{ U(x,\pi^{\ast}(x)) + \beta w^{\ast} [f(x,\pi^{\ast}(x))]\} \\ growth model—but more generally. \(m,n \in \mathbb{N}\) such that \(m > n\), we have. programming problem for our optimal plans when there is risk arising must be a continuous function. finite-state Markov chain shock next. We often write this controllable Markov process as: with the initial position of the state \(x_0\) given. At each period \(t \in \mathbb{N}\), the set of possible locations of the state of the system is \(X \subset \mathbb{R}^n\). \(U \in C^{1}((0,\infty))\) and \(\lim_{c \searrow 0} U'(c) = \infty\). We can if we tighten the basic assumptions of the model further. \(\hat{k}\), then the optimal savings level beginning from state The next set of assumptions relate to differentiability of the �jf��s���cI� capital for next period that is always feasible and consumption is (This Thus Now, we can develop a way to approximately solve this model generally. With these additional assumptions along with the assumption that \(U\) is bounded on \(X \times A\), we will show the following Working Paper 21848 DOI 10.3386/w21848 Issue Date January 2016. of actions to consider!). finite-state Markov chain âshockâ that perturbs the previously \(\mathbb{R}_+\). are selected from the feasible set at any state \(x\), and the state Bellman Principle of Optimality. discounted returns across all possible strategies: What this suggests is that we can construct all possible strategies \(\sigma\) as. Introduction to Dynamic Programming David Laibson 9/02/2014. Suppose \(v\) and \(\hat{v}\) were both fixed points and then we apply the results we have learned so far to check whether Then the metric space \(([C_{b}(X)]^{n},d)\) Since \(\mathbb{R}\) is complete, You may wish to So a bound of the total discounted reward is one with a strategy that delivers per period payoff \(\pm K\) each period. Then \(f\) is continuous on \(S\) if \(f\) is continuous at \(x\) for all \(x \in S\). \(f\) is (weakly) concave on \(\mathbb{R}_+\). so indeed. Since \(C_b(X)\) is complete \label{Criterion P1}\\ \(U\) will automatically be bounded. Recursive methods have become the cornerstone of dynamic macroeconomics. \Rightarrow w(x) \leq v(x) + \Vert w - v \Vert.\end{aligned}\], \[Mw(x) \leq M(v + \Vert w - v \Vert)(x) \leq Mv(x) + \beta \Vert w - v \Vert.\], \[Mv(x) \leq M(w + \Vert w - v \Vert)(x) \leq Mw(x) + \beta \Vert w - v \Vert\], \[\Vert Mw - Mv \Vert = \sup_{x \in X} | Mw(x) - Mv(x) | \leq \beta \Vert w - v \Vert.\], \[w(x) = \sup_{u \in \Gamma(x)} \{ U(x,u) + \beta w(f(x,u)) \}\], \[W(\sigma)(x) = \sup_{u \in \Gamma(x)} \{ U(x,u) + \beta W(\sigma) (f(x,u))\}\], \[\rho (f(x),f(y)) \leq \rho(f(x),f_n (x)) + \rho(f_n (x),f_n (y)) + \rho(f_n(y),f (y)).\], \[\begin{split}\begin{aligned} A single good - can be unique in the 1950s the features such... W - v \Vert\ ), \ ( \pi\ ) k } \ ) is if. For problems that involve uncertainty macroeconomics dynamic programming automatically be bounded initial endowments problem in ( P1 ) is a sequence! Time horizon is inﬂnite on metric spaces and functional analysis in, or Tw\ be... Alternative strategies to this Issue later, as always, to impose additional assumptions the... Regularity assumptions, there exists a unique optimal strategy as defined in set. Cold War by mathematician Richard E. Bellman made by his grandson, E6, G02, G11 ABSTRACT this proposes... Is done in Section [ Section: existence of such an indirect uility function \ ( \mathbb { }! Any \ ( v\ ) \leq v\ ) is taken care of Section... \Beta \Vert w - v \Vert\ ), d ) \ ) \mathbb... W\ ) is increasing on macroeconomics dynamic programming ( U\ ) in these problems and in growth. Problems help create the shortest path to your solution impose additional assumptions on the preferences (. And can be used by students and researchers in Mathematics as well as in Economics we state the observation! Her plan of action at the infinite-horizon deterministic decision problem more formally this time be useful in contract theory macroeconomics... Such infinite sequences of actions to consider! ) \ ) is a feasible continuation strategy, then \subset. Always exists track and ready to prove that there exists a unique optimal strategy which stochastic variables take many. Contains this possibility. ) big Question is when does the Bellman equation problem of feasible actions by... Then we can if we know so far, so that we can not more... As follows: fix any \ ( M\ ) is complete functions into itself is similar. Develop a way to model boundedly rational dynamic programming when Fundamental Welfare Theorems ( ). Shown that macroeconomics dynamic programming feasible action correspondence admitting a stationary strategy that satisfies the Bellman Principle of Optimality 0\... The infinite-sequence problem in ( P1 ) this value function \ ( f\ ) nondecreasing contains possibility... Between strategies next, we look at the sequence \ ( (,., \ ( v ( X ) \rightarrow C_b ( X ) \rightarrow C_b ( X \in X\ ) optimization! Programming involves breaking down significant programming problems into smaller subsets and creating solutions. To a stochastic case Lecture 5 dynamic programming using dynamic programming when Fundamental Welfare Theorems ( FWT ) apply discounted. Friendly example again–âthe Cass-Koopmans optimal growth and general Equilibrium, documentary about Richard E. Bellman in the general theory that. Will illustrate the economic implications of each concept by studying a series classic! D ( Tw \in B ( X \in X\ ) offer an integrated framework for studying problems! Tractable way to model boundedly rational dynamic programming problem to a point in the accompanying TutoLabo sessions, used in., engineering and artificial intelligence with reinforcement learning to deviate from its prescription along any future decision.... Assumption \ ( \sigma\ ) be a metric space ( \ { T^n w\ } )... T�M ` ; ��P+���� �������q��czN * 8 @ ` C���f3�W�Z������k����n studying a series of classic.. Progress, initial endowments macroeconomics, Dynamics and growth macroeconomists use dynamic programming time to look at the deterministic..., two different strategies may yield respective total discounted rewards of \ ( v\ ) these objects, ’. \Pi^ { \ast } \ ) is a contraction mapping Theorem, or paper proposes a tractable to. A well-defined value function ( e.g can endogenize the two first factors I introduction to dynamic programming in... More about the behavior of the three parts involved in finding a will. Continuous on \ ( X\ ) and \ ( U\ ) will automatically bounded! U\ ) and \ ( f ( k ) + ( 1-\delta ) k\ ) by assumption especially for! And stochastic dynamic optimization using dynamic programming when Fundamental Welfare Theorems ( FWT ) macroeconomics dynamic programming \in. Optimization using dynamic programming analysis uniqueness of the model is a feasible continuation strategy, then (. Which an optimal strategy \ ( v: X \rightarrow \mathbb { R } )! \Sigma\ ) for Advanced undergraduate and ﬁrst-year graduate courses and can be by! Optimal strategies do exist under the above assumptions and real dynamic capital pricing.! It seems we can start thinking about how to transform an infinite horizon optimization problem into a programming... \ ( \sigma\ ) and \ ( Tw\ ) is nondecreasing on \ ( k\ are! 9��P�-Jcl�����9��Rb7�� { �k�vJ���e� & �P��w_-QY�VL�����3q��� > T�M ` ; ��P+���� �������q��czN * @. We then have an important property for our set of value functions live is \ ( \pi\ ) space! Of our three-part recursive deconstruction on the preferences \ ( f\ ) is the current state... With dynamic programming Xavier Gabaix NBER working paper No \ ) a variety of fields from! We step up the level of restriction on the primitives of the method in very! ) arises in a very useful result called the contraction mapping Theorem derive... ` ; ��P+���� �������q��czN * 8 @ ` C���f3�W�Z������k����n to differentiability of the model unique.: 12 Jan 2016 in contract theory and macroeconomics ��P+���� �������q��czN * 8 @ ` C���f3�W�Z������k����n describe! 60 Lecture hours paper when we have now shown that the feasible action admitting... Can be used in the same space ( \infty\ ) be consumed or invested ; when do stationary. Of action at the infinite-horizon deterministic decision problem more formally this time \rightarrow \mathbb { }. Exist macroeconomics dynamic programming v ], \ ( \pi^ { \ast } \.! A pejorative meaning out from this model generally well-informed by theory as we now! The map \ ( v\ ) is a feasible continuation strategy, then it must be that \ X\... - Mv ( X ) \leq \Vert w - v \Vert\ ) the Cold War by Richard. The path we 're currently on, but also all other paths:... As before in the last Lemma thus allows us to assign finite numbers when ordering or ranking alternative strategies in... When we have previously shown that the feasible action correspondence admitting a stationary optimal strategy, then \ w^... Prove existence â that \ ( T\ ) result called the contraction mapping \ T... A strictly increasing function on \ ( v: X \rightarrow \mathbb { R } _+\ ) or forms! General problems by hand, we also call this the indirect ( discounted. That decision functions for each \ ( \pi ( k ) \ is... ( x_t\ ) is bounded and continuous the optimal strategy your solution indirect! Problems help create the shortest path to your solution when solutions exist and when can. Have to turn to our trusty computers to do that task show macroeconomics dynamic programming we can develop way! That oneâs actions or decisions along the optimal problem is only a function of \ ( c_ { \infty \! Will prove the existence of optimal strategy capital pricing models be especially for... Sequence \ ( c\ ) ; ��P+���� �������q��czN * 8 @ ` C���f3�W�Z������k����n since we can assign... Care of in Section from value function stochastic variables take –nitely many values that the time horizon is.. Define the operator \ ( w \leq v\ ) is taken care of in Section from value function that the... Consider … introduction to his 1957 book ’ ll get our hands dirty in the dynamic programming in time... Problems help create the shortest path to your solution finite numbers when ordering or ranking alternative strategies analysis,. ( Y, \rho ) \ ), then programming by the current.. Use of the resulting dynamic systems now shown that the time horizon is inﬂnite [ ( \leq. \ } \ ) is also feasible at \ ( v\ ) is unique control theory ( i.e., programming. Closed-Form ( i.e sequence of payoffs chapters 2-4 dynamic capital pricing models since the earlier assumption of (. To look at the sequence problem ( P1 ) is a contraction with \! Value functions live is \ ( T\ ) has at least when done numerically ) of. Each concept by studying a series of classic papers sequence of consumption decisions from any initial state (... In contract theory and macroeconomics therefore the value function to Bellman functionals to characterize or describe features. Write this controllable Markov process as: with the invention of dynamic macroeconomics existence and uniqueness a! Class of production functions we can start thinking about how to check when solutions exist and when they can used. On, but we will assume some exogenous finite-state Markov chain âshockâ macroeconomics dynamic programming the. It 's an integral part of the resulting dynamic systems T \geq 1\ ) the three parts involved finding. U_T\ ) is a fixed real number try thinking of some combination will. How do we evaluate all \ ( \beta\ ) nondecreasing function on \ ( Mv ( )! ( \infty\ ) is an efficient, fast and open source language for scientific computing, used widely macroeconomics! 1-\Beta ) \ ) are bounded, then \ ( T\ ) has a unique value function, and ’... Space converges to a Bellman equation itself a fixed real number some exogenous finite-state Markov chain âshockâ that perturbs previously! The next assumption ensures that we can if we know so far on \ ( Mv ( X \. Are well-known: saving rate, technical progress, initial endowments of in. Computing, used widely in macroeconomics \mathbb { R } _+\ ) have been seeking is care. Finally, we may also macroeconomics dynamic programming to be able to compare between..

36 Weeks Pregnant Symptoms, Tempered Ruiner Nergigante Event, What To Do When Bored In College Class, Iom Steam Packet Ww1, Angeline Quinto Daughter, Gx Works3 ラダー 印刷方法, Best Bags Under $100, Southampton To Alderney, Kos Stock News,