Time-Inconsistent Control Theory with Finance Applications

Description

This book is devoted to problems of stochastic control and stopping that are time inconsistent in the sense that they do not admit a Bellman optimality principle. These problems are cast in a game-theoretic framework, with the focus on subgame-perfect Nash equilibrium strategies. The general theory is illustrated with a number of finance applications. In dynamic choice problems, time inconsistency is the rule rather than the exception. Indeed, as Robert H. Strotz pointed out in his seminal 1955 paper, relaxing the widely used ad hoc assumption of exponential discounting gives rise to time inconsistency. Other famous examples of time inconsistency include mean-variance portfolio choice and prospect theory in a dynamic context. For such models, the very concept of optimality becomes problematic, as the decision maker’s preferences change over time in a temporally inconsistent way. In this book, a time-inconsistent problem is viewed as a non-cooperative game between the agent’s current and future selves, with the objective of finding intrapersonal equilibria in the game-theoretic sense. A range of finance applications are provided, including problems with non-exponential discounting, mean-variance objective, time-inconsistent linear quadratic regulator, probability distortion, and market equilibrium with time-inconsistent preferences. Time-Inconsistent Control Theory with Finance Applications offers the first comprehensive treatment of time-inconsistent control and stopping problems, in both continuous and discrete time, and in the context of finance applications. Intended for researchers and graduate students in the fields of finance and economics, it includes a review of the standard time-consistent results, bibliographical notes, as well as detailed examples showcasing time inconsistency problems. For the reader unacquainted with standard arbitrage theory, an appendix provides a toolbox of material needed for the book. Preface Dedication Acknowledgements Contents 1 Introduction 1.1 A Standard Control Problem 1.2 Dynamic Programming and the Concept of Time Consistency 1.3 Some Disturbing Examples 1.4 Approaches to Handling Time Inconsistency 1.5 Stopping Problems and Time Inconsistency 1.6 The Time-Inconsistent Framework 1.7 Notes on the Literature Part I Optimal Control in Discrete Time 2 Dynamic Programming Theory 2.1 Setup 2.2 Embedding the Problem 2.3 Time Consistency and the Bellman Principle 2.4 The Bellman Equation 2.5 On State Variables 2.6 Handling the Bellman Equation 2.7 Notes on the Literature 3 The Linear Quadratic Regulator 3.1 Problem Formulation 3.2 Solution 3.3 Notes on the Literature 4 A Simple Equilibrium Model 4.1 Setup 4.2 The Agent's Problem 4.3 Optimality 4.4 Some Results from Arbitrage Theory 4.5 Identifying the Stochastic Discount Factor 4.6 Identifying the Martingale Measure 4.7 Market Equilibrium 4.8 Notes on the Literature Part II Time-Inconsistent Control in Discrete Time 5 Time-Inconsistent Control Theory 5.1 Time Consistency 5.2 Basic Problem Formulation 5.3 The Game-Theoretic Formulation 5.4 Dependence on the Initial State 5.4.1 A Useful Function Sequence 5.4.2 The Recursion for J 5.4.3 The Extended Bellman System for the State-Dependent Case 5.4.4 Variations of the Bellman System for the State-Dependent Case 5.5 Nonlinear Function of the Expected Value 5.5.1 Another Useful Function Sequence 5.5.2 The Extended Bellman System for the Nonlinear Case 5.5.3 Variations of the Bellman System for the Nonlinear Case 5.6 Putting the Results Together 5.6.1 Variations of the Bellman System 5.7 The General Case 5.7.1 Variations of the Bellman System for the General Case 6 Extensions and Further Results 6.1 A More General Nonlinear Term 6.2 Infinite Horizon 6.2.1 The General Case 6.2.2 A Time-Invariant Problem 6.3 Kihlstrom–Mirman Preferences 6.3.1 The Simplest Model 6.3.2 Dependence on Present Time 6.4 Existence and Uniqueness 6.5 A Scaling Result 6.6 An Equivalent Time-Consistent Problem 7 Non-exponential Discounting 7.1 General Non-exponential Discounting 7.1.1 A General Discount Function 7.1.2 Infinite Horizon 7.2 Quasi-Hyperbolic Discounting 7.2.1 The Extended Bellman Equation 7.2.2 An Example with Logarithmic Utility 7.2.3 Two Equivalent Standard Problems 7.3 Generalized Euler Equation 7.3.1 A Variational Equation 7.3.2 The Harris and Laibson Model 7.4 Notes on the Literature 8 Mean-Variance Portfolios 8.1 Portfolios with Constant Risk Aversion 8.2 Portfolios with State-Dependent Risk Aversion 8.3 Notes on the Literature 9 Time-Inconsistent Regulator Problems 9.1 A Quadratic Expectation Term 9.2 A State-Dependent Quadratic Term 9.3 Notes on the Literature 10 A Time-Inconsistent Equilibrium Model 10.1 Setup 10.2 The Problem of the Agent 10.3 Equilibrium Definitions 10.4 Intrapersonal Equilibrium 10.4.1 First-Order Conditions 10.4.2 Identifying the Stochastic Discount Factor 10.5 Market Equilibrium Part III Optimal Control in Continuous Time 11 Dynamic Programming Theory 11.1 Setup 11.2 The Infinitesimal Operator 11.3 Embedding the Problem 11.4 Time Consistency and the Bellman Principle 11.5 The Hamilton–Jacobi–Bellman Equation 11.6 Verification Theorem 11.7 A Generalized HJB Equation 11.8 Handling the HJB Equation 11.9 Notes on the Literature 12 The Continuous-Time Linear Quadratic Regulator 12.1 Problem Formulation 12.2 Solution 12.3 Notes on the Literature 13 Optimal Consumption and Investment 13.1 Setup 13.2 The Problem of the Agent 13.3 Optimality 13.4 Notes on the Literature 14 A Simple Equilibrium Model 14.1 Setup 14.2 Market Equilibrium 14.3 Notes on the Literature Part IV Time-Inconsistent Control in Continuous Time 15 Time-Inconsistent Control Theory 15.1 The Model 15.2 Problem Formulation 15.3 An Informal Derivation of the Extended HJB Equation 15.3.1 Deriving the Equation 15.3.2 Existence and Uniqueness 15.4 A Verification Theorem 15.5 The General Case 15.6 Notes on the Literature 16 Special Cases and Extensions 16.1 The Case when G=0 16.2 The Case with No State Dependence 16.3 Generalizing H and G 16.4 A Driving Point Process 16.5 Infinite Horizon 16.6 A Scaling Result 16.7 An Equivalent Time-Consistent Problem 17 Non-exponential Discounting 17.1 The General Case 17.2 Infinite Horizon 17.3 Optimal Investment and Consumption for Log Utility 17.4 Notes on the Literature 18 Mean-Variance Control 18.1 The Simplest Case 18.2 A Point Process Extension 18.3 Mean-Variance with Wealth-Dependent Risk Aversion 18.3.1 The HJB System for a General γ(x) 18.3.2 A Special Choice of γ(x) 18.4 Notes on the Literature 19 The Inconsistent Linear Quadratic Regulator 19.1 Problem Formulation 19.2 Solution 19.3 Notes on the Literature 20 A Time-Inconsistent Equilibrium Model 20.1 The Model 20.2 Equilibrium Definitions 20.2.1 Intrapersonal Equilibrium 20.2.2 Market Equilibrium 20.3 Main Goals of the Chapter 20.4 The Extended HJB Equation 20.5 Determining Market Equilibrium 20.6 Summary of Standard Results 20.7 The Stochastic Discount Factor 20.7.1 A Representation Formula for M 20.7.2 Interpreting the Representation Formula 20.8 Equilibrium with Non-exponential Discounting 20.8.1 Generalities 20.8.2 Logarithmic Utility Function 20.8.3 Power Utility Function Part V Optimal Stopping Theory 21 Optimal Stopping in Discrete Time 21.1 The General Case 21.1.1 Setup 21.1.2 Embedding the Problem 21.1.3 The Optimal Value Process and the Optimal Strategy 21.1.4 The Snell Envelope 21.2 Special Cases 21.2.1 Markovian Setting 21.2.2 Exponential Discounting 21.2.3 Infinite Horizon 21.3 Example: A Simple Secretary Problem 21.3.1 Finite-Horizon Case 21.3.2 Infinite-Horizon Case 21.4 Notes on the Literature 22 Optimal Stopping in Continuous Time 22.1 The General Case 22.1.1 Setup 22.1.2 The Snell Envelope Theorem 22.2 Specializing to a Diffusion Setting 22.2.1 Setup 22.2.2 A Dynamic Programming Argument 22.2.3 Variational Inequalities 22.2.4 Connections to the General Case 22.3 Example: Optimal Time to Sell an Asset 22.4 Notes on the Literature Part VI Time-Inconsistent Stopping Problems 23 Time-Inconsistent Stopping in Discrete Time 23.1 Problem Formulation 23.2 Useful Function Sequences 23.3 The Recursion for J 23.4 The Recursion for V 23.5 The General Case 23.6 Variations of the Bellman System 23.7 Non-exponential Discounting 23.7.1 Finite Horizon 23.7.2 Infinite Horizon 23.7.3 Quasi-Hyperbolic Discounting 23.8 Examples 23.8.1 Time-Inconsistent Secretary Problem 23.8.2 Costly Procrastination 23.9 Notes on the Literature 24 Time-Inconsistent Stopping in Continuous Time 24.1 Problem Formulation 24.2 Time-Inconsistent Variational Inequalities 24.2.1 Heuristic Derivation 24.2.2 A Verification Argument 24.2.3 The General Case 24.3 Special Cases 24.3.1 The Case when G=0 24.3.2 Non-exponential Discounting 24.3.3 The Case with No State Dependence 24.4 Examples 24.4.1 Selling an Asset with Non-exponential Discounting 24.4.2 Selling an Asset with Mean-Variance Preferences 24.5 Notes on the Literature 25 Time-Inconsistent Stopping Under Distorted Probabilities 25.1 Discrete Time 25.1.1 Setup 25.1.2 Defining the Game 25.1.3 The Basic Recursion 25.1.4 The Recursion for G 25.1.5 Putting the Results Together 25.1.6 The Algorithm for a Finite Horizon 25.1.7 A Verification Theorem 25.1.8 Connections to the Snell Envelope 25.2 Continuous Time 25.2.1 Setup 25.2.2 Defining the Game 25.2.3 Time-Inconsistent Variational Inequalities 25.3 Notes on the Literature A Basic Arbitrage Theory A.1 Portfolios A.2 Arbitrage A.3 Girsanov and the Market Price of Risk A.4 Martingale Pricing A.5 Hedging A.6 Stochastic Discount Factors A.7 Dividends A.8 Consumption A.9 Replicating a Consumption Process References Index