Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory

Description

For practicing engineers and scientists who design and analyze signal processing systems, i.e., to extract information from noisy signals ― radar engineer, sonar engineer, geophysicist, oceanographer, biomedical engineer, communications engineer, economist, statistician, physicist, etc. A unified presentation of parameter estimation for those involved in the design and implementation of statistical signal processing algorithms. Preface 1 Introduction 1.1 Estimation in Signal Processing 1.2 The Mathematical Estimation Problem 1.3 Assessing Estimator Performance 1.4 Some Notes to the Reader References Problems 2 Minimum Variance Unbiased Estimation 2.1 Introduction 2.2 Summary 2.3 Unbiased Estimators 2.4 Minimum Variance Criterion 2.5 Existence of the Minimum Variance Unbiased Estimator 2.6 Finding the Minimum Variance Unbiased Estimator 2.7 Extension to a Vector Parameter References Problems 3 Cramer-Rao Lower Bound 3.1 Introduction 3.2 Summary 3.3 Estimator Accuracy Considerations 3.4 Cramer-Rao Lower Bound 3.5 General CRLB for Signals in White Gaussian Noise 3.6 Transformation of Parameters 3.7 Extension to a Vector Parameter 3.8 Vector Parameter CRLB for Transformations 3.9 CRLB for the General Gaussian Case 3.10 Asymptotic CRLB for WSS Gaussian Random Processes 3.11 Signal Processing Examples References Problems 3A Derivation of Scalar Parameter CRLB 3B Derivation of Vector Parameter CRLB 3C Derivation of General Gaussian CRLB 3D Derivation of Asymptotic CRLB 4 Linear Models 4.1 Introduction 4.2 Summary 4.3 Definition and Properties 4.4 Linear Model Examples 4.5 Extension to the Linear Model References Problems 5 General Minimum Variance Unbiased Estimation 5.1 Introduction 5.2 Summary 5.3 Sufficient Statistics 5.4 Finding Sufficient Statistics 5.5 Using Sufficiency to Find the MVU Estimator 5.6 Extension to a Vector Parameter References Problems 5A Proof of Neyman-Fisher Factorization Theorem (Scalar Parameter) 5B Proof of Rao-Blackwell-Lehmann-Scheffe Theorem (Scalar Parameter) 6 Best Linear Unbiased Estimators 6.1 Introduction 6.2 Summary 6.3 Definition of the BLUE 6.4 Finding the BLUE 6.5 Extension to a Vector Parameter 6.6 Signal Processing Example References Problems 6A Derivation of Scalar BLUE 6B Derivation of Vector BLUE 7 Maximum Likelihood Estimation 7.1 Introduction 7.2 Summary 7.3 An Example 7.4 Finding the MLE 7.5 Properties of the MLE 7.6 MLE for Transformed Parameters 7.7 Numerical Determination of the MLE 7.8 Extension to a Vector Parameter 7.9 Asymptotic MLE 7.10 Signal Processing Examples References Problems 7A Monte Carlo Methods 7B Asymptotic PDF of MLE for a Scalar Parameter 7C Derivation of Conditional Log-Likelihood for EM Algorithm Example 8 Least Squares 8.1 Introduction 8.2 Summary 8.3 The Least Squares Approach 8.4 Linear Least Squares 8.5 Geometrical Interpretations 8.6 Order-Recursive Least Squares 8.7 Sequential Least Squares 8.8 Constrained Least Squares 8.9 Nonlinear Least Squares 8.10 Signal Processing Examples References Problems 8A Derivation of Order-Recursive Least Squares 8B Derivation of Recursive Projection Matrix 8C Derivation of Sequential Least Squares 9 Method of Moments 9.1 Introduction 9.2 Summary 9.3 Method of Moments 9.4 Extension to a Vector Parameter 9.5 Statistical Evaluation of Estimators 9.6 Signal Processing Example References Problems 10 The Bayesian Philosophy 10.1 Introduction 10.2 Summary 10.3 Prior Knowledge and Estimation 10.4 Choosing a Prior PDF 10.5 Properties of the Gaussian PDF 10.6 Bayesian Linear Model 10.7 Nuisance Parameters 10.8 Bayesian Estimation for Deterministic Parameters References Problems 10A Derivation of Conditional Gaussian PDF 11 General Bayesian Estimators 11.1 Introduction 11.2 Summary 11.3 Risk Functions 11.4 Minimum Mean Square Error Estimators 11.5 Maximum A Posteriori Estimators 11.6 Performance Description 11.7 Signal Processing Example References Problems 11A Conversion of Continuous-Time System to DIscrete-TIme System 12 Linear Bayesian Estimators 12.1 Introduction 12.2 Summary 12.3 Linear MMSE Estimation 12.4 Geometrical Interpretations 12.5 The Vector LMMSE Estimator 12.6 Sequential LMMSE Estimation 12.7 Signal Processing Examples - Wiener Filtering References Problems 12A Derivation of Sequential LMMSE Estimator 13 Kalman Filters 13.1 Introduction 13.2 Summary 13.3 Dynamical Signal Models 13.4 Scalar Kalman Filter 13.5 Kalman Versus Wiener Filters 13.6 Vector Kalman Filter 13.7 Extended Kalman Filter 13.8 Signal Processing Examples References Problems 13A Vector Kalman Filter Derivation 13B Extended Kalman Filter Derivation 14 Summary of Estimators 14.1 Introduction 14.2 Estimation Approaches 14.3 Linear Model 14.4 Choosing an Estimator 15 Extensions for Complex Data and Parameters 15.1 Introduction 15.2 Summary 15.3 Complex Data and Parameters 15.4 Complex Random Variables and PDFs 15.5 Complex WSS Random Processes 15.6 Derivatives, Gradients, and Optimization 15.7 Classical Estimation with Complex Data 15.8 Bayesian Estimation 15.9 Asymptotic Complex Gaussian PDF 15.10 Signal Processing Examples References Problems 15A Derivation of Properties of Complex Covariance Matrices 15B Derivation of Properties of Complex Gaussian PDF 15C Derivation of CRLB and MLE Formulas A1 Review of Important Concepts A1.1 Linear and Matrix Algebra A1.2 Probability, Random Processes. and Time Series Models A2 Glossary of Symbols and Abbreviations Index