Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators

Preface xi

1 Introduction 1

1.1 Multivariate analysis in a nutshell 2

1.2 The path that lies ahead 13

2 Vector and function spaces 15

2.1 Metric spaces 16

2.2 Vector and normed spaces 20

2.3 Banach and Lp spaces 26

2.4 Inner Product and Hilbert spaces 31

2.5 The projection theorem and orthogonal decomposition 38

2.6 Vector integrals 40

2.7 Reproducing kernel Hilbert spaces 46

2.8 Sobolev spaces 55

3 Linear operator and functionals 61

3.1 Operators 62

3.2 Linear functionals 66

3.3 Adjoint operator 71

3.4 Nonnegative, square-root, and projection operators 74

3.5 Operator inverses 77

3.6 Fréchet and Gâteaux derivatives 83

3.7 Generalized Gram–Schmidt decompositions 87

4 Compact operators and singular value decomposition 91

4.1 Compact operators 92

4.2 Eigenvalues of compact operators 96

4.3 The singular value decomposition 103

4.4 Hilbert–Schmidt operators 107

4.5 Trace class operators 113

4.6 Integral operators and Mercer’s Theorem 116

4.7 Operators on an RKHS 123

4.8 Simultaneous diagonalization of two nonnegative definite operators 126

5 Perturbation theory 129

5.1 Perturbation of self-adjoint compact operators 129

5.2 Perturbation of general compact operators 140

6 Smoothing and regularization 147

6.1 Functional linear model 147

6.2 Penalized least squares estimators 150

6.3 Bias and variance 157

6.4 A computational formula 158

6.5 Regularization parameter selection 161

6.6 Splines 165

7 Random elements in a Hilbert space 175

7.1 Probability measures on a Hilbert space 176

7.2 Mean and covariance of a random element of a Hilbert space 178

7.3 Mean-square continuous processes and the Karhunen–Lòeve Theorem 184

7.4 Mean-square continuous processes in L2 (E,B(E), mu) 190

7.5 RKHS valued processes 195

7.6 The closed span of a process 198

7.7 Large sample theory 203

8 Mean and covariance estimation 211

8.1 Sample mean and covariance operator 212

8.2 Local linear estimation 214

8.3 Penalized least-squares estimation 231

9 Principal components analysis 251

9.1 Estimation via the sample covariance operator 253

9.2 Estimation via local linear smoothing 255

9.3 Estimation via penalized least squares 261

10 Canonical correlation analysis 265

10.1 CCA for random elements of a Hilbert space 267

10.2 Estimation 274

10.3 Prediction and regression 281

10.4 Factor analysis 284

10.5 MANOVA and discriminant analysis 288

10.6 Orthogonal subspaces and partial cca 294

11 Regression 305

11.1 A functional regression model 305

11.2 Asymptotic theory 308

11.3 Minimax optimality 318

11.4 Discretely sampled data 321

References 327

Index 331

Notation Index 334