Prof. Herman J. Bierens
Department of Economics
The Pennsylvania State University
University Park, PA 16802, USA
Email: hbierens@psu.edu
The objective of this mini-course is to show the students the various ways unknown regression functions and density functions can be estimated without using parametric specifications.
I will first review the basics of kernel density and regression estimation, including point-wise and uniform convergence, asymptotic normality, bias elimination and convergence rate boosting.
An alternative approach to kernel estimation is to approximate unknown functions via series expansions. This is based on Hilbert space theory, where function spaces are endowed with similar properties as Euclidean spaces. In Euclidean spaces vectors can be represented as linear combinations of orthogonal basis vectors. In Hilbert function spaces we have a similar property; any function in the Hilbert space can be represented by a linear combination of an infinite sequence of orthogonal functions. Therefore, I will briefly review Hilbert space theory and show how these orthogonal functions can be chosen. As to the latter, I will focus on orthogonal polynomials and how to generate them recursively. Moreover, I will show how density and distribution function can be constructed on the basis of these orthogonal polynomials. Furthermore, I will show by a few examples how to use this approach to estimate econometric models semi-nonparametrically.
Neural net approximations are related to approximations via orthogonal polynomials, except that the orthogonality is not used; only linear independence. I will explain why neural nets can approximate any function.