In this guided tour I will explain in detail how to conduct nonlinear least squares estimation, using the example of a homogenous CES production function. However, before you continue, please read NLLS.PDF first to learn about the CES production function and how to estimate its parameters via EasyReg.
The homogenous CES production function takes the form
where Q is output, K is capital, L is labor, and U is an error term satisfying
The data I will use has been artifically generated: 500 replications of
With Y = l
EasyReg builds up a nonlinear regression model recursively, starting from the X variables X(1) and X(2), by creating new X variables using linear and/or multiplicative combinations and linear and nonlinear transformations of previously selected or created X variables. See NLTRANS.PDF for a list of all available transformations.
As explained in NLLS.PDF, the transformations for building the CES production function involved are
Then the nonlinear regression function g(x,b) is equal to X(13).
Admittedly, this is a primitive programming language, but because it is so primitive its execution is fast.
The procedure for importing data and selecting the dependent and independent variables is the same as in the OLS module. I assume that you have run an OLS regression before, so that you already know how to do that.
The first relevant window page is:
In order to make X(3), double click X(2) = 1, and click the "Selection OK" button. Then the window changes to:
Since this is not the final model, check the "Don't bother me anymore!" box, and click "Clear".
I recommend to add a comment to the transformation before you continue. Then the window changes to:
Click "Done" and then click the "Transformation OK" button.
The new X variable X(3) = b(1)X(2) = b(1) {= ln(gamma)} has now been made.
To make X(4) = b(2)X(2) = b(2), repleat this procedure: Double click X(2) = 1 again, click the "Selection OK" button, select "Linear combination", add the comment "rho", and click the "Transformation OK" button. The window changes to:
To make X(5) = b(3)X(2) = b(3), repleat this procedure: Double click X(2) = 1 again, click the "Selection OK" button, select "Linear combination", add the comment "alpha", and click the "Transformation OK" button. The window changes to:
To make X(6), double click X(1) and X(4)
and click the "Selection OK" button. The window changes to:
Double click "Multiply" and click the "Transformation OK" button. Then X(6) = X(1)X(4) is created, and the window changes back to:
To make X(7) = EXP(X(6)), double click X(6) and click "Selection OK":
Double click "EXP(z)" and click "Transformation OK". Then X(7) = EXP(X(6)) is created:
It is now pretty straightforward to complete the build up of the CES production function by creating the remaining X variables X(8) through X(13) as specified above:
We are now done. Click the "Select the final model" button. Then the next window appears:
If you want to use the same specification once more with this or another data set, it is recommended that you store the model in a template, by clicking the "Store" button. Then the model will be stored as template file TEMPLATE_NLLS.002 in the current sub-folder EASYREG.DAT. (The existing template TEMPLATE_NLLS.001 corresponds to the previous version of this guided tour)
If you run this nonlinear regression again, you will jump to the first window, and the last template file which is compatible with the initial X variables will be loaded.
If you click "Store", the window changes to:
Click "Model is OK". Then the following window appears:
Recall that b(2) should be greater or equal to -1, b(3) should be contained in the unit interval [0,1], and b(1) is unrestricted. However, asymptotic theory of nonlinear regression requires that the parameter space is closed and bounded. Therefore, I will confine b(1) to the interval [-100,100], b(2) to the interval [-1,100], and b(3) to the interval [0,1]. After entering the lower and upper bounds involved, the window changes to:
Click "Bounds OK". Then the following window appears:
Click "Start", which starts the Nelder and Mead simplex iteration. It is strongly recommended to restart the iteration from different random start values, as well as from the last iteration result, in order to check whether you have reached the global minimum of the objective function (which is the sum of squared residuals).
Once you are confident that the global minimum has been reached, click "Done". Then the window changes to:
Finally, click "Continue". Then the estimation results appear.
Recall that the true parameter values are b(1) = 0, b(2) = 0, and b(3) = 0.25. Except for b(2) the NLLS estimators are close to these values, but b(2) is not significantly different from zero.
In order to test the joint hypotheses b(1)= b(2) = 0 and b(1)= b(2) = 0, b(3) = 0.25, click the "Options" button, which opens the "Options" menu:
and click menu item "Wald test of linear parameter restrictions".
In order to test the joint hypothesis b(1)= b(2) = 0, double click b(1) and b(2), and click "Test joint significance".
As expected, the joint hypothesis b(1)= b(2) = 0 is not rejected.
In order to test the joint hypothesis b(1)= b(2) = 0, b(3) = 0.25, click "More tests", double click b(1), b(2) and b(3), and click "Test linear restrictions".
The null hypothesis involved takes the form of three linear equations:
Again as expected, the null hypothesis involved is not rejected.
The "Back" button brings you back to the "What to do next?" window:
Note that the Wald test results have been appended to the output. When you click menu item "Done" while leaving the check box "Write output to EASYREG.DAT\OUTPUT.TXT when done" checked, you will return to the EasyReg main window, and the output will be appended to file OUTPUT.TXT in the current sub-folder EASYREG.DAT.
The menu item "Write residuals to the input file" is useful if you want to analyse the NLLS residuals further.
If you want to conduct the ICM test of the correctness of the functional form of the model, you have to read and understand the key papers involved first:
To conclude this guide tour, let us see what happens if you click menu item "Compute and plot the kernel estimate of the error density".
Recall that the error terms U have been drawn from the standard normal distribution. The picture confirms this.
Dependent variable:
Y = ln(Q/L)
Characteristics:
ln(Q/L)
First observation = 1
Last observation = 500
Number of usable observations: 500
Minimum value: -3.4043000E+000
Maximum value: 3.4399900E+000
Sample mean: 8.1536220E-002
X variables:
X(1) = ln(L/K)
X(2) = 1
NLLS model:
Model: y = g[x,b] + u, where
X(1)=ln(L/K)
X(2)=1
X(3)=b(1) {= ln(gamma)}
X(4)=b(2) {= rho}
X(5)=b(3) {= alpha}
X(6)=X(1).X(4)
X(7)=EXP[X(6)]
X(8)=X(7)-X(2)
X(9)=X(5).X(8)
X(10)=LOG[X(9)+1]/X(9)
X(11)=(EXP[X(6)]-1)/X(6)
X(12)=X(1).X(5).X(10).X(11)
X(13)=X(3)-X(12)
g[x,b] = X(13), where x = (X(1),..,X(13))' and b = (b(1),..,b(3))'
-100 <= b(1) <= 100
-1 <= b(2) <= 100
0 <= b(3) <= 1
The objective function (RSS) has been minimized using the simplex method
of Nelder and Mead. The algorithm involved is a Visual Basic translation
of the Fortran algorithm involved in:
W.H.Press, B.P.Flannery, S.A.Teukolsky and W.T.Vetterling, 'Numerical
Recipes', Cambridge University Press, 1986, pp. 292-293
Estimation results:
Parameters Estimate t-value H.C. t-value(*)
[p-value] [H.C. p-value]
b(1) -0.015548 -0.291 -0.287
[0.77099] [0.77418]
b(2) -0.236400 -1.440 -1.431
[0.14991] [0.15248]
b(3) 0.265879 7.744 8.134
[0.00000] [0.00000]
(*) Based on White's heteroskedasticity consistent variance matrix.
[The two-sided p-values are based on the normal approximation]
RSS: 490.212216
s.e. 0.993148
R-square: 0.1303
n: 500
Wald test:
b(1) -0.015548 -0.291 -0.287(*)
b(2) -0.236400 -1.440 -1.431(*)
b(3) 0.265879 7.744 8.134
(*): Parameters to be tested
Null hypothesis:
b(1) = b(2) = 0
Wald test: 2.39
Asymptotic null distribution: Chi-square(2)
p-value = 0.30228
Significance levels: 10% 5%
Critical values: 4.61 5.99
Conclusions: accept accept
Test result on the basis of the heteroskedasticity consistent variance
matrix:
Wald test: 2.45
Asymptotic null distribution: Chi-square(2)
p-value = 0.29400
Significance levels: 10% 5%
Critical values: 4.61 5.99
Conclusions: accept accept
Wald test:
b(1) -0.015548 -0.291 -0.287(*)
b(2) -0.236400 -1.440 -1.431(*)
b(3) 0.265879 7.744 8.134(*)
(*): Parameters to be tested
Null hypothesis:
1.b(1)+0.b(2)+0.b(3) = 0.
0.b(1)+1.b(2)+0.b(3) = 0.
0.b(1)+0.b(2)+1.b(3) = 0.25
Null hypothesis in matrix form: Rb = c, where
R =
1. 0. 0.
0. 1. 0.
0. 0. 1.
and c =
0.
0.
0.25
Wald test on the basis of the standard variance matrix:
Wald test statistic: 3.94
Asymptotic null distribution: Chi-square(3)
p-value = 0.26828
Significance levels: 10% 5%
Critical values: 6.25 7.81
Conclusions: accept accept
Wald test on the basis of White's heteroskedasticity consistent
variance matrix:
Wald test statistic: 3.65
Asymptotic null distribution: Chi-square(3)
p-value = 0.30232
Significance levels: 10% 5%
Critical values: 6.25 7.81
Conclusions: accept accept