WEBVTT
Kind: captions
Language: en
00:00:00.930 --> 00:00:04.340
In a typical research paper that
uses multiple regression analysis,
00:00:04.573 --> 00:00:07.590
we do many different regression models.
00:00:07.590 --> 00:00:09.210
And the reason for that is that
00:00:09.210 --> 00:00:10.830
we want to do model comparisons.
00:00:10.830 --> 00:00:12.390
Now we will take a look at,
00:00:12.390 --> 00:00:14.610
why we compare models and how we do that?
00:00:15.427 --> 00:00:16.680
In Hekman's paper,
00:00:16.680 --> 00:00:18.480
which is our example for this video,
00:00:18.480 --> 00:00:21.690
we will be focusing on their first study.
00:00:21.690 --> 00:00:26.207
So they say that they used hierarchical
moderated regression analysis.
00:00:26.207 --> 00:00:27.570
So what does that mean?
00:00:27.570 --> 00:00:29.760
The hierarchical here is the key term,
00:00:29.760 --> 00:00:33.120
it simply means that you're
estimating multiple models.
00:00:33.120 --> 00:00:36.300
Start with a simple one then add more variables,
00:00:36.300 --> 00:00:38.850
compare, add more variables and compare.
00:00:39.067 --> 00:00:42.067
The moderated part here means that
00:00:42.067 --> 00:00:43.890
they have interaction terms in their model.
00:00:43.890 --> 00:00:45.840
They could just as well have said that
00:00:45.840 --> 00:00:47.100
they used regression analysis,
00:00:47.100 --> 00:00:50.040
because we use regression analysis nearly always
00:00:50.040 --> 00:00:51.480
in a hierarchical way.
00:00:51.480 --> 00:00:54.990
And it's obvious based on
the regression results that
00:00:54.990 --> 00:00:56.670
they contain interaction terms.
00:00:56.670 --> 00:01:00.780
So this is a bit unnecessary,
complicated way of saying,
00:01:00.780 --> 00:01:03.660
we did regression, we estimated multiple models.
00:01:04.160 --> 00:01:06.180
Now let's look at the actual models,
00:01:06.180 --> 00:01:07.020
and the modeling results,
00:01:07.020 --> 00:01:09.420
and the logic for multiple model comparisons.
00:01:10.520 --> 00:01:14.030
They say that they entered in the first model,
00:01:14.030 --> 00:01:16.230
they have the control variables only here.
00:01:16.797 --> 00:01:21.750
And in the second model, they included
some of the interesting variables.
00:01:21.750 --> 00:01:24.240
So we'll be focusing on the first two models,
00:01:24.240 --> 00:01:25.530
Model 1 and Model 2.
00:01:25.747 --> 00:01:27.817
Model 1 has control variables only,
00:01:27.817 --> 00:01:31.440
Model 2 is controls and
some interesting variables.
00:01:31.773 --> 00:01:33.950
The logic in model comparison,
00:01:33.950 --> 00:01:35.610
when we do that kind of comparison,
00:01:35.610 --> 00:01:36.687
is to ask the question,
00:01:36.687 --> 00:01:40.130
do the interesting variables
and the controls together
00:01:40.130 --> 00:01:43.350
explain the dependent variable
more than the controls only.
00:01:43.350 --> 00:01:47.010
If the control variables and the
interesting variables together
00:01:47.010 --> 00:01:48.990
don't explain the data more
than the controls only,
00:01:48.990 --> 00:01:54.060
then we conclude that the interesting
variables are not very useful in
00:01:54.060 --> 00:01:55.740
explaining the dependent variable,
00:01:55.740 --> 00:01:58.410
and we can conclude that they
don't really have an effect.
00:01:59.060 --> 00:02:02.730
How we do a model comparison is that
we compare the R-squared statistic.
00:02:02.730 --> 00:02:06.740
So here they have the adjusted
R-squared and the actual R-squared.
00:02:07.240 --> 00:02:08.610
The model comparison,
00:02:08.610 --> 00:02:10.847
if we just want to assess the magnitude,
00:02:10.847 --> 00:02:13.770
how much better the Model 2 is?
00:02:13.770 --> 00:02:18.210
In small samples, the more appropriate
statistic is the adjusted R-squared.
00:02:18.210 --> 00:02:23.577
However, the adjusted R-squared statistic
doesn't really have a well-known test.
00:02:23.577 --> 00:02:26.880
So instead of looking at the adjusted R-squared,
00:02:26.880 --> 00:02:29.220
we test the R-squared difference.
00:02:29.570 --> 00:02:31.813
They present the R-squared difference here.
00:02:31.813 --> 00:02:35.730
So this is the difference between the first
model R-squared and the second model R-squared,
00:02:35.730 --> 00:02:38.610
and they have some stars here.
00:02:39.027 --> 00:02:41.007
So the important question is,
00:02:41.007 --> 00:02:44.610
does the second model explain the
data better than the first model?
00:02:44.610 --> 00:02:47.760
The adjusted R-squared difference is 4,
00:02:47.760 --> 00:02:53.580
the actual R-squared difference is 7 or 0.07 %,
00:02:53.580 --> 00:02:58.800
so the interesting variables explain the data
a bit more than the control variable only.
00:02:59.150 --> 00:03:03.800
Now we will be focusing on
these test statistics here.
00:03:03.800 --> 00:03:05.220
So where do these stars come from?
00:03:05.753 --> 00:03:08.850
These stars come from an F test that
00:03:08.850 --> 00:03:11.040
tests the null hypothesis that
00:03:11.040 --> 00:03:17.190
all the regression coefficients for
every variable added to this model are zero.
00:03:17.773 --> 00:03:20.310
We look at the logic of the test now.
00:03:21.127 --> 00:03:25.606
So the idea of the F test
between the first two models is
00:03:25.606 --> 00:03:28.410
that it is a nested model comparison test.
00:03:28.710 --> 00:03:31.290
So one model is nested in another,
00:03:31.290 --> 00:03:34.130
that means that one model is
a special case of another.
00:03:34.547 --> 00:03:39.780
So in this case Model 2 is the
unrestricted model or unconstraint model,
00:03:39.780 --> 00:03:44.040
Model 1 is the restricted
model or constraint model.
00:03:45.000 --> 00:03:50.070
So, why can we say that Model 1 is a special
case of a more general model, Model 2?
00:03:50.070 --> 00:03:52.360
The reason is that Model 1,
00:03:52.593 --> 00:03:55.353
which leaves out these variables,
00:03:55.353 --> 00:03:58.710
is the same model as Model 2,
00:03:58.710 --> 00:04:02.610
except that the effects of these
variables are constrained to be zero.
00:04:02.827 --> 00:04:05.047
So by leaving out variables,
00:04:05.047 --> 00:04:09.570
we constrain the regression coefficient
of that variable to be zero.
00:04:09.570 --> 00:04:10.800
And that's the reason,
00:04:10.800 --> 00:04:15.750
why we say that this model is a
constrained version of that model here.
00:04:15.750 --> 00:04:18.960
The effects of the last three
variables are freely estimated,
00:04:18.960 --> 00:04:21.360
here they are constrained to be 0.
00:04:22.277 --> 00:04:24.930
So how do we test these differences,
00:04:24.930 --> 00:04:30.840
whether the difference in R-squared is more
than what we can expect by a chance of only?
00:04:31.123 --> 00:04:34.123
Remember that every time we
add something to the model,
00:04:34.123 --> 00:04:35.803
the R-squared can only go up.
00:04:35.803 --> 00:04:37.660
It can stay the same or go up,
00:04:37.810 --> 00:04:39.400
typically it goes up.
00:04:39.400 --> 00:04:43.020
So is that increase in R-squared
statistically significant?
00:04:43.287 --> 00:04:44.520
To answer that question,
00:04:44.520 --> 00:04:45.870
we do the t test.
00:04:45.870 --> 00:04:48.320
And let's do a t test by hand now.
00:04:48.320 --> 00:04:52.627
We need to first have the degrees
of freedom for the first two models,
00:04:52.627 --> 00:04:54.810
to do the F test.
00:04:55.210 --> 00:04:57.760
The degrees of freedom for the regression model is
00:04:57.760 --> 00:05:02.377
n, the sample size, minus k, the
number of estimated parameters,
00:05:02.377 --> 00:05:06.453
or regression coefficients, or
number of variables in the model,
00:05:06.453 --> 00:05:08.360
minus 1 for the intercept.
00:05:08.360 --> 00:05:13.410
So we have a sample that provides
us with 113 units of information,
00:05:13.410 --> 00:05:15.870
we estimate for the first model,
00:05:15.870 --> 00:05:17.910
effects of 15 variables,
00:05:17.910 --> 00:05:19.320
we estimate the intercept,
00:05:19.320 --> 00:05:25.560
so we have 97 degrees of freedom
remaining for the restricted model.
00:05:26.210 --> 00:05:30.730
In the unrestricted model, we
estimate three more things,
00:05:30.730 --> 00:05:36.060
so it's 113 - 18 - 1 = 94 degrees of freedom,
00:05:36.060 --> 00:05:36.887
for that model.
00:05:37.037 --> 00:05:39.570
So these degrees of freedom
calculations are pretty simple,
00:05:39.570 --> 00:05:41.730
it's just basic subtraction.
00:05:42.730 --> 00:05:45.360
Now we need to have an F statistic as well.
00:05:45.360 --> 00:05:51.370
And the F statistic can be are
defined based on the R-squared values.
00:05:51.370 --> 00:05:56.760
So it's the R-squared difference divided
by the degrees of freedom difference,
00:05:56.760 --> 00:05:59.280
divided by that thing there.
00:05:59.930 --> 00:06:01.730
So that's the F statistic,
00:06:01.730 --> 00:06:04.923
your econometrics textbook will explain,
00:06:05.000 --> 00:06:06.770
where that comes from.
00:06:06.770 --> 00:06:10.000
But importantly we are here interested in,
00:06:10.000 --> 00:06:15.060
how much the R-squared increases
per the degrees of freedom consumed,
00:06:15.060 --> 00:06:16.680
when we estimate the model.
00:06:16.680 --> 00:06:22.200
Quite often we compare increased
explanation against increased complexity,
00:06:22.200 --> 00:06:24.150
that's a fairly general comparison,
00:06:24.150 --> 00:06:26.490
which we use in multiple different tests.
00:06:26.907 --> 00:06:28.737
So we do that,
00:06:28.737 --> 00:06:29.947
we plug in the numbers,
00:06:29.947 --> 00:06:33.473
we get the result of 3.22.
00:06:33.473 --> 00:06:36.323
We compare that against the proper F distribution,
00:06:36.573 --> 00:06:39.810
we get a p-value of 0.026,
00:06:39.810 --> 00:06:41.820
which has one significance star.
00:06:42.303 --> 00:06:45.153
So they presented two stars,
00:06:45.153 --> 00:06:48.063
The reason, I have no idea,
00:06:48.063 --> 00:06:52.260
but I've done this example in multiple classes,
00:06:52.260 --> 00:06:56.040
over multiple years and I don't
know why this is different.
00:06:56.040 --> 00:06:58.350
It could be that there's a typo in the paper,
00:06:58.350 --> 00:07:03.080
or it could be, that's probably the case,
00:07:03.080 --> 00:07:05.390
because this kind of difference,
00:07:05.773 --> 00:07:10.170
getting that because of rounding error
in the R-squared is quite unlikely.
00:07:10.487 --> 00:07:13.067
So that's the idea of F test,
00:07:13.067 --> 00:07:15.313
you take a constraint model,
00:07:15.363 --> 00:07:17.040
and you take an unconstraint model,
00:07:17.040 --> 00:07:21.240
you calculate the difference per
the degrees of freedom difference,
00:07:21.240 --> 00:07:23.880
you scale it with this thing,
00:07:23.880 --> 00:07:28.830
and then you will get a test statistic that
you compare against the F distribution.
00:07:29.297 --> 00:07:32.327
In more complicated models
00:07:32.327 --> 00:07:36.510
for which we don't know, how
they behave in small samples,
00:07:36.510 --> 00:07:39.480
we use the chi-square distribution
instead of the F distribution.
00:07:39.480 --> 00:07:41.670
But the principle is the same.
00:07:42.453 --> 00:07:46.083
In practice, your software will
do the calculation for you,
00:07:46.083 --> 00:07:49.260
but it is useful to understand that
these calculations are not complicated,
00:07:49.260 --> 00:07:53.760
and have a little bit of understanding
of the logic behind the calculations.