WEBVTT WEBVTT Kind: captions Language: en 00:00:00.240 --> 00:00:05.529 Reliability is one way that the randomness can influence your study results. 00:00:05.529 --> 00:00:11.080 But it is not the only way and therefore it is important to understand how different sources 00:00:11.080 --> 00:00:14.749 of error are related and how they differ. 00:00:14.749 --> 00:00:20.320 Reliability and sampling error are two main sources of random error. 00:00:20.320 --> 00:00:24.810 So reliability error is considered measures of individuals. 00:00:24.810 --> 00:00:32.189 How much your individual measures vary from one measurement occasion to another? 00:00:32.189 --> 00:00:37.390 Sampling error on the other hand is another kind of process that produces random differences 00:00:37.390 --> 00:00:38.910 to your study results. 00:00:38.910 --> 00:00:46.480 So sampling error refers to the composition of the sample and how it varies from one random 00:00:46.480 --> 00:00:48.050 sample to another. 00:00:48.050 --> 00:00:53.330 So the idea here is that reliability is measure of the same individuals - how much they vary 00:00:53.330 --> 00:00:58.980 from one study to another if we do the same measurement producers. 00:00:58.980 --> 00:01:04.549 Sampling error refers to how much different repeated random samples on the same population 00:01:04.549 --> 00:01:06.810 influence the study results. 00:01:06.810 --> 00:01:16.570 Then definition for reliability is how much measure of trait of individual varies over 00:01:16.570 --> 00:01:18.229 measurement occasions. 00:01:18.229 --> 00:01:23.640 For sampling error how much statistical estimation varies over repeated samples because different 00:01:23.640 --> 00:01:26.039 samples contains different individuals. 00:01:26.039 --> 00:01:28.249 So this again is a bit different. 00:01:28.249 --> 00:01:33.060 Causes of reliability and sampling error are also different. 00:01:33.060 --> 00:01:35.929 Reliability is related the quality of your measurement instrument. 00:01:35.929 --> 00:01:38.530 It's not related to sample size. 00:01:38.530 --> 00:01:45.270 And sampling error on the other hand is related to sample size and population heterogeneity 00:01:45.270 --> 00:01:49.119 only if you have random sample. 00:01:49.119 --> 00:01:52.190 Consequences of these two sources of error are also different. 00:01:52.190 --> 00:01:58.399 So reliability if you have unreliable measures then statistical associations using those 00:01:58.399 --> 00:02:06.280 unreliable measures will be inconsistent and biased unless you explicitly take reliability 00:02:06.280 --> 00:02:13.770 into account in your statistical models which can be a bit complicated to do. 00:02:13.770 --> 00:02:19.280 Sampling error on the other hand simply influences the efficiency of estimates. 00:02:19.280 --> 00:02:24.670 If you have a high sapling error then your results could still be unbiased. 00:02:24.670 --> 00:02:26.670 They're just less precise. 00:02:26.670 --> 00:02:32.610 So these two sources of random error have different - very different - consequences. 00:02:32.610 --> 00:02:37.110 Sampling error is easy to reduce by increasing sample size. 00:02:37.110 --> 00:02:43.160 To reduce unreliability you have to improve your measurement practices. 00:02:43.160 --> 00:02:45.850 Let's take a look at the bathroom scale example. 00:02:45.850 --> 00:02:50.740 So reliability here is whether... 00:02:50.740 --> 00:02:57.500 Sampling error is whether these real weights here - these people here - are an accurate 00:02:57.500 --> 00:02:59.540 presentation of the population. 00:02:59.540 --> 00:03:03.060 So is this a representative sample? 00:03:03.060 --> 00:03:05.510 That's what the sampling error is about. 00:03:05.510 --> 00:03:11.880 If we happen to take shorter people than on average on the population by chance to our 00:03:11.880 --> 00:03:18.950 sample then we will have larger sampling error than we would have if our people in the sample 00:03:18.950 --> 00:03:21.750 are closer to the population mean. 00:03:21.750 --> 00:03:28.490 Then reliability is whether these real weights and the bathroom scale readings actually agree. 00:03:28.490 --> 00:03:33.980 So assuming complete perfect validity there would be no other measurement. 00:03:33.980 --> 00:03:36.300 So these are two different quantitative. 00:03:36.300 --> 00:03:38.910 Is a sample representative? 00:03:38.910 --> 00:03:40.470 That's the sampling error. 00:03:40.470 --> 00:03:45.140 Are measures representative of these actual values? 00:03:45.140 --> 00:03:49.040 How closely they match that's the question of reliability. 00:03:49.040 --> 00:03:51.040 Assuming perfect validity. 00:03:51.040 --> 00:03:56.040 We can also take a look at this through the hierarchy of reliability and validity. 00:03:56.040 --> 00:04:00.850 So reliability is whether you have the same sample measured again do you get the same 00:04:00.850 --> 00:04:03.290 result. 00:04:03.290 --> 00:04:07.920 Sampling error goes to the statistical conclusion about validity and is about if you have a 00:04:07.920 --> 00:04:11.670 new sample from the same population do you get the same result. 00:04:11.670 --> 00:04:16.999 If you don't get the same result from a new sample then either you have reliability problem 00:04:16.999 --> 00:04:19.409 or you have sampling error issue. 00:04:19.409 --> 00:04:24.849 If you have a different result from the same sample then it's a reliability issue. 00:04:24.849 --> 00:04:30.479 Then finally we have the external validity which is about new population. 00:04:30.479 --> 00:04:36.400 So if our results don't generalize if we repeat the study with the new population and we don't 00:04:36.400 --> 00:04:42.379 get the same result then if we have ruled out sampling error we have ruled out unreliability 00:04:42.379 --> 00:04:46.800 then it's a issue of external validity or generalizability. 00:04:46.800 --> 00:04:54.659 So these are different sources of random error or why your measurement results could be different 00:04:54.659 --> 00:04:56.389 from one study to another. 00:04:56.389 --> 00:05:00.249 And it's important to understand what the differences are because they are sometimes 00:05:00.249 --> 00:05:00.859 confusing.