WEBVTT WEBVTT Kind: captions Language: en 00:00:00.060 --> 00:00:05.160 If you're interested in research methods it's  useful to know where you can learn more about what   00:00:05.160 --> 00:00:11.220 methods are actually used in your field and where  to get advice if you're stuck with the problem. 00:00:11.220 --> 00:00:14.400 Of course if you're taking a  course then it's a good idea   00:00:14.400 --> 00:00:17.490 to ask the course staff but there  are also other options available   00:00:17.490 --> 00:00:21.780 and you should familiarize yourself with  these options already during the course. 00:00:21.780 --> 00:00:26.640 The first option that's available  is that many academic organizations   00:00:26.640 --> 00:00:31.290 offer research methods training or have  interest groups for research methods. 00:00:31.290 --> 00:00:36.240 I'm a member of the academic management  research methods division and that's one   00:00:36.240 --> 00:00:40.590 great place to learn about about  different methodologies. This is   00:00:40.590 --> 00:00:46.860 their website that contains some resources  but more importantly these people organize   00:00:46.860 --> 00:00:51.120 trainings at conferences and you  can participate if you're a member. 00:00:51.120 --> 00:00:54.690 So consider if you're a member  of academic management consider   00:00:54.690 --> 00:00:56.370 joining the research methods division. 00:00:56.370 --> 00:01:01.920 Of course other academic organizations  have similar interest groups or units   00:01:01.920 --> 00:01:04.260 or divisions or groups or whatever they're called. 00:01:05.190 --> 00:01:07.500 The academic management research methods division   00:01:07.500 --> 00:01:12.360 also organizes pre-conference workshops  or professional development workshops. 00:01:12.360 --> 00:01:18.300 Here are some examples. For example they have  this kind of sessions where you can meet people   00:01:18.300 --> 00:01:24.600 who know a lot about methods. This is for  example if you like the paper by Akunis and   00:01:24.600 --> 00:01:31.200 Wanderberg then you can go and see Bob Wanderberg  and ask questions from him. You may also want to   00:01:31.200 --> 00:01:36.030 ask questions from these other people that are  experts in methods or some fields of methods. 00:01:36.030 --> 00:01:40.050 So there are these open settings where  people who know generally about methods   00:01:40.050 --> 00:01:43.710 come and you can talk to them. You  can even take your paper with them   00:01:43.710 --> 00:01:48.450 or give a regression table to them and ask  them what can I do about this and then you   00:01:48.450 --> 00:01:53.520 get one-on-one feedback. So that's a very  useful way of seeing people informally. 00:01:53.520 --> 00:02:02.430 These conferences also host a various number of  introductions to certain topics or workshops or   00:02:02.430 --> 00:02:10.560 short courses about topics. For example I  liked to it's outliers around based on the   00:02:10.560 --> 00:02:16.720 article by Herman Aguinis and his students.  So they have been presenting this article of   00:02:16.720 --> 00:02:24.010 how to identify define and handle outliers  a half-day workshop for multiple years. So   00:02:24.010 --> 00:02:27.550 if you want to know more about outliers  or ask questions how you should deal with   00:02:27.550 --> 00:02:32.710 outliers in your research then this kind  of workshop is very valuable. Of course   00:02:32.710 --> 00:02:37.150 there are tens of other workshops as  well that are equally high quality. 00:02:37.150 --> 00:02:43.510 Then outside conferences you may also want to  have a place to ask for help. You can of course   00:02:43.510 --> 00:02:48.370 use money - if you take a course then using  the course forum is a good idea but there are   00:02:48.370 --> 00:02:55.300 public forums that you can also ask. Here  are some examples. SEMNET is a 25 year old   00:02:55.300 --> 00:03:03.940 email list that has a couple of thousand - if  I remember correctly its 3500 members - that   00:03:03.940 --> 00:03:09.670 are either interested in methods to - so that  they can learn more or are experts in methods   00:03:09.670 --> 00:03:15.190 particularity structure regression modeling but  also others. So you can post your questions if   00:03:15.190 --> 00:03:18.790 you haven't data analytical question like  how do I interpret the chi-square statistic   00:03:18.790 --> 00:03:24.190 in structural regression model then you can  post it there and you will certainly get lots   00:03:24.190 --> 00:03:29.740 of responses. You will find out that there often  multiple different perspectives on an issue and   00:03:29.740 --> 00:03:36.640 then you will learn more and then you can make  your own judgment on which advice to follow. 00:03:36.640 --> 00:03:40.750 Even if your question is not related to structural   00:03:40.750 --> 00:03:44.230 regression modeling this is a big list  of people then you can ask like what's   00:03:44.230 --> 00:03:47.980 the best place to ask questions about  and somebody on the list will know. 00:03:47.980 --> 00:03:53.170 Then we have the academic management research  matters division list. That's a bit smaller.   00:03:53.170 --> 00:03:58.960 There's a few hundred people - maybe a thousand  people - and the questions are less specific to   00:03:58.960 --> 00:04:04.960 data analysis problems but they are more about  research design. Like you have a problem of   00:04:04.960 --> 00:04:09.370 whether you want to - whether should you include  a control or not that's a good place to ask. 00:04:09.370 --> 00:04:15.430 Then if you are interested in a learning about  particular software. Softwares typically has   00:04:15.430 --> 00:04:21.880 some forums for example Stata has as stata  list dot org and people with all levels of   00:04:21.880 --> 00:04:27.670 expertise in Stata ask questions. Like  you could ask how do I get started or   00:04:27.670 --> 00:04:32.020 you could ask my multi-level model  doesn't converse here's the output   00:04:32.020 --> 00:04:36.010 what should I do. And there will be people  who are willing to answer your questions. 00:04:36.010 --> 00:04:41.350 So there are many places on the internet  that you can ask these questions. Because   00:04:41.350 --> 00:04:46.240 the email lists are fairly large it  guarantees that there is some forms   00:04:46.240 --> 00:04:51.490 of self censoring. So if someone is going to  answer your question and they know that it's   00:04:51.490 --> 00:04:57.190 going to go to thousands of their colleagues  they want to be sure that their email response   00:04:57.190 --> 00:05:00.940 is well written and correct and that's a  good thing for you if you ask questions. 00:05:00.940 --> 00:05:07.120 Then it's also possible to take courses.  I recommend also looking outside your own   00:05:07.120 --> 00:05:11.560 department. One of the best courses that I  have taken was in psychology department and   00:05:11.560 --> 00:05:18.070 that really taught me a lot of things and also  when you learn methods for other applications   00:05:18.070 --> 00:05:24.400 then it gives you a broader picture of what  you can do with them in your own field as well. 00:05:24.400 --> 00:05:31.840 Then there are also online courses. I recommend  Coursera but one problem with online courses is   00:05:31.840 --> 00:05:39.280 that basically anyone with the microphone can  record online video and put a price tag on it   00:05:39.280 --> 00:05:46.150 and post it on some services. So some of these  services that are available online are not very   00:05:46.150 --> 00:05:51.490 high quality. They rely on just having large  amount of students who take a course for free   00:05:51.490 --> 00:05:55.360 and the course may not be any good. The  problem is that if you're just looking at   00:05:55.360 --> 00:05:59.500 the course you may not know. So beware of.  I call these scams because some of these   00:05:59.500 --> 00:06:06.850 courses are taught by people who may not know  there the subsidiary as well as they should. 00:06:06.850 --> 00:06:12.820 So when you're looking at whether to take an  online course there are two things that you   00:06:12.820 --> 00:06:20.560 should look at. First is the person an expert.  So if you are taking an econometrics course then   00:06:20.560 --> 00:06:26.530 if it's run by an econometrics professor then  it's probably a better course than if it's run   00:06:26.530 --> 00:06:31.870 by a marketing professor. This is of course  another bullet proof criteria and there are   00:06:31.870 --> 00:06:35.770 some really good marketing professors  out there. But looking at whether the   00:06:35.770 --> 00:06:39.220 specialty of the person is the thing that  they're teaching is a good indication. 00:06:39.220 --> 00:06:44.080 Another that you can think that you  can look at is whether it's branded by   00:06:44.080 --> 00:06:48.850 a university. So if a course is branded  by University instead of being branded   00:06:48.850 --> 00:06:54.100 as a person teaching a course online then  that gives some credibility to the course   00:06:54.100 --> 00:07:02.020 because universities don't want anyone to be  teaching incorrect things using their brand. 00:07:02.020 --> 00:07:07.750 I recommend particularly the Duke  University courses on Coursera. Data   00:07:07.750 --> 00:07:12.370 analysis and statistical inference is a  great course. They use R and they cover   00:07:12.370 --> 00:07:16.990 the basics and then they go more advanced  to regression model. They don't really cover   00:07:16.990 --> 00:07:21.670 research design as much as they cover  the basics of statistics but that's a   00:07:21.670 --> 00:07:29.860 really really valuable course. It's one of the  most highly rated massive online courses ever. 00:07:29.860 --> 00:07:37.300 Then another nice way of learning about things is  to do examples online. University of California   00:07:37.300 --> 00:07:42.970 Los Angeles statistics department has these data  analysis examples. This is their old website so   00:07:42.970 --> 00:07:49.960 the new website is a bit different but the idea  is that they have a list of different statistical   00:07:49.960 --> 00:07:55.570 analysis that you could apply for data and  they have different web pages for different   00:07:55.570 --> 00:08:00.340 statistical software. They have data. They  have SAS which is a bit older software that   00:08:00.340 --> 00:08:07.540 I can't recommend anymore. Then there is a SPSS  that I don't recommend although it's commonly   00:08:07.540 --> 00:08:12.160 used. There's mPlus which is a specialized package  for structure regression modeling and there's R. 00:08:12.160 --> 00:08:16.750 So you can compare how a particular  analysis would be executed in these   00:08:16.750 --> 00:08:22.510 five difference software packages and  also these websites tell you how you   00:08:22.510 --> 00:08:27.010 interpret the results when you would apply  the analysis. So that's quite useful. They   00:08:27.010 --> 00:08:31.840 have lots of examples from textbooks. For  example they - I think the prestese data   00:08:31.840 --> 00:08:36.460 is that I used in a different video  is analyzed in one of these examples. 00:08:36.460 --> 00:08:42.880 Then it's always a good idea to read  books and if you want to learn about   00:08:42.880 --> 00:08:48.400 statistical analysis methods instead  of finding a book that says something   00:08:48.400 --> 00:08:54.010 about everything. You should find the best book  about a particular method that you want to use. 00:08:54.010 --> 00:08:58.270 Here are some of my favorites.  For regression analysis I like   00:08:58.270 --> 00:09:03.940 Wooldridge's introductory econometrics and  Cohen's regression book is a classic as well   00:09:03.940 --> 00:09:07.630 although I think a proper econometrics  book is better than Cohen's book. 00:09:07.630 --> 00:09:15.520 Then if you want to do logistic regression  analysis then Hosmer and Lemeshow is a classic.   00:09:15.520 --> 00:09:21.160 So sturdivant and Lemeshow also teaching a  course about logistic regression analysis   00:09:21.160 --> 00:09:26.590 on Coursera - using Stata. So you can follow  that course and for mixed effects regression   00:09:26.590 --> 00:09:31.660 analysis Rabe-Hesketh's book on multi-level  and longitudinal modeling using data is a   00:09:31.660 --> 00:09:36.730 great applied book and if you want to do surveys  there are a couple of books that I recommend.   00:09:36.730 --> 00:09:41.320 So depending on what you want to do - find  the best book about the particular design. 00:09:41.320 --> 00:09:46.510 Let's assume that you want to do a survey  project where you send out questionnaires   00:09:46.510 --> 00:09:51.790 with multiple index scales and multiple  indicator scales and you want to analyze them   00:09:51.790 --> 00:09:56.590 with factor analysis and you want to analyze  the scale scores with regression analysis. 00:09:56.590 --> 00:10:02.590 So what you do is that you find a book about  survey sampling. You find a book about survey   00:10:02.590 --> 00:10:08.020 instrumentation by Dillman. You find a book  about measurement the factor analysis for   00:10:08.020 --> 00:10:14.500 example Devilish is a great book and then  you find a book about regression analysis.   00:10:14.500 --> 00:10:19.780 You study those books while you do your  project and you will learn as you do. 00:10:19.780 --> 00:10:27.190 Yet another good great resource is  research methods journals. So you   00:10:27.190 --> 00:10:33.250 probably follow some journals that deal with  the topic of your study but there are also   00:10:33.250 --> 00:10:37.750 journals that are about research methods  and these fall into two broad categories. 00:10:37.750 --> 00:10:43.030 The first category is technical research  methods journals that focus on the   00:10:43.030 --> 00:10:49.810 development of new techniques and you will  know these journals based because you can't   00:10:49.810 --> 00:10:53.350 understand what they're talking about.  So these are mathematical journals that   00:10:53.350 --> 00:10:58.030 present simulation studies and things like  that that are meant for people who develop   00:10:58.030 --> 00:11:02.440 analysis techniques for others. So they  are not really for applied researcher. 00:11:02.440 --> 00:11:06.340 Then there's the other category  applied research methods journals   00:11:06.340 --> 00:11:11.860 such as organizational research methods and  psychological methods that are primarily   00:11:11.860 --> 00:11:19.510 aiming for researchers who are interested  in applying techniques to their field. For   00:11:19.510 --> 00:11:25.360 example the organizational research methods  or editorial statement says that it's meant   00:11:25.360 --> 00:11:30.460 for advancing the understanding of research  methods and current practice research methods   00:11:30.460 --> 00:11:34.990 in the field of management. And psychological  methods is the same but it's for psychologists. 00:11:34.990 --> 00:11:41.920 These journals are meant for researchers who  have a PhD done using quantitative techniques.   00:11:41.920 --> 00:11:48.280 So it's not our the best place to learn the  basics but once you learn the basics following   00:11:48.280 --> 00:11:53.350 one of these research methods journals is a  good way of keeping up with what's going on   00:11:53.350 --> 00:11:57.190 in the field and what are the best latest  and greatest things that you could apply. 00:11:57.190 --> 00:12:04.630 There are - One nice way of following these  journals is that many journals present their   00:12:04.630 --> 00:12:11.200 article assess an RSS feed. So it's here. You  can google a bit more about this RSS feeds but   00:12:11.200 --> 00:12:18.130 the idea is that this link provides a list of  articles that the journal publishes and you can   00:12:18.130 --> 00:12:24.610 use an RSS reader software to subscribe to this  link and then once you have subscribed every   00:12:24.610 --> 00:12:29.920 time this article - this journal publishes new  articles you will be notified and you will get   00:12:29.920 --> 00:12:35.980 the abstract and the title and authors and then  you can decide whether you want to read it or not. 00:12:35.980 --> 00:12:43.210 So here is my RSS feed or I use feedly  which is an online software. I log in   00:12:43.210 --> 00:12:47.380 and I in and I click on those RSS links  and I subscribe them into feedly and   00:12:47.380 --> 00:12:54.040 this is my academic feed. All my journal  articles. I also follow some others and I   00:12:54.040 --> 00:13:00.400 follow some 20-30 different journals. I  change these quite frequently so I know   00:13:00.400 --> 00:13:05.440 what's going on around the topics that I  study and around the methods that I use. 00:13:05.440 --> 00:13:09.310 So this is a great way - the RSS  reader to keep up with your field and   00:13:09.310 --> 00:13:15.520 you should subscribe to also some research  methods journal if nothing else subscribe   00:13:15.520 --> 00:13:18.970 to organizational research methods. And  as a disclaimer I'm an editorial board. 00:13:20.570 --> 00:13:25.610 Then you should work with more experienced people.  So your supervisor is a great resource if he or   00:13:25.610 --> 00:13:31.100 she knows about quantitative techniques.  Also some other colleagues could be good   00:13:31.100 --> 00:13:36.050 resources for you. If you have data analysis  problem it may be a good idea to ask somebody   00:13:36.050 --> 00:13:40.190 who is more experienced than you to become  a co-author to help you with the analysis.   00:13:40.190 --> 00:13:45.710 And if you have a good idea then people  tend to agree with this kind of requests. 00:13:45.710 --> 00:13:49.760 Then you can also ask your supervisor  - if you're a very beginner - if the   00:13:49.760 --> 00:13:54.800 he or she has data sets that would need to be  analyzed and with that you can practice with. 00:13:54.800 --> 00:14:03.560 Then at some point you start your own projects.  So for me when I was starting doing learning   00:14:03.560 --> 00:14:08.210 research methods I started by reading a book and  I understood basically nothing from the book and   00:14:08.210 --> 00:14:13.910 I was not motivated to read the book at all. I  did a project. I decided that I will do a survey   00:14:13.910 --> 00:14:19.610 study. I did the study and then I applied  structurally regression modeling because I   00:14:19.610 --> 00:14:27.140 saw that other people applied that as well. Then I  did something. I got some results. I submitted the   00:14:27.140 --> 00:14:33.410 paper to conference. I got the reviewer comments  back and then the reviewer comments said that the   00:14:33.410 --> 00:14:37.250 author really doesn't seem to understand what  he's doing and the review of course was right   00:14:37.250 --> 00:14:42.830 because I was just applying method by looking  at what others had done with the same method   00:14:42.830 --> 00:14:48.860 and without really knowing what I was doing  but the good thing was that then that little   00:14:48.860 --> 00:14:55.340 project really motivated me by indicating what  things I should know that I did not and then I   00:14:55.340 --> 00:15:01.040 started studying about those particular things and  eventually building up my competence. So working   00:15:01.040 --> 00:15:06.980 on datasets while at the same time studying  at least that's what worked for me as well. 00:15:06.980 --> 00:15:11.990 I mentioned the psychology department course  about structural regression modeling that's   00:15:11.990 --> 00:15:16.490 the best one that I have taken. In that  course I was all the time applying the   00:15:16.490 --> 00:15:22.220 techniques that they taught in the class on  my own data set the nights after the course   00:15:22.220 --> 00:15:27.440 and in the mornings I could ask the instructor  some questions that - like how do you apply this   00:15:27.440 --> 00:15:32.450 technique to my data? What does this statistic  of this result mean in the context my data? 00:15:33.590 --> 00:15:38.090 Also if you want - if you're going  to be investing a lot of effort in   00:15:38.090 --> 00:15:45.680 data collection - plan well ahead because the  quality of your study is mostly determined by   00:15:45.680 --> 00:15:52.070 how well your sample and do you measure  the right things and do you measure the   00:15:52.070 --> 00:15:58.010 things correctly - if you omit for example  a control variable in a survey study it's   00:15:58.010 --> 00:16:03.230 nearly impossible to collect it afterwards.  So planning a lot before actually doing   00:16:03.230 --> 00:16:08.780 is a good advice if you want to do high  quality research without redoing a lot. 00:16:08.780 --> 00:16:15.830 Then finally there are two kinds of  answers to all scientific questions.   00:16:15.830 --> 00:16:19.400 Sometimes somebody tells you that you don't need   00:16:19.400 --> 00:16:23.060 to care about heteroscedasticity. You  don't need to care about this thing or   00:16:23.060 --> 00:16:26.990 that thing. Just apply this technique and  you're gonna be fine in all situations. 00:16:26.990 --> 00:16:34.430 Such simple answers are unfortunately not  often right. Sometimes we have a complex   00:16:34.430 --> 00:16:40.070 problem and the complex problem requires a  complex solution. So you have to actually go   00:16:40.070 --> 00:16:46.670 and study but then studying things will  produce better research and it'll make   00:16:46.670 --> 00:16:53.330 you - it's also more motivating for your que  when you know that you can trust your results.