He is working towards rendering a 200-year old idea serviceable to educational research, became a methodologist almost by accident and is now our guest at DIPF. In an interview with DIPFblog, Professor Ph.D. David Kaplan from the University of Wisconsin-Madison, a specialist in applied statistics for educational research, talks about his life as a Humboldt fellow in Germany, different ways of working, and Bayesian statistics.
Which was your first impression of DIPF and when was it?
Kaplan: About ten years ago, I met Eckhard Klieme at a conference. He invited me to come to DIPF and give a talk. That’s how it began. Since, I have been here at least once a year, to give talks, workshops or to collaborate with Eckhard and other members of the staff on various reports, such as PISA. We saw each other quite often because we travel to the same PISA meetings, so we became pretty close friends and I have always really enjoyed coming here. Over the years I have felt increasingly more comfortable and at home here, it was very natural to come here for a sabbatical. I love it.
Now that I am staying for a longer time, I have noticed very interesting differences in working style. This one is almost a funny anecdote: Once I came and thought nobody was here because all the doors were closed. I thought it was a holiday and no one had told me. Where I work in the US the doors are always open and people always talk to each other. Here, you make an appointment – which doesn’t mean that it’s not easy to talk to people and it is always easy to talk to someone. But it’s different.
I am also beginning to understand things better. I have had the opportunity to talk to the others about the German research community, not just about DIPF. They explained how research is organised in Germany. I think it is the most remarkable thing I have ever seen: In the US, we don’t have a system quite like this, where you have universities but also these research institutes like the Leibniz Association and Max Planck Society that support research in a federally funded way, somewhat separate from universities.
Apart from the great experience – what is the main benefit of your sabbatical?
Even though my doctorate is in education, I mostly come from statistics and methodology. I have learned more about education from Eckhard Klieme than I have ever learned in school. He and the others have really helped me understand what all of this is about.
Also, one of the things that I learned very early on in graduate school is that for the kind of work I do – which is mostly methodological development – , it’s a very good idea to associate yourself with someone who is really smart about the relevant content – in this case the people I work with at DIPF are really intelligent, thoughtful scholars. If there is a problem that desires a methodological solution, that’s my contribution. The group of ours here works on large-scale assessments like PISA. People like Eckhard Klieme, Nina Jude, and Susanne Kuger to a certain extent are not trained statisticians. They are trained as top educational researchers; and they are very sophisticated at the development of data and how it is being used but sometimes there are these interesting questions that require a statistical solution or statistical advice.
How did you become what you are?
When I was an undergraduate studying psychology, I was particularly interested in cognitive psychology. Yet, as an undergraduate I went to a small university and in those days to do research in cognitive psychology required a lot of equipment – and this little university didn’t have it. So when I was looking for classes to take, the only classes that were really offered were in methods and statistics. At one point, I started to really enjoy the methods more than I enjoyed what the methods were being applied to. So I really started to like statistics, but always applied statistics, applied to educational science, social science, and so on. To make a very long story short, basically I pursued quantitative methodology programs and then ended up with a Ph.D. in education, with a focus on methods. Methodology just really intrigues me because regardless of the subject area people who work on methodology are really interested in finding out how people know things. There is a philosophy to this, which is epistemology. The idea is that the methods that I develop or extend help people like Eckhard Klieme and the others here to learn something from the data that would otherwise not be possible to know.”
How would you explain your work to an outsider?
In a very general way, what people like me do is: We use the theories of mathematics and statistics to develop tools for other people to use. So the analogy would be engineering. If you are an engineer, you usually use the theories of physics or chemistry to build things like bridges, chemicals, or drugs for someone else to use. However, the aspect of building these tools requires that the tools are tested – to make sure that the bridge doesn’t fall down or the drug is not poisonous. People like me develop the tools and test them before we give them away.
My specific area asks how we quantify uncertainty. I am talking about the uncertainty concerning what we think we know about relationships in education. To give an example I actually use in my classes, the relationship between parental income and a child’s educational achievement in the US. When educational researchers do a study like that, they usually come up with roughly the same answer each time: There is a positive relation, it is pretty strong, the parental income has a bearing on the educational attainment of the child. Now one could say: If that’s my research question I have very high precision, and I know quite a lot. Therefore, my uncertainty is limited. But now suppose you’re going to look at something we don’t have a lot of information about – you could even take some PISA-data or some other data-set – and look at a country that is not like the US, some country you have never looked at before, to see whether the parental wealth relates to educational achievement. In such a situation, I might not have the same level of certainty anymore, I am actually really uncertain.
The area that I work in is called Bayesian statistics and it’s a particular field of statistics – it’s a philosophy if you want to think of it that way. It explicitly deals with quantifying the level of uncertainty that an individual researcher might have about a particular relationship. The aim is to directly model the researcher’s uncertainty as part of the analysis. The reason why I am interested in this is because for the first time in years we are now able to actually do this. Prior to the Bayesian revolution, this kind of analysis was not feasible because the software had not been developed and no one knew how to go about it. As an idea, it has been around for 200 years and it just had to wait until computer software was developed. My interest in the last ten years has been to develop Bayesian statistics tools specifically for educational research.
DIPFblog 