I’m taking a graduate level sociology course, Graduate Research Methods. I am learning all the different ways to “design” a research program in the social sciences. What strikes me is the degree to which much social science research strives to look and feel like “real” science (i.e., the physical sciences) by making it quantitative, usually through the use of statistics. While it is true that interest is growing in “qualitative” social science (ethnographic studies, oral histories) research methods, the quants still rule the roost, for the most part, says my professor. It is impressive how hard social scientists work to make their research quantitatively relevant.
For my purposes here, I call this getting from words to numbers. For example, social scientists use surveys, interviews, and often large sample populations to solicit their raw data. Then they analyze the narrative responses with numbers.
Most of my career has been working in the reverse, numbers to words – that is, explaining hard-core engineering and technology in narrative form – magazine articles, face to face presentations, books, and the like. Most engineers and scientists (and lawyers, I heard from the head of the writing program at my daughter’s Alma Mater) lack good communication skills. Fortunately for my career (and paying my kids’ tuition bills), I am educated as a chemical engineer but I love to write and ended up combining the two into a satisfying career in the energy industry.
I’ve been somewhat obsessed by mathematical and computer models these days – models used in financial engineering, global climate change, environmental assessments, economic development, and many other endeavors. I hope my next book (proposal currently with an agent seeking a publisher) will be on this subject. While models are essential, called in an article published recently in SIAM News (Society for Industrial and Applied Mathematics) the third pillar of science, they also can be abused easily. More critically, most people, educated or not, have little understanding of how these models work or how they impact our decisions, opinions, and those of our political and cultural leaders.
We are talking about MODELS in my class as well. This week, I had an epiphany. Whether you are designing a survey to ferret out some aspect of social behavior, handicapping the presidential election, forecasting what stocks and equities will do, or assessing future impact of climate change, everyone working with a model has a similar root problem: the quality of the data, the accuracy of the measurements, the utility of the raw material that is fed to the “model.”
In a survey, how you phrase a question has everything to do with the nature of the response, as well as a host of other signals, like body language, attitude and cooperation of your subject, and many others. In more quantitative models, the accuracy of your data (temperature measurements, e.g.) and the validity of your assumptions have everything to do with the quality of your output.
garbage in is always going to result in garbage out, qualitative or quantitative. For the most part, models are developed to explain the past and then make some forecast, prediction, or statement about the future (social behavior, consumer prices, inventory, weather patterns, economic growth, etc).
When we are forming an opinion about world events (our internal “models” of the world), conducting a survey for social science research, or trying to forecast what the economy will do in the next months and years, all of us should pay more attention to the quality of the raw data regardless of whether we are converting words into numbers or numbers into words. It’s hard enough to forecast under the best of circumstances. But it’s damn near impossible if the data you’ve collected about the past is suspect.
If you enjoyed this article, please consider sharing it!
