I’m a quantitative scholar of American religion and politics, so I am used to everyone from religion scholars unused to working with data to anxious parents asking me random questions.
“Are Latter-day Saints attending church with less frequency in the last few years?”
“How similar are Muslim, Buddhist and Hindu Americans in their political views?”
“Are young Southern Baptists less conservative on abortion than their parents?”
Those questions and many more like them are completely valid and worthwhile to ask. But until about 10 years ago, my answer was something like, “It is statistically difficult, if not impossible, to answer with any degree of accuracy.” (This is how quantitative scholars talk.) Until the mid-2000s, the data that most researchers had available was limited in ways that made many potential areas of inquiry problematic.
For instance, the General Social Survey is the gold standard for any researcher who is interested in how American life has changed since 1972. The GSS contains a wealth of questions focused on a variety of topics, including sexuality, drug use, social connections, religious behavior and views of politics and government.
And it is what we call longitudinal: Every year or two, the team at NORC at the University of Chicago conducts face-to-face surveys with about 2,200 respondents and makes the data publicly available. The GSS is therefore an unbelievably valuable resource and has been the foundation for thousands of articles, books and lectures over the last four decades.
The GSS has one major drawback, however: its small sample size. The total number of people ever interviewed by the NORC team is 64,814.
So, let’s say that I wanted to understand the religious attendance patterns of of Latter-day Saints between 2008 and 2018. If I were to use the GSS data, there are a total of 147 LDS identifiers in the data during that 10-year span. But if broken down in individual years, it’s about 20 respondents each year. It’s difficult to learn much about any group by only analyzing the behavior of 20 of them. Thus, it’s statistically impossible to understand how Mormon religious behavior has changed using the GSS.
However, beginning in 2006 a new survey became publicly available that opened up an entirely new world to social scientists — the Cooperative Election Study. In the 2006 version of the CES, there were 36,421 respondents. There were 32,800 in 2008. In total, the CES boasts a sample size of nearly 447,000 respondents. That’s nearly seven times the size of the entire 46 years of the GSS.
To return to our example of Latter-day Saints, there were 764 LDS members in the 46 years of the GSS. In the 2018 wave of the CES, there were 823 Mormons, but in total there are nearly 7,000. That means that a social scientist can now trace the church attendance of just LDS members under the age of 35, say, or nonwhite Latter-day Saints. It’s also possible to have some reasonable level of statistical certainty surrounding how they have voted in presidential elections dating back to 2008.
Questions about the composition of small religious groups like Hindus or Buddhists were difficult to answer before the advent of large surveys like the CES. It was statistically impossible to trace the religiosity of just 18- to 22-year-old people in college before the introduction of these large-scale surveys. Trying to explore combinations, like people who make six figures a year but have only a high school diploma, was unrealistic when a sample only contained five people who fit those criteria. In the CES and other new instruments, those people number in the hundreds.
I can’t count the number of times people have asked me if there is any time in American history that the country has had lower levels of religiosity than it does right now. Unfortunately, we can’t answer that with any degree of certainty. High-quality data doesn’t really exist prior to the 1970s. And, to be honest, we only have limited statistical ability between the 1970s and the mid-2000s.
I think about these issues quite a lot and am of two minds about it.
The first emotion is one of deep sadness for how much we will never know about the contours of American society just a few decades ago. A religious picture of the United States in the 1940s is incredibly blurry, if not entirely opaque, when seen through the lens of quantitative data.
But I have a great sense of excitement when I consider how much better we will understand American society from this point forward. The questions that social science can ask and answer have never been greater and are expanding by the day as social scientists gather new data and compare what we find to what has come before.
The possibility of knowing more things with a greater sense of certainty should be a thrilling opportunity for all of us.
