Gun Control, Politics, Statistics

About that Pew Poll on Gun Rights

Pew recently published results of a poll showing increasing support for gun rights and declining support for gun control. The results are somewhat startling, and a bit counterintuitive (support for gun rights increased especially among black respondents), so it’s not terribly surprising that the poll received a fair amount of news coverage.


Time series of responses to Pew poll on gun rights.

Pew is one of the most reputable pollsters around. That said, there are a few things to consider here that probably deserve scrutiny before accepting the conclusion that the majority of the country has truly shifted their attitudes and now firmly oppose gun control measures. The first is the question wording:

What do you think is more important – to protect the right of Americans to own guns, OR to control gun ownership?

Survey researchers generally like to ask questions like this, where respondents are forced to make a choice between competing values rather than simply asking people what they think about one or the other. It tends to induce greater variation in responses and (hopefully) reveal subtle differences in attitudes that might otherwise go unmeasured. For instance, if you ask respondents whether they want to reduce the budget deficit, most will always yes. If you ask respondents whether they want to reduce funding for Social Security or Medicare, most say no. Only by asking respondents to consider a trade-off between the two do you get useful information about where the public’s priorities lie.

That said, I’m not a fan of the wording here because it seems to me rather imprecise and complicated. My guess is Pew keeps asking it because they’ve been asking it for so many years and want to be able to track changes in the time series. But a question like this is really asking about more than just support for “the right of Americans to own guns,” but also how respondents think about the notion of government “control.” And that means there are a number of additional considerations about government that go into a question like this, and a lot of complicated and shifting notions about what “control” of “gun ownership” might mean.

It would be nice to know what respondents think “control gun ownership” means in 2014 versus the 1990s. My guess is they may not be thinking of the same things. In the 90s there were a number of gun control initiatives in the news–the Brady bill, the assault weapons ban, etc.–that were very likely on the minds of survey respondents when they were asked to make this calculation. With the exception of the months immediately after Newtown, when there was a brief decline in support for gun rights, gun control policy has largely been absent from the national agenda.

At the same time, the notion that the “right of Americans to own guns” might be imperiled has been the refrain across conservative media since President Obama’s election in 2008. You may recall the boom in gun buying after the president’s first inaugural. (I look forward to watching what looks to be a excellent Frontline documentary on the NRA’s public relations campaign in the months after the Newtown tragedy.) These sorts of media messages have a real impact on voter’s perceptions of political matters, particularly around such vague concepts as restrictions on “the right of Americans to own guns.”

The poll did include a single follow up question that (somewhat) got at how respondents view guns. The survey asked whether gun ownership does more to “protect people from becoming victims of crime” or “put people’s safety at risk” (with the order of the two response categories randomly alternated). Pew noted that there was a sizable increase in respondents saying the former, that guns do more to protect people (from 48 percent in Dec. 2012 to now 57 percent) than put people’s safety at risk. So more respondents now support gun rights because more see the value of guns in protecting people from crime? Case closed.

The problem is this increase is entirely attributable to a decline in the percentage of people who previously refused to offer a response in 2012, the last (and only?) time this particular question was asked by Pew. In fact, the percentage saying guns put people’s safety at risk remained flat (it went from 37 to 38 percent). So did a significant number of Americans really shift their fundamental view of guns? Seems unlikely to me. Instead, the increase Pew notes probably reflects the decline in the politicization of guns since the weeks after Newtown (when the 2012 poll was fielded). Perhaps Americans who felt uncomfortable affirming their belief that guns could be helpful in protecting people — after young children had just been gunned down at school — are now more willing to say so publicly.

This may explain some of the increase, but probably not all of it. The poll found a 25 percentage point increase in the number of Black respondents who said guns did more to protect people from crime than put people’s safety at risk. These respondents actually went from being among the most concerned about guns putting people’s safety at risk to being majority pro-gun (by a 13 percentage point margin). Some of this change may be sampling error (the overall sampling margin of error for Black respondents on the latest poll is +/- 9.8 percentage points). Some of the change may reflect real shifts in attitudes; in June, the New York Times ran a feature on minority gun ownership and the NRA’s efforts in this regard. But the size of the apparent change makes me think something else is going on here.

Let’s remember that this poll was conducted in the days after Ferguson and the Garner grand jury decision in New York (Dec. 3-7). In the full report released by Pew, the item on gun control appeared as number 53 in the questionnaire. Unfortunately, most of the earlier items have been “HELD FOR FUTURE RELEASE.” This is not good practice, although it’s not the worst sin in terms of polling transparency.* Pew is a legit pollster. I would be shocked if they stacked the first part of the questionnaire with items that would have overly influenced the way respondents were thinking about issues of government control or public safety (the two values respondents are asked to weigh in the gun rights question). But it is worrisome that the two preceding questions that have been released were about Obama’s handling of race relations and whether “it would be a good idea or a bad idea for more police officers to wear body cameras that would record their interactions.” (Other questions about Garner and Ferguson that have been released appeared later in the questionnaire.)

The ordering of questions may not have ultimately mattered much regardless though. These grand jury decisions and the nationwide protests that followed were almost certainly on many respondents’ minds. Indeed, 75 percent of all respondents in the poll said they had heard “a lot” about Ferguson. The salience of these instances of violence, police brutality, and the chaos that followed the grand juries’ inactions wouldn’t explain the gradual increase in support for gun rights since the 90s; I suspect the change in administrations and the decline of gun control policy as a focus of media attention and national policy are major explanations for that. But I would bet that recent news has a lot to do with the spike in support for gun rights in this particular poll compared to the earlier periods of the Obama administration. For many respondents, it seems plausible that reports of rioting and/or frustration with government as an effective recourse for public safety, may have been at the top of their heads when asked to make a vague choice between values associated with gun ownership.

The bottom line is that findings in a single poll, even one conducted by an organization as sophisticated and respected as Pew, deserve to be examined a little more skeptically. These are only estimates of public opinion, and sampling error doesn’t come close to capturing all the uncertainty in these numbers. What policymakers are ultimately concerned with are particular, granular changes to current law, and public opinion, typically oblivious to such specifics, may only ever offer limited guidance on the wisdom of those decisions.

My worry is that with numbers of like these and headlines like these:

Screen Shot 2014-12-13 at 4.56.13 PM

the polls become a self-fulfilling prophecy in terms of public policy — not necessarily because they reveal fundamental shifts in public opinion but because they are interpreted that way by leaders in Washington.

* The American Association for Public Opinion Research’s transparency initiative does call for survey organizations to release “any preceding questions or instructions that might reasonably be expected to influence responses to the reported results” within 30 days of a request. Pew was an early supporter of the transparency initiative.



Comments are closed.

Follow me on Twitter

%d bloggers like this: