Why Pollsters Got The Primary So Wrong

By Bryon Allen/Wilson Research Strategies ~ For those of you paying more attention to campaigns in Kansas, Missouri, Michigan, and Tennessee in the run-up to next week’s big block of primaries, the Oklahoma primaries may have slipped by unnoticed.

But, the Oklahoma political insiders got a bit of surprise last Tuesday as Jari Askins beat sitting Attorney General Drew Edmondson in the Democratic primary for Governor and James Lankford took a first place finish into a run-off for the Republican nomination in the Fifth Congressional District.

This came as a particular surprise because two highly reported polls taken immediately before the primary showed Edmondson leading Askins by 16 in the case of the Tulsa World’s Oklahoma Poll (conducted by SoonerPoll.com) and 11 in the case of the Sooner Survey and the same Tulsa World poll showed Lankford trailing Kevin Calvey by eight.

Now the Oklahoma blogosphere is atwitter with discussion of how the polls could have gotten it so wrong. We are reminded, more than a little, of the discussion in the wake of another major polling failure—the failure of many outlets to correctly model the New Hampshire Democratic primary of 2008.

AAPOR, the professional organization of the polling community, conducted a major investigation of the New Hampshire polling and released a multi-hundred page report of findings.

We think that many of those findings apply in a substantial way to the failure of the public polling in Oklahoma to correctly analyze the outcome of this week’s primaries. But the biggest one is this, primary elections require very well-crafted likely voter models to find the population of actual primary voters and sample from them.

At the very least a good primary voter model will need to:
~ Rely heavily on past turnout behavior to ensure an unbiased, baseline, population of primary voters from whom to sample.
~ Allow for flexibility to include populations of less frequent primary voters who are highly engaged and interested to capture the fluctuation in sub-group turnout that can cause surprise outcomes.

While we don’t have the data in hand to evaluate the two flawed Oklahoma polls’ baseline likely voter models or how they handled the possibility of surges in sub-groups of more irregular voters, we would recommend that both pollsters spend some time analyzing their use of previous behavior and current engagement data as they model the likely electorate in future primary in hopes of avoiding these issues going forward.

Share |