Wisconsin Recall Polls: Why Did The Partisan Surveys Get It Wrong?

Why Did Dems' Polls Get Wisconsin Wrong?

"It was a recall."

That's the explanation offered by AFL-CIO political director Mike Podhorzer for a series of publicly released internal surveys by Democratic pollsters that predicted a much closer margin in the Wisconsin recall election than the actual ballot results.

Those internal polls had shown Republican Gov. Scott Walker leading Democratic challenger Tom Barrett by only a percentage point or two. The Associated Press reported that Walker won the June 5 election by just under seven percentage points, 53.1 to 46.3 percent.

"Anticipating what the turnout is going to be in such a one-off election is extremely difficult, and that led to different assumptions by different pollsters," Podhorzer explained in a conference call with reporters on Wednesday. "A lot of pollsters were really scratching their heads about how to do this."

No doubt pollsters were challenged by the unusual nature of the Wisconsin recall race. But partisan and independent pollsters faced the same obstacles, and the latter produced an overall more accurate forecast of Walker's victory.

As The Huffington Post reported on Tuesday, the final round of independent polls showed considerable variation, presumably due in part to the differing turnout assumptions that Podhorzer identified. Collectively, however, those polls anticipated the ultimate result. As reflected in the HuffPost Pollster chart (filtered to exclude surveys sponsored by campaigns or campaign committees), the independent polls found Walker leading Barrett by a margin of just over six points (51.7 to 45.5 percent).

However, a series of publicly released internal polls sponsored by the Barrett campaign and allied campaign committees showed a much closer race. If the HuffPost Pollster chart had been filtered to show the Democratic polls alone, Walker would have led by just under two points (49.8 to 47.9 percent). The last two publicly released Democratic polls, fielded within the final 10 days before the election, were conducted by Lake Research Partners and Garin-Hart-Yang and showed, respectively, a dead-even race and a one-point Walker lead.

2012-06-08-Blumenthal-WIDemPolls.png

This pattern is not unusual. Scholarly assessments have long observed that publicly released internal campaign polls tend to skew the horse race as compared to more independent surveys, boosting their sponsor by an average of two to three points and dropping their opponent by roughly the same amount.

What explains the pattern? Mark Mellman, a Democratic pollster whose surveys for the Democratic Governors Association were among those released in Wisconsin, said the reason is that campaign pollsters share only their "best numbers."

"In every case, [these are] legitimate numbers," Mellman told HuffPost, "but the best numbers that they do get." In other words, poll results tend to vary randomly on a bell curve, and campaigns have an obvious incentive to release only the most favorable results they obtain.

Mellman's own Wisconsin release had included results for two earlier polls in March and May showing Walker leading by margins of six and seven points respectively. He noted that those numbers likely never would have seen the light of day but for the apparent narrowing trend evident in the closer result of his final poll.

Celinda Lake, whose automated poll conducted over the Memorial Day weekend showed a tied race, told The Huffington Post that her poll had modeled turnout to match the 2010 electorate, but she believes "there was a surge of Republican voters even beyond 2010." She also said that swing voters "broke to Walker over the last days because they were just sick of the recall and didn't think there were grounds for firing him."

A representative of Garin-Hart-Yang was contacted for this story but chose not to comment.

Unfortunately, neither candor about the potential for cherry-picked results nor caveats about head scratching over turnout models were in evidence the week before the election. Both the Barrett campaign and the Wisconsin Democratic Party used the public release of internal surveys to try to discredit a poll conducted by Marquette University Law School that showed Walker leading by seven points (52 to 45 percent). Those who spun the internal polls claimed to be quite certain about the likely turnout.

The Barrett campaign released a statement labeling the Marquette poll an "outlier" and "not credible" because, the campaign argued, it undersampled younger voters, oversampled "some of the reddest counties in the state" and failed more generally to "sufficiently record surging Democratic enthusiasm."

Mike Tate, chairman of the Wisconsin Democratic Party, agreed. In a May 30 conference call with reporters, he described the Marquette poll as "out of step with everything else that is out there, and clearly with the political reality."

Tate also took his criticism a step further. He directly attacked Charles Franklin, the University of Wisconsin, Madison, political scientist who led the Marquette polling team.

"It's unfortunate that Marquette University associates itself with professor Franklin," Tate said. "If he was a pollster, that's what he would do for a living -- he wouldn't teach. I think we have to be careful about using professors instead of pollsters to see what people think."

Full disclosure: Charles Franklin and this reporter co-founded the site Pollster.com, the forerunner of HuffPost Pollster.

Asked why the same poll had shown President Barack Obama leading GOP challenger Mitt Romney, Tate said "It's not my job to explain Charles Franklin's bad math," adding, "It doesn't make any sense."

Again, Walker prevailed over Barrett by seven points, the same margin forecast by the Marquette survey. The network exit poll also showed that recall election voters favored Obama over Romney by seven points (51 to 44 percent), nearly matching the result on the Marquette poll among likely recall election voters (50 to 44 percent).

The actual ballot results and the exit poll also allow for an assessment of the substantive claims Democrats had made about the Marquette poll. Contrary to the criticism, its regional allocation of likely voters closely matched the geographic distribution of the actual turnout, except that the Marquette poll slightly overstated the contribution of voters in Milwaukee City, who went heavily for Barrett.

2012-06-08-Blumenthal-Marquettecomparison.png

The Marquette survey came very close to matching the exit poll in terms of the proportion of black and Hispanic voters and union members in the electorate and in the margin separating self-identified Democrats and Republicans.

Consistent with the Democrats' criticism, it did include significantly fewer voters under the age of 44 (29 percent) than did the exit poll (38 percent). Age, however, is one measure that exit polls seem to get a bit wrong, often describing an electorate that is younger than that indicated by the actual voting records.

These details aside, a more important point is that this sort of analysis is impossible to conduct on the internal partisan polls because none of them has released any details about the demographic, regional or partisan composition of its samples. If the campaign pollsters were "scratching their heads" about the turnout, there is no way for the public to know what their assumptions were.

And what about the attacks on Franklin's professionalism? "I think the world of Charles Franklin," said Lake, who was surprised to learn of Tate's assault. "I think Charles Franklin is one of the finest pollsters in our business."

Does Tate owe him an apology?

That notion is "absurd," Graeme Zielinski, communications director for the Wisconsin Democratic Party, told HuffPost via email, refusing to back off an inch. "Charles Franklin's methodology, his motivations, his funding and his insidious ways with the press who use him as a human quote machine are all up to question."

This episode teaches two lessons. First, given the historical pattern, be skeptical of publicly released partisan polls, especially if little or nothing is disclosed about their methodology. On average, they're probably skewed two or three points in favor of their sponsor.

Second, when the results come wrapped in the sort of mindless mud-slinging practiced by the chairman of the Wisconsin Democratic Party, don't trust them at all.

Before You Go

Popular in the Community

Close

What's Hot