WASHINGTON -- At first glance, two new polls of likely Republican caucus-goers released on Wednesday appear to tell very different stories: One shows little change, with Ron Paul holding a slight lead over Mitt Romney. The second shows Romney surging into first place.
Timing explains much of the difference in interpretation -- Romney and Paul have both gained significantly since early December, with less change apparent in the last week -- although the subtle but consistent differences in the findings highlight a big inherent challenge of polling the Iowa caucuses. Methodologies do matter, and we may not know which snapshots are the most accurate until votes are counted next Tuesday night.
CNN and Time, whose poll shows Mitt Romney running slightly ahead of Ron Paul (25 to 22 percent) followed by Rick Santorum (16 percent) and Newt Gingrich (14 percent), had last surveyed likely Republican caucus-goers the first week of December.
Yet the Democratic firm Public Policy Polling (PPP), whose survey gives Ron Paul a similarly narrow lead over Romney (24 to 20 percent), followed by Gingrich (13 percent) and the remaining candidates clustered near 10 percent, had last polled just 10 days earlier.
When compared to the firm's most recent poll, the new PPP results show little change, but when compared to a survey it conducted in early December -- roughly the same time as the last CNN/Time poll -- the trends found by the organizations are very similar.
As shown in the table below, both find huge drops for Newt Gingrich (-14 for PPP, -19 for CNN/Time), and smaller but nearly identical increases for Mitt Romney (+4 for PPP, +5 for CNN/Time) and Ron Paul (+6 for PPP, +5 for CNN/Time. The two pollsters also find increases for Rick Santorum, although CNN/Time shows a bigger gain than PPP (+11 vs. +4).
Those changes -- the dramatic collapse of support for Gingrich and the slower rise for Paul and Romney -- are consistent with the trends captured by other polls, as seen in the HuffPost Pollster chart below, based on all available public polls.
If we had only the most recent polls from PPP and CNN/Time to consider, we might attribute the slight divergence between them to random sampling error, as the differences for each candidate in their most recent polls fall well within the respective margins of error (with the possible exception of the higher support for Rick Santorum on the latest CNN/Time poll). However, an examination of all the polls from both organizations shows some consistent gaps.
The most striking involves support for Mitt Romney. The three polls conducted by CNN and Time have all shown Romney's support roughly three points higher than the overall trend line based on all Iowa polls. The five PPP surveys conducted since October have tracked more closely with the standard trend line based on all public polls.
The two organizations use different methodologies, so some consistent "house effects" should not come as a surprise: CNN/Time uses live interviewers to call a sample of registered Republicans drawn from the list of registered voters provided by the Iowa Secretary of State. PPP uses an automated, recorded voice methodology to call a sample of registered Republicans and independents drawn from the same lists. Both organizations use a combination of self-reported intent to participate and past voting history to select likely caucus-goers, although the details of those procedures are different.
The New York Times' Nate Silver argues that the omission of registered independents from the CNN/Time survey likely results in an underestimation of Paul's support. The caucus rules limit participation to registered Republicans, but registered Democrats and independents can easily switch their registration on caucus night.
The 2008 entrance poll showed that 13 percent of Republican caucus-goers identified as independent (which is not quite the same as reporting that they had been previously registered as independents), and Ron Paul won 29 percent of their vote, far better than he did with Republican identifiers (7 percent).
Similarly, Paul's lead on the current PPP survey comes from voters in its sample who self-identify as independents or Democrats. On the most recent survey, Romney runs slightly ahead of Paul among Republicans (22 to 20 percent) but Paul trounces him in the smaller independent subgroup (46 to 11 percent). The same pattern holds on the previous PPP surveys and those from other organizations: Paul's support is consistently much higher among self-identified independents than among Republicans.
Ironically, however, the biggest differences between the CNN/Time and PPP polls comes not in terms of support measured for Ron Paul, but rather in their estimation of support for Romney and (on the most recent survey) for Rick Santorum. The three CNN/Time polls have found consistently less less support for Paul than PPP has, but the differences have been at most a percentage point or two.
One important potential shortcoming for both surveys: Neither samples mobile phones. As the Washington Post reports, the most recent estimates find that 29 percent of Iowa adults had a cellphone but no landline in their household. The Washington Post/ABC survey conducted in early December found that Paul did better than Romney on interviews conducted over cellphones, while Romney did better than Paul on landline interviews.
We will have many more polls to chew over this week, although we would do well to heed the advice of Republican media consultant Mike Murphy. He warns via Twitter that the "poor SOB's" who participated in past caucuses "are getting, maybe, 863 phone calls a day (robo, events, ID)" from the campaigns. Conducting a telephone poll in this environment is getting more challenging.