My wife asked me if I would run to the grocery store last weekend. Given my general lack of focus, I wanted to make sure I got everything we needed. After consultation with my wife, here was my list:
•Eggs
•Peaches
•Iced tea
•Ketchup
•Hamburgers
I happily ran off to the Giant (a DC-area supermarket), quickly worked my way through the store, and checked out in record time. My total was $27.25.
When I got back home to unpack the groceries, my wife was looking at the receipt and asked me what I had bought. I told her I stuck exactly to the list. She was surprised, because she had gone shopping for the same five things last week, and it only cost her $22.15.
Hmm... we double checked, and sure enough, we had bought the exact same five items. However, I particularly like peaches and ketchup, and so where my wife bought one peach, I had bought five. Where my wife bought one small bottle of ketchup, I bought the giant family size bottle. Buying just those two items in different quantities increased the bill by $5.10.
So what does this have to do with polling, and particularly the post Labor Day CNN headline that says "Nine Weeks Out, A Near Even Race"?
At the grocery story, even though my wife and I bought the same things, my bill was higher because I bought more of some items on the list. In polling, we see a very similar phenomenon--candidates appearing to gain support depending on what groups of people are polled, and how these groups are weighted by the pollsters.
All polls rely on the underlying sample--and to the extent that different underlying groups vote in different proportions, the results could change. To give you one example, the Economist/YouGov poll from August 26-29 is one poll that provides detailed breakdowns of vote support by demographic groups. The overall result of this poll had Hillary Clinton up by five points. But, when you look under the surface at key groups, you can see differences:
•Female voters: Clinton +10
•Male voters: Trump +2
•Under age 30: Clinton +26
•Over age 65: Trump +6
•Income Under 30K: Clinton +7
•Income Between 50K and 100K: Clinton +1
These numbers aren't surprising, given that throughout this election, Hillary Clinton has consistently held significant leads with women, African American voters and college educated voters. Donald Trump has consistently held leads with white men, and non-college educated voters.
But what would happen if the pollsters only polled women? Or if they only polled seniors? The results of the poll would be much different. If more women were polled, we might expect Hillary's numbers to be higher. If more seniors were polled, we would expect Trump to narrow the gap, or even take the lead.
Now consider this: What would happen if either of these groups actually turned out in higher-than-expected numbers on Election Day? Every poll has a turnout model that weights the proportion of voters that makes up their aggregate total. These models often get less attention from the media, but they matter a lot. As you look at the polls, it is not just the aggregate totals that matter--it's the proportion of voters in each block that matters, and how many of them will actually vote on Election Day.
Just as my supermarket tab was more expensive because I bought a lot more peaches and ketchup than my wife, if I overweight a key group one way or another in the poll results, it might appear that the race is tightening or widening, depending on the results--when that may not actually be the case.
As a sound consumer of polls, it's important to look under the surface for shifts in support amongst key groups, and to understand the turnout models. Ultimately, who turns out on November 8th will dictate the only poll results that matter, the election.