Our latest recap of the accuracy of polling in American elections reinforced something we’ve been saying for a while: “The polls are all right”. In particular, the polls of the presidential general elections have historically been quite accurate.
Polls conducted in the final 21 days before the last five presidential general election had a weighted average error of 4.0 points. We define the error as the absolute difference between the poll’the margin between the top two candidates and the actual vote share at the margin. For example, if the poll gave the Republican candidate a lead of 3 percentage points, but the Democrat won the election by 2 points, if the poll had a 5 point error.) And even in 2016, when many people blamed the polls for not for predicting Events, Trump”s victory, and the polls within 21 days of the election performed decently well, with the weighted average error of 4.9 points.
But in the United States, the Electoral College picks the winner of the presidential election, which means the state polls are what really matter. And the state polls did have an off year, and in 2016 (although still pretty close to the long-term average). They had a weighted average error of 5.3 points, compared with 3.2 for the national polls. Plus, that error is systematically overestimated Democrats: the State polls had a weighted average statistical bias (a metric that tells us which the direction error better place to live in) 3.5 of points toward Hillary Clinton.
The State polls are more error-prone than the national polls
The Weighted average error and the statistical bias of national and state polls in the final 21 days before the presidential general election, among polls in the Genesis Brand’in the pollster ratings database
| The Weighted Average Error | The Weighted Average Statistical Bias | |||
|---|---|---|---|---|
| The Cycle | The National | The State | The National | The State |
| 2000 | 3.8 | 4.6 | R+3.1 | R+2.2 |
| 2004 | 2.2 | 3.5 | D+0.9 | A 1.2 |
| 2008 | 2.2 | 3.8 | 0.0 | A 1.2 |
| 2012 | 3.3 | 3.7 | R+3.2 | R+2.3 |
| 2016 | 3.2 | 5.3 | D+2.4 | T+3.5 |
| All the years | 2.9 | 4.3 | R+0.5 | D+0.6 |
However, it is not unusual for the state polls to be less accurate than the national ones (although they did differ in the lot in 2016). Since 2000, the state polls have a weighted average error of 4.3 points, while the national polls have a weighted average error of 2.9 points. Some have states that are the more accurate polls than the others, though, that. Thanks to our pollster ratings data setnow , we can quantify which states that’ the presidential general election polls are the most and least accurate, which can help us better understand the state polls we’ll get this year, and it is later.
Below is a table of the weighted average, error, and statistical bias of each state’s polls in the last five presidential general election for states that had at least 15 polls conducted during the last 21 days of the campaign. This eliminates the states where the sample size of polls is too small to draw meaningful conclusions and narrows the list to each election”s main battleground states, which tend to get polled the most often anyway).
Swing state polls are usually pretty good
The Weighted average error and the statistical bias of the state polls in the final 21 days before the presidential general election, for states with at least 15, such as polls, among polls in the Genesis Brand’in the pollster ratings database
| The 2000 Election | |||
|---|---|---|---|
| The State | No. of the Polls | The Weighted Average Error | The Weighted Average Statistical Bias |
| Michigan | 21 | 4.1 | R+3.6 |
| Pennsylvania | 21 | 3.8 | R+2.3 |
| In New York City | 17 | 7.7 | R+7.7 |
| Florida | 15 | 3.3 | A 1.0 |
| The 2004 Election | |||
| The State | No. of the Polls | The Weighted Average Error | The Weighted Average Statistical Bias |
| Florida | 28 | 4.9 | A 4.3 |
| Ohio | 25 | 3.4 | D+2.5 |
| Pennsylvania | 24 | 2.0 | R+0.1 |
| Iowa | 16 | 2.8 | R+0.5 |
| The 2008 Election | |||
| The State | No. of the Polls | The Weighted Average Error | The Weighted Average Statistical Bias |
| Ohio | 31 | 3.5 | D 1.1 |
| Florida | 29 | 2.1 | R+0.3 |
| North Carolina At Chapel Hill | 28 | 2.4 | A 1.6 |
| Pennsylvania | 28 | 2.4 | R+0.8 |
| Virginia | 24 | 2.2 | D+0.1 |
| Missouri | 18 | 1.4 | 0.0 |
| Colorado | 16 | 3.4 | R+1.9 |
| New Hampshire | 16 | 4.8 | D 1.1 |
| Nevada | 15 | 7.2 | R+7.2 |
| The Election of 2012 | |||
| The State | No. of the Polls | The Weighted Average Error | The Weighted Average Statistical Bias |
| Ohio | 44 | 1.8 | R+0.2 |
| Florida | 32 | 2.4 | R+1.9 |
| Virginia | 27 | 3.4 | R+3.0 |
| Colorado | 21 | 4.0 | R+3.9 |
| Wisconsin | 21 | 2.9 | R+2.5 |
| Iowa | 19 | 3.8 | R+3.6 |
| New Hampshire | 19 | 3.6 | R & 3.3 |
| The 2016 Election | |||
| The State | No. of the Polls | The Weighted Average Error | The Weighted Average Statistical Bias |
| Florida | 36 | 3.8 | D 2.9 |
| Pennsylvania | 29 | 4.8 | A 4.6 |
| North Carolina At Chapel Hill | 26 | 5.7 | D 5.2 a |
| New Hampshire | 21 | 4.8 | D 3.2 |
| Nevada | 21 | 3.1 | R+1.8 |
| Michigan | 20 | 4.7 | D 4.4 |
| Virginia | 19 | 2.8 | D+1.3 |
| Ohio | 18 | 5.8 | D+5.8 |
| Arizona | 17 | 2.4 | T+1.5 |
| Colorado | 16 | 2.7 | R+1.3 I |
| Georgia | 16 | 3.0 | A 1.8 |
The first thing that might jump out to you are the trouble spots in the 2016 election — one state polls had a statistical bias toward Democrats, for instance. In Michigan, the polls had a weighted average statistical bias of 4.4 points toward Democrats, and in the Pennsylvania polls had a weighted average statistical bias of 4.6 points toward Democrats. North Carolina and Ohio that get less attention but were even more inaccurate.
But the setting of the 2016 aside, the most consistent story tells of the table is how good the swing-state polls are typically. Their weighted average, error is usually not that high — even-in-the-states-that tripped us up in the end of 2016. In 2008, for example, the North Carolina polls had a weighted average error of just 2.4 points. And while Ohio polls missed the mark in 2016, they were dead-on in 2012 (1.8 points). The weighted average error of the Florida polls has been below 4.0 in four out of the five elections. Pennsylvania polls exhibited very low to the weighted average errors of 2.0, in 2004), and 2.4 (in 2008) before their off-year in 2016. And the polls of the other, likely to 2020 ” swing states, such as Arizona (2.4 in 2016), and Wisconsin (2.9 in 2012), have also boasted the very low-weighted average errors in the past.
None of these states appears to be systematically a person to the poll, either. Just like the polls overall are not getting any more or less accurate over time, the error and the statistical bias of a state poll bounces around unpredictably from year to year. While there are some reasons to be concerned that’s some flaws in the state-level to the 2016 polls, including the lack of weighting by education, have not been fixed, the accuracy of the state’s polls, one election does not appear to have any bearing on their accuracy to the next.
So as we enter the thick of the 2020 general election, we can be secure in the knowledge that the polls of the swing states are about the trustworthy on the polls can get — though that, of course, polls aren’t perfect. Even an error of 3 points can be the difference in close election.
