How Accurate Have Senate Polls Been, And What Could That Mean For November?

The 2016 election was an off year for state-level polls of the presidential race. But for polls of U.S. Senate races, it was a perfectly ordinary year — better than average, even. And Senate polls in the 2018 election were even more accurate.

In the 2017-18 cycle, U.S. Senate polls conducted in the three weeks leading up to the general election had a weighted average error of 4.3 points. (As always, we define error as the absolute difference between a poll’s margin and the actual election’s margin between the top two candidates. For example, if a poll gave the Democratic candidate a lead of 2 percentage points but the Republican won the election by 4 points, that would be a 6-point error.) In the 2015-16 cycle, it was 4.9 points. And the long-term average, for all U.S. Senate elections since 1998, is 5.2 points.

In the grand scheme of things, that’s pretty accurate — not as true as polls of presidential general elections have historically been, but better than polls of presidential primaries or U.S. House races.

What’s more, Senate polling has been remarkably consistent in its accuracy from year to year. Since the 2001-02 cycle, the weighted average error of U.S. Senate polls conducted in the final 21 days before the election has never been lower than 4.2 points (its score in 2005-06) or higher than 5.4 points (2013-14). But that doesn’t mean there haven’t been any surprises in Senate races. The battle for the U.S. Senate, of course, is waged on a state-by-state basis, and while polling errors may average out to around 5 points across these races, they are higher in some states and lower in others.

You can see this clearly in the table below, which shows the weighted average error of U.S. Senate polls by cycle for states that had at least 10 polls conducted during the last 21 days of the campaign. (This threshold prevents us from showing states where the sample size of polls is too small to draw meaningful conclusions; it also focuses our attention on the states with the most competitive Senate races each cycle, as close races tend to attract polling interest.) The table also shows the weighted average statistical bias of Senate polls in each state, which tells us whether that polling error went in favor of Democrats or Republicans.

For instance, in the 2015-16 cycle, polls of most Senate races were pretty good, as 10 of the 12 states with enough polling to analyze had weighted average errors of 4.5 points or lower. In Missouri (2.8) and Nevada (3.0), the polls were downright excellent. But two states were trouble spots: Ohio (5.8) and Wisconsin (6.0). In both states, the polls overestimated Democrats, delivering the party a nasty surprise on election night.

In the 2017-18 cycle, polls of the U.S. Senate races in Missouri, Michigan and especially Arizona and Florida were quite accurate — all had weighted average errors of 4.3 or below. But the race in Alabama proved difficult to poll, with a weighted average error of 6.0. (The fact that it was an irregularly scheduled special election — and the Republican candidate was abandoned by his party after allegations of sexual misconduct emerged — probably contributed to the challenges.)

You can do the same exercise for other years, too. In 2013-14, Kansas, Iowa and Georgia saw the largest weighted average errors, each 6 points or greater. In 2005-06, Pennsylvania polls were particularly inaccurate (their weighted average statistical bias was 6 points toward Republicans), while Missouri polls were exceptionally good (their weighted average error was 2.4 points). In 2003-04, Pennsylvania and Oklahoma polls missed, while three other states’ were very reliable.

But much like the accuracy of swing-state polls in presidential elections, it’s difficult to predict where these trouble spots will be. Some states have very accurate Senate polls one year but particularly inaccurate ones in later cycles, or vice versa. Take Florida: In 2003-04, polls of the Sunshine State had a very low weighted average error of 2.5 points. In 2009-10, their error was a shaky 6.2 points.

That said, some states have an impressive track record of polling accuracy. Colorado, for example, appears six times in the table above, and only once has its polls’ weighted average error exceeded 3.5 points (it was 4.5 points in the 2001-02 elections — still pretty good!). We don’t want to make too much of this — it could just be that Colorado is on a polling hot streak, so to speak, and hasn’t yet hit a rough patch this century.

On the other hand, it could also be a vote of confidence in polls of Colorado’s 2020 U.S. Senate race, which will once again be closely watched. Other 2020 Senate battleground states that have historically produced pretty accurate Senate polls include Arizona (a weighted average error of 4.2 points in 2015-16 and 3.2 points in 2017-18) and North Carolina (4.7 in 2007-08, 2.8 in 2013-14 and 4.5 in 2015-16). On the other hand, polls of Georgia (6.1 in 2013-14) and Iowa (6.6 in 2013-14) have missed the mark significantly in recent elections — which doesn’t necessarily mean they will again in 2020, but we can’t rule it out.

We’ve already seen some pretty unbelievable — perhaps literally so — Senate polls this cycle. Those polls were obviously conducted several months before Election Day, so it’s not fair to compare them to the polls in this analysis. But if polls within the final 21 days of the campaign are still showing the same thing, it might be time to believe them. If the historical pattern holds, Senate polls should be pretty accurate once again in 2020 — at least in most states. Just remember there will probably be a state or two where they misfire.