More Candidates, More Polling Error?
Earlier this week, I pointed out that Louisiana tends to have fewer polls in races for the U.S. Senate than expected, even after adjusting for the level of electoral competition. It is not at all clear why this is the case, but I have a guess: Louisiana elections are harder to poll accurately, so fewer pollsters are willing to try.
Polls in Louisiana miss the mark a bit more than in other states
For polls of statewide contests for governor or U.S. Senate seats taken during the final three weeks of the campaign, Louisiana ranks 37th among states in average error (that is, the absolute value of the difference between the actual margin of victory and the predicted margin of victory in polls). Polls in the state miss the actual margin by six point on average. Nationally, the average error in these races is about five points.
One point may not sound like much, but it means that polls here are 20 percent less accurate than the national average. By contrast, if Louisiana was 20 percent better than the national average, it would rank in the top ten for polling accuracy.
Why is there more error in Louisiana?
I suspect it has to do with our unique election system. Except for Louisiana, the polls included in this analysis are from general elections for the U.S. Senate and governor's office (and, in one instance, a general election runoff from Georgia). As with the presidential election, a general election is a competition between two major party nominees who won their party's primary and, perhaps, a smattering of minor-party or independent candidates. Because minor-party and independent candidates rarely get significant shares of the vote, the vast majority of these elections are for all intents and purposes contests between two candidates.
In Louisiana, however, we do not really have a general election because we do not have party primaries. Instead, everyone runs on the same ballot together in an initial round that we sometimes call a 'jungle primary' or an 'open primary'. If no one gets a majority in the first round, then the top two go to a runoff.
The only exceptions are the 2008 and 2010 U.S. Senate elections when Louisiana temporarily used the system common to other states (party primaries followed by a general election).
The polls in the data for Louisiana cover the 2008 and 2010 U.S. Senate general elections, three primaries for the U.S. Senate when we used our current election system (2002, 2004, and 2014), five gubernatorial primaries (1999, 2003, 2007, 2011, and 2015), two U.S. Senate runoffs (2002 and 2014), and two gubernatorial runoffs (2003 and 2015).
This means that the polls from Louisiana tend to have more candidates than the polls from most states. The average number of candidates on the ballot in Louisiana's races for governor and U.S. Senate since 1995 is 10.7. Many of these candidates are non-factors in the election, but an average of 3.2 candidates get at least ten percent of the vote. After dropping the 2008 and 2010 U.S. Senate elections, when Louisiana used the more traditional system found in other states, an average of 3.5 candidates won at least ten percent of the vote. That's one and half more candidates than is typical in every other state.
Why would more candidates make polling more difficult?
Having a bunch of candidates on the ballot makes things more difficult on pollsters in two ways. First, it makes the decision about whom to vote for more challenging. Voters glean a lot of information from their party identification. When faced with a decision between one Democrat and one Republican (and maybe a few other minor-party or independent candidates), it is easier to make a guess about who you think you'll end up voting for on Election Day than if you have to decide among several Democrats and several Republicans. This uncertainty will bleed into how they respond to polls.
It also complicates the survey process for measuring those decisions. More candidates means a lengthier list of response options, even when pollsters focus only on the most viable candidates. For example, one recent poll of the U.S. Senate race in Louisiana included eleven candidate names (of the 24 candidates running). Another included seven. Longer questions make it more difficult for respondents to accurately recall and process all the names read to them over the phone. Many respondents will satisfice; that is, they pick a name almost a random, pick a name they may have heard of recently before the interview, or simply say they do not know in order to move on through the survey. These types of answers are poor indicators of what they will end up doing come Election Day as a result.
It's probably because of Louisiana's primary elections
The average error in Louisiana polls differs across the stages of the election. In the primary phase, the average is error 6.7 points. Based on these polls alone, Louisiana would drop to 41st among states in polling accuracy. However, when looking just at polls from runoff elections, which have only two candidates, the average error shrinks to the national average at 5.1.
The challenge for this kind of analysis - and the reason it remains speculative - is that it relies on so few elections in Louisiana. The data include just five U.S. Senate races and five gubernatorial races from the state. The number of candidates is fixed to the specific election, so it is difficult to separate its impact from other features of a campaign.
One strategy to firm up the analysis is to look within elections and compare the average polling error in the primary to the average polling error in the runoff when the number of candidates drop to two. There are four elections that went to runoffs in the data. The table below shows hos the polling error changes across the two stages of the election.
In three of the four elections the magnitude of the average error drops. In only one does the error increase, and the difference is much smaller than the size of the drop in the other three elections.
It is no smoking gun, but it certainly suggests that more candidates on the ballot makes polling more difficult. Because Louisiana's unique election system means we usually have many more candidates on the ballot in our primaries than other states have in their general elections, it should not come as a shock that polls in the state tend to miss further from the mark.