Asking
Only logged in members can reply and interact with the post.
Join SimilarWorlds for FREE »

Pollsters were wrong. AGAIN. Please explain why.




I can only think of two reasons why a predicted Kamala grand slam instead became a hat trick for Republicans (they now control the white house, senate, and house of representatives).

First possibility : millions of people lied to pollsters.

2nd possibility : those Pollsters lied to us.

Which do you think it was?
Nibblesnarf · 26-30, M
As the results finalize, it looks like swing state polling may have been a bit better than it was the last two presidential elections. But the familiar trend did emerge: polls underestimated Trump's performance almost across the board. At least some of the misses in polling averages were too much to credibly explain via random sampling error.

It should be noted that a few pollsters did very well. I'll give a shoutout to AtlasIntel, which polled the top battleground states with a freaky level of accuracy. (I'll also give a different kind of shoutout to Ann Selzer, whose final Iowa poll was a catastrophic miss.)

Now, to your question: polling is less of a science than many people believe, and there are numerous ways that things can go wrong. Here are a few.

1. Typically, pollsters don't publish their raw data. Once they've gathered their data, most will perform calculations that artificially make the demographics of their sample match the expected electorate. To put it another way, pollsters' assumptions are baked directly into their final published numbers. Done properly, this usually leads to better results. But it also makes it easy for bias to seep in (whether it be intentional or unintentional). Even an unbiased pollster can go astray if their assumptions about the electorate are wrong. I suspect (but don't know for sure) that this was part of the problem.

2. Poll "herding" seems to be a real phenomenon. Statistically speaking, outlier polls should happen sometimes. But nobody wants to actually be the outlier. So what does a pollster do if the election is near, and their result is far off of the polling average? Well, it seems that some pollsters in this situation muck with their numbers to make their result closer to the polling average - that is, closer to the "herd". The herding in the final few weeks was conspicuous enough that it was visible to the naked eye. Herding can cause the published polls to miss real shifts in the closing weeks. My guess is that herding didn't really create error this time. But it still means that some polls in the last few weeks weren't really giving us new information.

3. As Becky mentioned, it seems like Trump's voter base is just less likely on average to participate in polls. Pollsters seem to believe this was their main problem in 2020, and I think it likely accounted for part of the underestimation of Trump in 2024. It'll be interesting to see how polling does in the future, when Trump is no longer on the ballot.

4. When you see an election forecast, it's usually based on some method of aggregating polls together. This means that the forecast you see (e.g. "98% chance of a Hillary Clinton win") has already passed through two "filters": that of the pollster, and that of the aggregator. Averaging polls together is great because it should largely smooth out the effects of random sampling error. But the aggregator's modeling decisions and biases may have significant sway over their results. They might exclude or give lesser weight to pollsters they consider to be "low quality". (For example: fivethirtyeight chose to exclude Rasmussen from their 2024 model, for... somewhat questionable reasons, IIRC. And with the results in, Rasmussen appears to have been one of the most accurate pollsters of this election.)
Nibblesnarf · 26-30, M
@SusanInFlorida There was an experimental pollster in 2016 that published absolutely everything: their raw data, their weights and demographic assumptions, and details about their methodology. An interested reader could use this information to reproduce their published results. Unfortunately, this didn't become the norm in the industry.

Maybe pollsters keep some of this stuff close to the chest as a kind of "secret recipe", by which they hope to keep an edge over their competitors? I agree that it's shady. Poll results are reported with an air of scientific objectivity behind them, but in reality, they're often as much a product of pollster opinions/assumptions as they are of real world data.

I also remember an interesting exercise from the 2016 election cycle: someone gave the same raw data to four different pollsters, and asked them to process it as usual. There was relatively wide variance in the results, even though all four "polls" came from the same data.
SusanInFlorida · 31-35, F
@Nibblesnarf i dont need to see the "secret recipe". but it would be nice to see the ingredients, and if leading questions were used by the pollsters to drive the results in a manner favorable to whoever is paying the bills.
Nibblesnarf · 26-30, M
@SusanInFlorida I think big name pollsters usually do publish the exact questions they asked, so at least that level of transparency is often available. And the concern about leading questions (or other types of biased design) is more than valid. I've been polled a few times, and I did observe some unscrupulous behaviors.

With the other key ingredients, readers/watchers are usually left in the dark. I think some pollsters at least list the voter traits/demographics they choose to weight, but that's still very partial information. And there can be a devil in the hidden details if the poll sample is abnormal on a trait the pollster decided not to weight.
bowman81 · M
Third possibility the pollsters lied to themselves. They have routinely over-represented Democratic voters in their samples. They just keep on making the same sampling mistakes and shooting themselves in the foot.

I think there is some slanting toward giving those who are paying for the polls what they want to hear going on as well.
SusanInFlorida · 31-35, F
@bowman81 @bowman81 i'm leaning into the theory that they did this deliberately, to hoover up as much campaign money as possible until the last second. no candidate or campaign will pay for bad news.
bowman81 · M
@SusanInFlorida follow the money! 🥴
@bowman81 I have said for decades that polls are inaccurate and biased, they only select on average ot 1,500 people in specific areas to show what they wish to.
beckyromero · 36-40, F
There seems to be an element of mostly Trump supporters who simply won't respond to polls.

It's not that they lie, it's that they are not responding to begin with and are therefore unrepresented in the polling numbers and pollsters can't seem to get a handle on how to account for what seems to be a new phenomenon since 2016.
Avectoijesuismoi · 31-35
@SusanInFlorida Built in continuity
beckyromero · 36-40, F
@SusanInFlorida
people are already talking up Vance for 2028. premature.

Premature unless he is running as the incumbent.
SusanInFlorida · 31-35, F
@beckyromero you anticipated one of my upcoming top posts. It's not impossible that trump will have a heart attack, or a stroke. he seems quite agitated much of the time. and he is elderly . . . like 78? if he survives to the end of his term, he will be older than Biden is right now.
GerOttman · 61-69, M
Third, the pollsters are only polling in areas where people already agree with the desired results.
wildbill83 · 41-45, M
@GerOttman cities & college campuses...
GerOttman · 61-69, M
@wildbill83 Sure, it could be demographic selections as well.
SusanInFlorida · 31-35, F
@wildbill83 the college campus angle is a good one. this is a constituency with a LOT of free time on it's hands, and mistakenly believes that constantly foisting their opinions on others will change someone's mind.
This comment is hidden. Show Comment
SusanInFlorida · 31-35, F
@jshm2 i don't think kamala lost because of genitalia.

she lost because she's a low-IQ career politician with no legislative/policy successes to point to. that's why she had to run on "i'm not trump" as her platform
Patriot96 · 56-60, C
Pollsters had her up in Iowa by 3 percent last week. Trump stomped her
SusanInFlorida · 31-35, F
@Patriot96 i'm guessing this would be telephone polls. no way people in NY or California are taking planes to Iowa, and knocking on doors.
@SusanInFlorida There has been a massive influx of calis and east coasters moving into rural areas of iowa. Where I currently live in the middle of nowhere north central iowa, housing is now in high demand, coass have skyrocketed, traffic patterns now reflect Los Angeles and New York, accidents have increased tenfold in the past few years, as well as traffic itself. My theory is that they are invading to turn this state blue and change the political and cultural dsmographics left, like what they have done to my Beloved HomeState of Oregon and Washington State, which started in the 90's.
SusanInFlorida · 31-35, F
@NativePortlander1970 agreed. i read a WSJ article a while ago that theorized that Texas' jobs boom (luring employers and workers from California) was going to politically transform the state.

the new arrivals had no clue that their high tax/high regulatory acceptance are what makes california unacceptable. they will probably vote for the sorts of policies in Texas, once they settle in.
I don't actually care. Polls mean nothing AT ALL until people actually VOTE.
Avectoijesuismoi · 31-35
She should have listened to Donovan and tried to "Catch The Wind" she had a better chance of doing that
Musicman · 61-69, MVIP
Another option is Conservatives are afraid to say they support Trump for fear of reprisals if he lost. No need to have those armed IRS agents knocking on your door because you don't drink the blue Kool Aid.
SusanInFlorida · 31-35, F
@Musicman it would be homeland security or the secret service, not the IRS, right?
Musicman · 61-69, MVIP
@SusanInFlorida Actually the FBI or DOJ. The IRS are good too. Audit conservatives and drain their bank accounts with fines and penalties after they audit them. Even if the citizens are correct just paying accountants and lawyers will hurt them financially.
SusanInFlorida · 31-35, F
@Musicman i have an easy solution. disallow ALL non-profit designations. Every "organization" gets taxed:

- NPR/PBS
- the catholic church
- the red cross
- private colleges and universities

that will weed out the pretenders and non-viable players. Then the IRS can concentrate on making the survivors pay their fair share. taxes will probably go down on "real people"
They don't control the US House, yet.
@SusanInFlorida


These are all of the most contested counties that haven't declared yet, the repubs are in the lead, have You noticed they're all in blue states? I'll wager 100 to 1 Vegas odds they're trying to use fraudulent ballots to put the dems ahead.
SusanInFlorida · 31-35, F
@NativePortlander1970 i was watching CNN this morning. Their spin: "math is difficult. democrats might not get there"

considering they're already behind 12 seats, this is a remarkable semi-concession by CNN.
@SusanInFlorida I'm old enough to remember that when the system was fully analog we all knew by the next morning who was in control, once the system went digital and mail in state wide, that's when the issues truly started.

 
Post Comment