Who is this guy and how serious should we take this information? This is by far the highest number I've seen for Trump so far.
He's quite a well known pollster. Up until recently he was responsible for Five Thirty Eight, but it got sold and he left.
He got the 2016 election wrong (71 Hilary, 28 trump)
He got the 2020 election right (89 Biden, 10 Trump)
Right and wrong are the incorrect terms here, but you get what I mean.
He didn’t get it wrong. He said the Clinton Trump election was a tight horse race, and Trump had one side of a four sided die.
The state by state data wasn’t far off.
Problem is, people don’t understand statistics.
If someone said Trump had over a 50% probability of winning in 2016, would that be wrong?
In statistical modeling you don’t really have right or wrong. You have a level of confidence in a model, a level of confidence in your data, and a statistical probability that an event will occur.
So if my model says RFK has a 98% probability of winning, then it is no more right or wrong than Silver's model?
If so, then probability would be useless. But it isn't useless. Probability is useful because it can make predictions that can be tested against reality.
In 2016, Silver's model predicted that Clinton would win. Which was wrong. He knew his model was wrong, because he adjusted his model after 2016. Why change something that is working properly?
You're conflating things.
Your model itself can be wrong, absolutely.
But for the person above to say Silver got something wrong because a lower probability event happened is a little silly. It'd be like flipping a coin heads side up twice in a row and saying you've disproved statistics because heads twice in a row should only happen 1/4 times.
Silver made a prediction. That's the deliverable. The prediction was wrong.
Nobody is saying that statistical theory was disproved. But it's impossible to tell whether Silver applied theory correctly, and it doesn't even matter. When a Boeing airplane loses a door, that doesn't disprove physics but it does mean that Boeing got something wrong.
but it does mean that Boeing got something wrong.
Comparing it to Boeing shows you still misunderstand probability. If his model predicts 4 separate elections where each underdog candidate had a 1 in 4 chance of winning. If only 1 of those underdog candidates wins, then the model is likely working. But when that candidate wins everyone will say "but he said it was only a 1 in 4 chance!". It's as dumb as people being surprised by rain when it says 25% chance of rain. As long as you only get rain 1/4 of the time with that prediction, then the model is working. Presidential elections are tricky because there are so few of them, they test their models against past data to verify they are working. But it's just probability, it's not saying this WILL happen, it's saying these are the odds at this snapshot in time.
Presidential elections are tricky because there is only one prediction.
Suppose your model says Trump has a 28% chance of winning in 2024, and mine says Trump has a 72% chance of winning in 2024.
There will only be one 2024 election. And suppose Trump loses it.
If that outcome doesn't tell us anything about the relative strength of our models, then what's the point of using a model at all? You might as well write a single line of code that spits out "50% Trump", it is equally useful.
The point of a model is to make a testable prediction. When the TV predicts a 25% chance of rain, that means that it will rain on one fourth of the days that they make such a prediction. It doesn't have to rain every time.
But Silver only makes a 2016 prediction once, and then he makes a new model for the next election. So he has exactly one chance to get it right.
His model has always been closer state to state, election to election than anyone else's, which is why people use his models. He is basically using the same model and tweaking it each time, you make it sound like he's starting over from scratch. When Trump won, none of the prediction models were predicting he would win, but his at least showed a fairly reasonable chance he could. His competitors were forecasting a much more likely Hillary win while he was showing that trump would win basically 3 out of 10 times. In terms of probability that's not a blowout prediction. His model was working better than competitors. Additionally, he basically predicted the battleground states within a half percentage iirc, that happened to be the difference between a win/loss in some states.
So he has exactly one chance to get it right.
You're saying it hitting one of those 3 of 10 is "getting it wrong", that's the problem with your understanding of probability. By saying that you're showing that you don't actually internalize the purpose of a predictive model forecast. It's not a magic wand, it's just a predictive tool. That tool is useful if you understand what it's really saying, instead of extrapolating something it absolutely is not saying. If something says something will happen 3 of 10 times, it happening is not evidence of an issue with the model. A flawless model with ideal inputs can still show a 3 of 10 chance and should hit in 30% of scenarios. Certainly because we have a limited number of elections it's hard to prove the model, but considering he has come closer than competitors, it certainly seems he knows what he is doing.
First, we need to distinguish Silver's state-by-state prediction with his "win probability". The former was pretty unremarkable in 2016, and I think we can agree that like everyone else he incorrectly predicted WI, MI, and PA.
However, his win probability is a different algorithm. It considers alternate scenarios, eg Trump wins Pennsylvania but loses Michigan. It somehow finds the probability of each scenario, and somehow calculates a total probability of winning. This does not correspond to one specific set of states that Silver thinks Trump will win. In 2016, it came up with a 28% probability of Trump winning.
You say that's not "getting it wrong". In that case, what would count as "getting it wrong"? Are we just supposed to have blind faith that Silver's probability calculation, and all its underlying assumptions, are correct? Because when the candidate with a higher win probability wins, that validates Silver's model. And when that candidate loses, that "is not evidence of an issue with the model". Heads I win, tails don't count.
If I built a model with different assumptions and came up with a 72% probability of Trump winning in 2016, that differs from Silver's result. Does that mean that I "got it wrong"? If neither of us got it wrong, what does it mean that Trump's probability of winning is simultaneously 28% and 72%?
And if there is no way for us to tell, even in retrospect, whether 28% is wrong or 72% is wrong or both are wrong, if both are equally compatible with the reality of Trump winning, then why pay any attention to those numbers at all?
I think you're missing the point of predictive modeling. It's probability of separate outcomes is built in. This isn't fortune telling, there is no crystal ball. Two predictive models can have different predictions and they both may have value. Just like separate meteorologists can have different forecasts, but predict accurately the same amount over time, all be it at different intervals. IIRC, the average meteorologist correctly predicts rain over 80% of the time. They are far over predicting by chance. But if you look at the forecast in more than one place you often get slightly different forecasts. They have different models and yet arrive at similar conclusions usually getting it mostly accurate. It's the same with political forecasts, they are only as valuable as your understanding of predictive modeling. If you think they are intended to mirror reality flawlessly, you will be sorely disappointed. That doesn't make the models "wrong", it doesn't make them "right" either. They are just models that usually predict a probable outcome.
I see what you're not getting! You are confusing giving the odds with making a prediction and those are very different.
Let's go back to the coin flips, maybe it'll make things more clear.
I or Silver might point out there's a 75% chance anything besides two heads in a row happening (which is accurate.) If, as will happen 1/4 times, two heads in a row does happen, does that somehow mean the odds I gave were wrong?
Same with Silver and the 2016 election.
I or Silver might point out there's a 75% chance anything besides two heads in a row happening (which is accurate.)
Is it?
Suppose I gave you two coins, which may or may not be weighted. You think they aren't, and I think they are weighted 2:1 towards heads. Your model predicts one head, and mine predicts two heads.
We toss and get two heads. Does that mean the odds I gave are right? Does it mean the odds you gave are wrong?
In the real world, your odds will depends on your priors, which you can never prove or disprove. If we were working with coins, then we could repeat the experiment and possibly update our priors.
But suppose we only have one chance to toss them, and after which they shatter. In that case, the model we use for the coins, weighted vs unweighted, is just a means to arrive at a prediction. The prediction can be right or wrong, but the internal workings of a one-shot model - including odds - are unfalsifiable. Same with Silver and the 2016 election.
The thing is, Nate Silver did not make a prediction about the 2016 race.
He said that Hilary had a higher chance of winning. He didn't say Hilary was going to win.
How can you falsify the claim "Clinton has a higher chance of winning"?
Alternately:
Silver said "Clinton has a higher chance of winning in 2016" whereas Michael Moore said "Trump has a higher chance of winning in 2016".
In hindsight, is one of these claims more valid than the other? Because if two contradictory claims are equally valid, then they are both meaningless.
You can't really falsify the claim “Clinton has a higher chance of winning”, at least the way Nate Silver models it. His model is based upon statistics, and he basically runs a bunch of simulations of the election. In more of these simulations, Clinton won, hence his claim. But we had exactly one actual election, and in the election, Trump won. Perhaps his model is just wrong, or perhaps the outcome matched one of the simulations in his model where Trump won. If we could somehow run the election hundreds of times (or observe what happened in hundreds of parallel universes) then maybe we could see if his model matched the outcome of a statistically significant number of election results. But nevertheless, Nate Silver had a model and statistics to back up his claim.
As for Michael Moore, I'm not sure exactly how he came up with his prediction, but I get the impression it was mostly a gut feeling based upon his observations of what was happening. Nevertheless, Michael Moore still could back up his statement by articulating why he was claiming that and the observations he had made.
Though one crucial difference is still the whole prediction thing. Michael Moore actually made a prediction of a Trump win. Whereas Nate Silver just stated that Clinton had a higher chance of winning, and once again that was not a prediction. So you're really comparing two different things here.
Silver made a prediction. That's the deliverable. The prediction was wrong.
Would you mind restating the prediction?
He predicted Clinton would win. That's the only reasonable prediction if her win probability was over 50%
If I say a roll of a 6-sided die has a >50% chance of landing on a number above 2, and after a single roll it lands on 2, was I wrong?
If anything, the problem is in the unfalsifiability of the claim.
Admittedly, 538 was pretty good about showing their work after. While individual events suffer from the unfalsifiability issue, 538 when Silver was around, did pretty good "how did we do for individual races/states" and compared their given odds to the actual results.
If you predict that a particular die will land on a 3-6 and it lands on a 2, then you were wrong. Predictions are occasionally wrong, that's unavoidable in the real world. Maybe the die wasn't fair and you should adjust your priors.
On the other hand, if you refuse to make a prediction but simply say a particular die has a >50% chance of landing above 2, then your claim is non-falsifiable. I could roll a hundred 1's in a row, and you could say that your probability is correct and I was just unlucky. That's why non-falsifiable claims are ultimately worthless.
Finally, if you claim that a theoretically fair die has a 2/3 probability of landing on 3-6 then you are correct, but that does not necessarily have anything to do with the real world of dice.
It's forecasting, not a prediction. If the weather forecast said there was a 28% chance of rain tomorrow and then tomorrow it rained would you say the forecast was wrong? You could say that if you want, but the point isn't to give a definitive prediction of the outcome (because that's not possible) it's to give you an idea of what to expect.
If there's a 28% chance of rain, it doesn't mean it's not going to rain, it actually means you might want to consider taking an umbrella with you because there's a significant probability it will rain. If a batter with a .280 batting average comes to the plate with 2 outs at the bottom of the ninth, that doesn't mean the game is over. If a politician has a 28% probability of winning an election, it's not a statement that the politician will definitely lose the election.
If the weather forecast said there was a 28% chance of rain tomorrow and then tomorrow it rained would you say the forecast was wrong?
Is it possible for the forecast to be wrong?
I think so. If you look at all the times the forecast predicts a 28% chance of rain, then it should rain on 28% of those days. If it rained, say, on half the days that the forecast gave a 28% chance of rain then the forecast would be wrong.
With Silver, the same principle applies. Clinton should win at least 50% of the 2016 elections where she has at least a 50% chance of winning. She didn't.
If Silver kept the same model over multiple elections, then we could look at his probabilities in finer detail. But he doesn't.
How about this:
Two people give the odds for the result of a coin flip of non-weighted coins.
Person A: Heads = 50%, Tails = 50%
Person B: Heads = 75%, Tails = 25%
The result of the coin flip ends up being Heads. Which person had the more accurate model? Did Person A get something wrong?
Person B's predicted outcome was closer to the truth.
Perhaps person A's prediction would improve if multiple trials were allowed. Perhaps their underlying assumptions are wrong (ie the coins are not unweighted).
Perhaps person A's prediction would improve
But in this hypothetical scenario of explicitly unweighted coins, Person A was entirely correct in the odds they gave. There's nothing to improve.
We are talking about testing a model in the real world. When you evaluate a model, you also evaluate the assumptions made by the model.
Let's consider a similar example. You are at a carnival. You hand a coin to a carny. He offers to pay you $100 if he flips heads. If he flips tails then you owe him $1.
You: The coin I gave him was unweighted so the odds are 50-50. This bet will pay off.
Your spouse: He's a carny. You're going to lose every time.
The coin is flipped, and it's tails. Who had the better prediction?
You maintain you had the better prediction because you know you gave him an unweighted coin. So you hand him a dollar to repeat the trial. You end up losing $50 without winning once.
You finally reconsider your assumptions. Perhaps the carny switched the coin. Perhaps the carny knows how to control the coin in the air. If it turns out that your assumptions were violated, then your spouse's original prediction was better than yours: you're going to lose every time.
Likewise, in order to evaluate Silver's model we need to consider the possibility that his model's many assumptions may contain flaws. Especially if his prediction, like yours in this example, differs sharply from real-world outcomes. If the assumptions are flawed, then the prediction could well be flawed too.
Probability is useful because it can make predictions that can be tested against reality.
Yes. But you'd have to run the test repeatedly and see if the outcome, i.e. Clinton winning, happens as often as the model predicts.
But we only get to run an election once. And there is no guarantee that the most likely outcome will happen on the first try.
If you can only run an election once, then how do you determine which of these two results is better (given than Trump won in 2016):
Clinton has a 72% probability of winning in 2016
Trump has a 72% probability of winning in 2016
You do it by comparing the state voting results to pre-election polling. If the pre-election polling said D+2 and your final result was R+1, then you have to look at your polls and individual polling firms and determine whether some bias is showing up in the results.
Is there selection bias or response bias? You might find that a set of polls is randomly wrong, or you might find that they're consistently wrong, adding 2 or 3 points in the direction of one party but generally tracking with results across time or geography. In that case, you determine a "house effect," in that either the people that firm is calling or the people who will talk to them lean 2 to 3 points more Democratic than the electorate.
All of this is explained on the website and it's kind of a pain to type out on a cellphone while on the toilet.
You are describing how to evaluate polling methods. And I agree: you do this by comparing an actual election outcome (eg statewide vote totals) to the results of your polling method.
But I am not talking about polling methods, I am talking about Silver's win probability. This is some proprietary method takes other people's polls as input (Silver is not a pollster) and outputs a number, like 28%. There are many possible ways to combine the poll results, giving different win probabilities. How do we evaluate Silver's method, separately from the polls?
I think the answer is basically the same: we compare it to an actual election outcome. Silver said Trump had a 28% win probability in 2016, which means he should win 28% of the time. The actual election outcome is that Trump won 100% of his 2016 elections. So as best as we can tell, Silver's win probability was quite inaccurate.
Now, if we could rerun the 2016 election maybe his estimate would look better over multiple trials. But we can't do that, all we can ever do is compare 28% to 100%.
Just for other people reading this thread, the following comments are an excellent case study in how an individual (the above poster) can be so confidently mistaken, even when other posters try to patiently correct them.
May we all be more respectful of our own ignorance.
But what if there's a 28% chance said poster is right?
He works for Peter Theil now, so I take everything he says with a huge grain of salt.
Because of Polymarket? Not everything is a conspiracy.
Because Peter Thiel is a right-wing billionaire piece of shit whose little bitch boy is J. D. Vance.
Okay. That's not in dispute. But partial ownership of a company doesn't make its employees your slaves. Especially when the company has nothing to do with ideological stuff.
Polling guru Nate Silver and his election prediction model gave Donald Trump a 63.8% chance of winning the electoral college in an update to his latest election forecast on Sunday, after a NYT-Siena College poll found Donald Trump leading Vice President Kamala Harris by 1 percentage point.
He's just a guy analizing the polls. The source is Fox News. He mentions in the article that tomorrow's debate could make that poll not matter.
Should you trust Nate or polls? They're fun but... Who is answering these polls? Who wants to answer them before even October?
So yeah take it seriously that a poll found that a lot of support for Trump exists. But it's just a moment of time for whoever they polled. Tomorrow's response will be a much better indication of any momentum.
analizing
I have shamed my family
Whi, is it not completeli spelled correctli?
It just seems strange because I don't think that many people are on the fence. Perhaps I'm crazy, but I feel most people know exactly who they're voting for already. Makes me wonder how valid this cross-section was that was used as the sample set. If it accurately represents the US, including undecided voters, then... 😮
but I feel most people know exactly who they’re voting for already
The cross-section of people you know are more politically off the fence than the entire nation. Those that aren't online at all are also more undecided and less likely to interact with you.
I listen to those news things that interview people on the street and I'm amazed at how many are uninformed and can go either way.
There's a Trump undercount in polling: Trump voters don't trust "MSM" and therefore don't answer calls from pollsters, or are embarrassed to admit they will vote for him.
Same goes for asking random people on the street.
There's also an undercount of young people who don't answer the phone.
I don’t know many people (boomers and younger) who answer the phone from numbers they do not recognize. I would like to imagine that the people who do answer strange numbers tend to be out of touch. Bias in the polls to fools or the lucky who are not spammed ?
And an undercount of women who are telling their husbands and anyone else who asks that they'll be voting Trump, but will actually vote for Harris when the time comes. And an undercount of bro-ski-s who claim to support Harris, but secretly hate the fact that they can't get a 'female' that will cater to their every whim and will vote Trump because he'll increase oppression of women. And an undercount of cat ladies.... etc. Most "high quality" models at least attempt to mitigate these over and undercounts, which definitely skews results, and why poll aggregators are important. It helps to eliminate biases in polling types. There's really only ONE poll that matters. VOTE! BRING YOUR FRIENDS!
Pollsters are compensating for that undercount of unlikely voters. 2016 they were low, 2020 still low but pretty close. They will have scaled it up to be more accurate this go around.
Except there's a few snags there. In between the 2020 election and now, there was an insurrection, Roe v. Wade was overturned, Trump was convicted of crimes and indicted for many more. These are things that a statistical process can't really account for when putting weight on how likely a respondant is to actually vote.
Trump lost in 2020. Do all of these events incentivize more people will turn out for him this time than in the last election? Or will less people turn out for him?
Every time something unprecedented happens it negatively impacts the ability for a scientific statistical process to predict the outcome. Science can't predict things there's no model for, and how do you can't have a model for something you haven't seen before. And a hell of a lot of unprecedented shit has happened. Maybe next time a convicted felon that tried to overthrow democracy runs in an election there can be accurate polling, but it's not going to be the case in this election.
There really is no way to know what will happen on election day. So there's else to do other than maximum effort until election day.
The issue isn't really people on the fence for Trump or Harris but mainly with generating turnout. After Biden's poor debate performance, people didn't change their mind and decide to vote for Trump, they became apathetic and maybe wouldn't show up to vote.
Harris doesn't need to persuade people to abandon Trump, she needs to get people excited to show up to vote.
He's not polling, he is aggregating all of the polls into a prediction model. Either way it is just a snapshot in time.
The key to doing statistics well is to make sure you aren't changing the results with any bias. This means enough samples, a good selection of samples, and weighing the outcome correctly. Even honest polling in pre-election is hard to get right, and because of that it's easy to make things lean towards results if you want to get certain results, or or getting paid to get those results.
There's only one poll that matters, and that poll should include as large of a sample as possible, and be counted correctly. Even though some will try to prevent that from happening.
It's a chance of winning, not a poll, so 64% is high but not insane. Silver is serious and it's a decent model. Knowing the model there's a pretty good chance this is a high point for Trump but it's not like he's pulling this out of nowhere, he has had similar models every election cycle since like 2008.
If it's overstaying Trump it's because his model is interpreting the data incorrectly because of the weirdness of this election cycle. I personally think that is likely the case here.
That used to be true, but in recent years he has gotten a lot more conservative, so I personally take his predictions with a huge grain of salt.
Yes, I kinda agree. Let's see his model's brier score in November :)
This isn't a poll. That's why the number is so high. His model is also automatically depressing Harris' numbers because of the convention right now. (It did the same thing to Trump after his convention)
Nate has been upfront in his newsletters about the factor dropping off the model after today, but then it's also the debate. Things are likely to be far more clear going into the weekend because we'll have post debate polling being published and no more convention adjustments.
You shouldn't take it seriously. The 24-hour news cycle depends on data like this. It just doesn't tell us anything.
Their models have been really accurate for the last several election cycles. They’re part of fivethirtyeight.com
No, Nate is not part of 538 anymore. He now works for a crypto betting website partly owned by Peter Thiel.
I'll let you decide how neutral that makes him.
Peter Thiel, the same guy who sold Republicans on JD the couch fucker Vance
While that is also my pet name for JD, keep in mind it is aspirational, not historic.
He's a degen gambler who admits in his book he was gambling up to $10k a day while running 538... It never made him go "huh maybe I fucked my employees because I'm a degen gambler."
Boo, what does this mean for 538?
Nate is not with 538 anymore. Disney didn’t renew his contract. However, he got to keep the model that he developed and publishes it for his newsletter subscribers. 538 had to rebuild their model from scratch this year with G Elliot Morris.
Now Nate hosts the podcast Risky Business with Maria Konnikova. The psychologist who became a professional poker player while researching a book. It’s pretty good.
Who is this guy and how serious should we take this information?
Well, he did predict Clinton would win in 2016 so there's that.
He's renowned for being wrong for several previous elections
All prediction models only give you odds, not flawless accuracy. He has been closer in every election than most everyone else in the prediction market.
Who is Nate Silver? Really?
Hey man there is a mountain of people who don't know things and are scared to ask. learning is always a good thing
Social media isn't a search engine. If an article is referring to someone by name in the title, they almost certainly have a Wikipedia page the questioner could read rather than requesting random strangers on a message board provide answers for them (in the form of multiple answers of varying bias and accuracy).
Wanting to learn isn't the problem, it's not spending the tiniest bit of personal effort before requesting service from other people.
Or he could have a conversation on a conversation forum.
Perish the thought!
Yeah. I think we take our easy navigation for granted sometimes. Like... I can get most information pretty quickly and not have a lot of trouble discerning what I need to do to get that information.
But not everyone is as "natural" at surfing. Maybe they have trouble putting things in perspective, they don't know how to use a tool like Wikipedia, or even - maybe they just don't like researching.
I'm so glad we have people that are great at keeping up with everything. But we have to remember that presenting and teaching information accurately and helpfully is a skill that we need desperately.
Who is this guy and how serious should we take this information? This is by far the highest number I've seen for Trump so far.
He's quite a well known pollster. Up until recently he was responsible for Five Thirty Eight, but it got sold and he left.
He got the 2016 election wrong (71 Hilary, 28 trump) He got the 2020 election right (89 Biden, 10 Trump)
Right and wrong are the incorrect terms here, but you get what I mean.
He didn’t get it wrong. He said the Clinton Trump election was a tight horse race, and Trump had one side of a four sided die.
The state by state data wasn’t far off.
Problem is, people don’t understand statistics.
If someone said Trump had over a 50% probability of winning in 2016, would that be wrong?
In statistical modeling you don’t really have right or wrong. You have a level of confidence in a model, a level of confidence in your data, and a statistical probability that an event will occur.
So if my model says RFK has a 98% probability of winning, then it is no more right or wrong than Silver's model?
If so, then probability would be useless. But it isn't useless. Probability is useful because it can make predictions that can be tested against reality.
In 2016, Silver's model predicted that Clinton would win. Which was wrong. He knew his model was wrong, because he adjusted his model after 2016. Why change something that is working properly?
You're conflating things.
Your model itself can be wrong, absolutely.
But for the person above to say Silver got something wrong because a lower probability event happened is a little silly. It'd be like flipping a coin heads side up twice in a row and saying you've disproved statistics because heads twice in a row should only happen 1/4 times.
Silver made a prediction. That's the deliverable. The prediction was wrong.
Nobody is saying that statistical theory was disproved. But it's impossible to tell whether Silver applied theory correctly, and it doesn't even matter. When a Boeing airplane loses a door, that doesn't disprove physics but it does mean that Boeing got something wrong.
Comparing it to Boeing shows you still misunderstand probability. If his model predicts 4 separate elections where each underdog candidate had a 1 in 4 chance of winning. If only 1 of those underdog candidates wins, then the model is likely working. But when that candidate wins everyone will say "but he said it was only a 1 in 4 chance!". It's as dumb as people being surprised by rain when it says 25% chance of rain. As long as you only get rain 1/4 of the time with that prediction, then the model is working. Presidential elections are tricky because there are so few of them, they test their models against past data to verify they are working. But it's just probability, it's not saying this WILL happen, it's saying these are the odds at this snapshot in time.
Presidential elections are tricky because there is only one prediction.
Suppose your model says Trump has a 28% chance of winning in 2024, and mine says Trump has a 72% chance of winning in 2024.
There will only be one 2024 election. And suppose Trump loses it.
If that outcome doesn't tell us anything about the relative strength of our models, then what's the point of using a model at all? You might as well write a single line of code that spits out "50% Trump", it is equally useful.
The point of a model is to make a testable prediction. When the TV predicts a 25% chance of rain, that means that it will rain on one fourth of the days that they make such a prediction. It doesn't have to rain every time.
But Silver only makes a 2016 prediction once, and then he makes a new model for the next election. So he has exactly one chance to get it right.
His model has always been closer state to state, election to election than anyone else's, which is why people use his models. He is basically using the same model and tweaking it each time, you make it sound like he's starting over from scratch. When Trump won, none of the prediction models were predicting he would win, but his at least showed a fairly reasonable chance he could. His competitors were forecasting a much more likely Hillary win while he was showing that trump would win basically 3 out of 10 times. In terms of probability that's not a blowout prediction. His model was working better than competitors. Additionally, he basically predicted the battleground states within a half percentage iirc, that happened to be the difference between a win/loss in some states.
You're saying it hitting one of those 3 of 10 is "getting it wrong", that's the problem with your understanding of probability. By saying that you're showing that you don't actually internalize the purpose of a predictive model forecast. It's not a magic wand, it's just a predictive tool. That tool is useful if you understand what it's really saying, instead of extrapolating something it absolutely is not saying. If something says something will happen 3 of 10 times, it happening is not evidence of an issue with the model. A flawless model with ideal inputs can still show a 3 of 10 chance and should hit in 30% of scenarios. Certainly because we have a limited number of elections it's hard to prove the model, but considering he has come closer than competitors, it certainly seems he knows what he is doing.
First, we need to distinguish Silver's state-by-state prediction with his "win probability". The former was pretty unremarkable in 2016, and I think we can agree that like everyone else he incorrectly predicted WI, MI, and PA.
However, his win probability is a different algorithm. It considers alternate scenarios, eg Trump wins Pennsylvania but loses Michigan. It somehow finds the probability of each scenario, and somehow calculates a total probability of winning. This does not correspond to one specific set of states that Silver thinks Trump will win. In 2016, it came up with a 28% probability of Trump winning.
You say that's not "getting it wrong". In that case, what would count as "getting it wrong"? Are we just supposed to have blind faith that Silver's probability calculation, and all its underlying assumptions, are correct? Because when the candidate with a higher win probability wins, that validates Silver's model. And when that candidate loses, that "is not evidence of an issue with the model". Heads I win, tails don't count.
If I built a model with different assumptions and came up with a 72% probability of Trump winning in 2016, that differs from Silver's result. Does that mean that I "got it wrong"? If neither of us got it wrong, what does it mean that Trump's probability of winning is simultaneously 28% and 72%?
And if there is no way for us to tell, even in retrospect, whether 28% is wrong or 72% is wrong or both are wrong, if both are equally compatible with the reality of Trump winning, then why pay any attention to those numbers at all?
I think you're missing the point of predictive modeling. It's probability of separate outcomes is built in. This isn't fortune telling, there is no crystal ball. Two predictive models can have different predictions and they both may have value. Just like separate meteorologists can have different forecasts, but predict accurately the same amount over time, all be it at different intervals. IIRC, the average meteorologist correctly predicts rain over 80% of the time. They are far over predicting by chance. But if you look at the forecast in more than one place you often get slightly different forecasts. They have different models and yet arrive at similar conclusions usually getting it mostly accurate. It's the same with political forecasts, they are only as valuable as your understanding of predictive modeling. If you think they are intended to mirror reality flawlessly, you will be sorely disappointed. That doesn't make the models "wrong", it doesn't make them "right" either. They are just models that usually predict a probable outcome.
I see what you're not getting! You are confusing giving the odds with making a prediction and those are very different.
Let's go back to the coin flips, maybe it'll make things more clear.
I or Silver might point out there's a 75% chance anything besides two heads in a row happening (which is accurate.) If, as will happen 1/4 times, two heads in a row does happen, does that somehow mean the odds I gave were wrong?
Same with Silver and the 2016 election.
Is it?
Suppose I gave you two coins, which may or may not be weighted. You think they aren't, and I think they are weighted 2:1 towards heads. Your model predicts one head, and mine predicts two heads.
We toss and get two heads. Does that mean the odds I gave are right? Does it mean the odds you gave are wrong?
In the real world, your odds will depends on your priors, which you can never prove or disprove. If we were working with coins, then we could repeat the experiment and possibly update our priors.
But suppose we only have one chance to toss them, and after which they shatter. In that case, the model we use for the coins, weighted vs unweighted, is just a means to arrive at a prediction. The prediction can be right or wrong, but the internal workings of a one-shot model - including odds - are unfalsifiable. Same with Silver and the 2016 election.
The thing is, Nate Silver did not make a prediction about the 2016 race.
He said that Hilary had a higher chance of winning. He didn't say Hilary was going to win.
How can you falsify the claim "Clinton has a higher chance of winning"?
Alternately:
Silver said "Clinton has a higher chance of winning in 2016" whereas Michael Moore said "Trump has a higher chance of winning in 2016".
In hindsight, is one of these claims more valid than the other? Because if two contradictory claims are equally valid, then they are both meaningless.
You can't really falsify the claim “Clinton has a higher chance of winning”, at least the way Nate Silver models it. His model is based upon statistics, and he basically runs a bunch of simulations of the election. In more of these simulations, Clinton won, hence his claim. But we had exactly one actual election, and in the election, Trump won. Perhaps his model is just wrong, or perhaps the outcome matched one of the simulations in his model where Trump won. If we could somehow run the election hundreds of times (or observe what happened in hundreds of parallel universes) then maybe we could see if his model matched the outcome of a statistically significant number of election results. But nevertheless, Nate Silver had a model and statistics to back up his claim.
As for Michael Moore, I'm not sure exactly how he came up with his prediction, but I get the impression it was mostly a gut feeling based upon his observations of what was happening. Nevertheless, Michael Moore still could back up his statement by articulating why he was claiming that and the observations he had made.
Though one crucial difference is still the whole prediction thing. Michael Moore actually made a prediction of a Trump win. Whereas Nate Silver just stated that Clinton had a higher chance of winning, and once again that was not a prediction. So you're really comparing two different things here.
Would you mind restating the prediction?
He predicted Clinton would win. That's the only reasonable prediction if her win probability was over 50%
If I say a roll of a 6-sided die has a >50% chance of landing on a number above 2, and after a single roll it lands on 2, was I wrong?
If anything, the problem is in the unfalsifiability of the claim.
Admittedly, 538 was pretty good about showing their work after. While individual events suffer from the unfalsifiability issue, 538 when Silver was around, did pretty good "how did we do for individual races/states" and compared their given odds to the actual results.
If you predict that a particular die will land on a 3-6 and it lands on a 2, then you were wrong. Predictions are occasionally wrong, that's unavoidable in the real world. Maybe the die wasn't fair and you should adjust your priors.
On the other hand, if you refuse to make a prediction but simply say a particular die has a >50% chance of landing above 2, then your claim is non-falsifiable. I could roll a hundred 1's in a row, and you could say that your probability is correct and I was just unlucky. That's why non-falsifiable claims are ultimately worthless.
Finally, if you claim that a theoretically fair die has a 2/3 probability of landing on 3-6 then you are correct, but that does not necessarily have anything to do with the real world of dice.
It's forecasting, not a prediction. If the weather forecast said there was a 28% chance of rain tomorrow and then tomorrow it rained would you say the forecast was wrong? You could say that if you want, but the point isn't to give a definitive prediction of the outcome (because that's not possible) it's to give you an idea of what to expect.
If there's a 28% chance of rain, it doesn't mean it's not going to rain, it actually means you might want to consider taking an umbrella with you because there's a significant probability it will rain. If a batter with a .280 batting average comes to the plate with 2 outs at the bottom of the ninth, that doesn't mean the game is over. If a politician has a 28% probability of winning an election, it's not a statement that the politician will definitely lose the election.
Is it possible for the forecast to be wrong?
I think so. If you look at all the times the forecast predicts a 28% chance of rain, then it should rain on 28% of those days. If it rained, say, on half the days that the forecast gave a 28% chance of rain then the forecast would be wrong.
With Silver, the same principle applies. Clinton should win at least 50% of the 2016 elections where she has at least a 50% chance of winning. She didn't.
If Silver kept the same model over multiple elections, then we could look at his probabilities in finer detail. But he doesn't.
How about this:
Two people give the odds for the result of a coin flip of non-weighted coins.
Person A: Heads = 50%, Tails = 50%
Person B: Heads = 75%, Tails = 25%
The result of the coin flip ends up being Heads. Which person had the more accurate model? Did Person A get something wrong?
Person B's predicted outcome was closer to the truth.
Perhaps person A's prediction would improve if multiple trials were allowed. Perhaps their underlying assumptions are wrong (ie the coins are not unweighted).
But in this hypothetical scenario of explicitly unweighted coins, Person A was entirely correct in the odds they gave. There's nothing to improve.
We are talking about testing a model in the real world. When you evaluate a model, you also evaluate the assumptions made by the model.
Let's consider a similar example. You are at a carnival. You hand a coin to a carny. He offers to pay you $100 if he flips heads. If he flips tails then you owe him $1.
You: The coin I gave him was unweighted so the odds are 50-50. This bet will pay off.
Your spouse: He's a carny. You're going to lose every time.
The coin is flipped, and it's tails. Who had the better prediction?
You maintain you had the better prediction because you know you gave him an unweighted coin. So you hand him a dollar to repeat the trial. You end up losing $50 without winning once.
You finally reconsider your assumptions. Perhaps the carny switched the coin. Perhaps the carny knows how to control the coin in the air. If it turns out that your assumptions were violated, then your spouse's original prediction was better than yours: you're going to lose every time.
Likewise, in order to evaluate Silver's model we need to consider the possibility that his model's many assumptions may contain flaws. Especially if his prediction, like yours in this example, differs sharply from real-world outcomes. If the assumptions are flawed, then the prediction could well be flawed too.
Yes. But you'd have to run the test repeatedly and see if the outcome, i.e. Clinton winning, happens as often as the model predicts.
But we only get to run an election once. And there is no guarantee that the most likely outcome will happen on the first try.
If you can only run an election once, then how do you determine which of these two results is better (given than Trump won in 2016):
You do it by comparing the state voting results to pre-election polling. If the pre-election polling said D+2 and your final result was R+1, then you have to look at your polls and individual polling firms and determine whether some bias is showing up in the results.
Is there selection bias or response bias? You might find that a set of polls is randomly wrong, or you might find that they're consistently wrong, adding 2 or 3 points in the direction of one party but generally tracking with results across time or geography. In that case, you determine a "house effect," in that either the people that firm is calling or the people who will talk to them lean 2 to 3 points more Democratic than the electorate.
All of this is explained on the website and it's kind of a pain to type out on a cellphone while on the toilet.
You are describing how to evaluate polling methods. And I agree: you do this by comparing an actual election outcome (eg statewide vote totals) to the results of your polling method.
But I am not talking about polling methods, I am talking about Silver's win probability. This is some proprietary method takes other people's polls as input (Silver is not a pollster) and outputs a number, like 28%. There are many possible ways to combine the poll results, giving different win probabilities. How do we evaluate Silver's method, separately from the polls?
I think the answer is basically the same: we compare it to an actual election outcome. Silver said Trump had a 28% win probability in 2016, which means he should win 28% of the time. The actual election outcome is that Trump won 100% of his 2016 elections. So as best as we can tell, Silver's win probability was quite inaccurate.
Now, if we could rerun the 2016 election maybe his estimate would look better over multiple trials. But we can't do that, all we can ever do is compare 28% to 100%.
Just for other people reading this thread, the following comments are an excellent case study in how an individual (the above poster) can be so confidently mistaken, even when other posters try to patiently correct them.
May we all be more respectful of our own ignorance.
But what if there's a 28% chance said poster is right?
He works for Peter Theil now, so I take everything he says with a huge grain of salt.
Because of Polymarket? Not everything is a conspiracy.
Because Peter Thiel is a right-wing billionaire piece of shit whose little bitch boy is J. D. Vance.
Okay. That's not in dispute. But partial ownership of a company doesn't make its employees your slaves. Especially when the company has nothing to do with ideological stuff.
He's just a guy analizing the polls. The source is Fox News. He mentions in the article that tomorrow's debate could make that poll not matter.
Should you trust Nate or polls? They're fun but... Who is answering these polls? Who wants to answer them before even October?
So yeah take it seriously that a poll found that a lot of support for Trump exists. But it's just a moment of time for whoever they polled. Tomorrow's response will be a much better indication of any momentum.
I have shamed my family
Whi, is it not completeli spelled correctli?
It just seems strange because I don't think that many people are on the fence. Perhaps I'm crazy, but I feel most people know exactly who they're voting for already. Makes me wonder how valid this cross-section was that was used as the sample set. If it accurately represents the US, including undecided voters, then... 😮
The cross-section of people you know are more politically off the fence than the entire nation. Those that aren't online at all are also more undecided and less likely to interact with you.
I listen to those news things that interview people on the street and I'm amazed at how many are uninformed and can go either way.
There's a Trump undercount in polling: Trump voters don't trust "MSM" and therefore don't answer calls from pollsters, or are embarrassed to admit they will vote for him.
Same goes for asking random people on the street.
There's also an undercount of young people who don't answer the phone.
I don’t know many people (boomers and younger) who answer the phone from numbers they do not recognize. I would like to imagine that the people who do answer strange numbers tend to be out of touch. Bias in the polls to fools or the lucky who are not spammed ?
And an undercount of women who are telling their husbands and anyone else who asks that they'll be voting Trump, but will actually vote for Harris when the time comes. And an undercount of bro-ski-s who claim to support Harris, but secretly hate the fact that they can't get a 'female' that will cater to their every whim and will vote Trump because he'll increase oppression of women. And an undercount of cat ladies.... etc. Most "high quality" models at least attempt to mitigate these over and undercounts, which definitely skews results, and why poll aggregators are important. It helps to eliminate biases in polling types. There's really only ONE poll that matters. VOTE! BRING YOUR FRIENDS!
Pollsters are compensating for that undercount of unlikely voters. 2016 they were low, 2020 still low but pretty close. They will have scaled it up to be more accurate this go around.
Except there's a few snags there. In between the 2020 election and now, there was an insurrection, Roe v. Wade was overturned, Trump was convicted of crimes and indicted for many more. These are things that a statistical process can't really account for when putting weight on how likely a respondant is to actually vote.
Trump lost in 2020. Do all of these events incentivize more people will turn out for him this time than in the last election? Or will less people turn out for him?
Every time something unprecedented happens it negatively impacts the ability for a scientific statistical process to predict the outcome. Science can't predict things there's no model for, and how do you can't have a model for something you haven't seen before. And a hell of a lot of unprecedented shit has happened. Maybe next time a convicted felon that tried to overthrow democracy runs in an election there can be accurate polling, but it's not going to be the case in this election.
There really is no way to know what will happen on election day. So there's else to do other than maximum effort until election day.
The issue isn't really people on the fence for Trump or Harris but mainly with generating turnout. After Biden's poor debate performance, people didn't change their mind and decide to vote for Trump, they became apathetic and maybe wouldn't show up to vote.
Harris doesn't need to persuade people to abandon Trump, she needs to get people excited to show up to vote.
He's not polling, he is aggregating all of the polls into a prediction model. Either way it is just a snapshot in time.
The key to doing statistics well is to make sure you aren't changing the results with any bias. This means enough samples, a good selection of samples, and weighing the outcome correctly. Even honest polling in pre-election is hard to get right, and because of that it's easy to make things lean towards results if you want to get certain results, or or getting paid to get those results.
There's only one poll that matters, and that poll should include as large of a sample as possible, and be counted correctly. Even though some will try to prevent that from happening.
It's a chance of winning, not a poll, so 64% is high but not insane. Silver is serious and it's a decent model. Knowing the model there's a pretty good chance this is a high point for Trump but it's not like he's pulling this out of nowhere, he has had similar models every election cycle since like 2008.
If it's overstaying Trump it's because his model is interpreting the data incorrectly because of the weirdness of this election cycle. I personally think that is likely the case here.
This quote sums it up:
𝘾𝙝𝙖𝙨𝙚
@chsrdn
In the future we won't elect presidents. We'll have a primary, then Nate Silver will go into a spice trance and pick the winner.
3:41 AM · 7 nov 2012
That used to be true, but in recent years he has gotten a lot more conservative, so I personally take his predictions with a huge grain of salt.
Yes, I kinda agree. Let's see his model's brier score in November :)
This isn't a poll. That's why the number is so high. His model is also automatically depressing Harris' numbers because of the convention right now. (It did the same thing to Trump after his convention)
Nate has been upfront in his newsletters about the factor dropping off the model after today, but then it's also the debate. Things are likely to be far more clear going into the weekend because we'll have post debate polling being published and no more convention adjustments.
You shouldn't take it seriously. The 24-hour news cycle depends on data like this. It just doesn't tell us anything.
Their models have been really accurate for the last several election cycles. They’re part of fivethirtyeight.com
No, Nate is not part of 538 anymore. He now works for a crypto betting website partly owned by Peter Thiel.
I'll let you decide how neutral that makes him.
Peter Thiel, the same guy who sold Republicans on JD the couch fucker Vance
While that is also my pet name for JD, keep in mind it is aspirational, not historic.
He's a degen gambler who admits in his book he was gambling up to $10k a day while running 538... It never made him go "huh maybe I fucked my employees because I'm a degen gambler."
Boo, what does this mean for 538?
Nate is not with 538 anymore. Disney didn’t renew his contract. However, he got to keep the model that he developed and publishes it for his newsletter subscribers. 538 had to rebuild their model from scratch this year with G Elliot Morris.
Now Nate hosts the podcast Risky Business with Maria Konnikova. The psychologist who became a professional poker player while researching a book. It’s pretty good.
Well, he did predict Clinton would win in 2016 so there's that.
He's renowned for being wrong for several previous elections
All prediction models only give you odds, not flawless accuracy. He has been closer in every election than most everyone else in the prediction market.
Who is Nate Silver? Really?
Hey man there is a mountain of people who don't know things and are scared to ask. learning is always a good thing
XKCD 1053
Social media isn't a search engine. If an article is referring to someone by name in the title, they almost certainly have a Wikipedia page the questioner could read rather than requesting random strangers on a message board provide answers for them (in the form of multiple answers of varying bias and accuracy).
Wanting to learn isn't the problem, it's not spending the tiniest bit of personal effort before requesting service from other people.
Or he could have a conversation on a conversation forum.
Perish the thought!
Yeah. I think we take our easy navigation for granted sometimes. Like... I can get most information pretty quickly and not have a lot of trouble discerning what I need to do to get that information.
But not everyone is as "natural" at surfing. Maybe they have trouble putting things in perspective, they don't know how to use a tool like Wikipedia, or even - maybe they just don't like researching.
I'm so glad we have people that are great at keeping up with everything. But we have to remember that presenting and teaching information accurately and helpfully is a skill that we need desperately.