Resolution Process: On the first day of the three months prior to the US presidential election (i.e. 1 Sep, 1 Oct, 1 Nov) I will record in a spreadsheet the probabilities (rounded to nearest integer) each forecasting product has most recently assigned to either a Democrat or Republican winning in each Electoral College region. When the actual results are finalized post-election, I will take a Brier score based on the forecasts made for the candidate that ultimately won, and add up the total for each forecasting product.
The forecasting product with the lowest Brier score resolves YES, all other forecasting products resolve NO.
Criteria for Inclusion: For an election forecasting product to be eligible to resolve YES, the forecasting product must:
make forecasts for all or nearly all Electoral College regions (e.g. if a forecasting product doesn't track the special districts in Maine and Nebraska, but covers all 50 state-level races, that's fine; so this criterion excludes Metaculus, which only covers 19 battleground states)
use numeric probabilities (can’t just label races as “toss-up” or “likely”, needs to be like a percentage chance out of 100)
be public with its numbers (so Nate Silver's election model, which will be paywalled, won't qualify)
be a cohesive product (so PredictIt, which has some election markets, won’t qualify unless they put together something systematic and organized in the fashion of Polymarket or Manifold)
not be too obscure or small; so a lone non-famous analyst making a forecasting product would not merit inclusion
An election forecasting product that someone adds to this question that fails any of these criteria for inclusion will resolve NO and I won’t track its probabilities.
Below are links to the forecasting products/models that meet the criteria for inclusion so far:
Took a tally of probabilities from each of these election models/products on Saturday.
To get a sense of how the models compare right now, here are some election scenarios:
Blue Wave - Harris wins every swing state and takes likely Rep states like Florida and Ohio
Red Wave - Trump wins every swing state and cracks likely Dem states like Virginia and Minnesota
New England Edge - Trump wins a tight race with a couple surprise edge states in New England
Sunbelt Surprise - Harris loses the north and east but wins a tight race with surprise upset in Texas
2020 Repeat - Harris wins, with the same combination of states as Biden won in 2020
2016 Repeat - Trump wins, with the same combination of states as Trump won in 2016
I went and did the brier score calculation in an excel spreadsheet for each of these scenarios, given the odds each election product gave on 1 Sep:
If only accounting for the Sep 1 odds and these hand-picked scenarios, FiveThirtyEight would be the most accurate forecast if there's a Blue Wave in November, and The Economist the most accurate if there is a Red Wave.
The Hill/Decision Desk and Manifold are very nearly tied in brier scores if the exact same states are won by Harris in 2024 as Biden won in 2020, while Polymarket and The Economist are virtually tied if Trump wins the same configuration of states in 2024 as he did in 2016.
The Hill/Decision Desk would be the most accurate forecast if either of the two hypothetical edge cases came to pass, but if Trump won the north while Harris won Texas, both The Economist and Polymarket would be pretty nearly as accurate.