Why this matters
Everyone has opinions about the future; almost nobody keeps score. Pythia is a personal forecasting workbench built on a simple discipline: write the probability down, update it with reasons as evidence arrives, and let a Brier score tell you how good your judgment actually is. Prediction markets (Kalshi, Polymarket) supply real questions with real resolution, so there’s no grading yourself on vibes.
What exists now
Phase 1, an internal workbench, is being built:
- Market dashboard pulling live prediction markets, with filtering and search
- Forecasts as update timelines, not one-shot guesses: every revision needs a probability and a reason, and nothing can be edited or deleted after the fact
- Brier scoring and calibration metrics at resolution
- Deliberate cognitive gates: the market price stays hidden until you’ve entered a base rate (outside view first), and you can’t submit without writing the counter-argument
- AI research briefs, category-aware, to speed up the outside view
What I’m looking for
Nothing formal yet; this is a workbench being built for one user (me). But if you’re into forecasting, calibration, or decision science, I’d genuinely enjoy comparing notes.
Open questions
- Does forced structure (base-rate gate, counter-argument gate) measurably improve calibration, or just add friction?
- Is this forever a personal tool, or does “forecasting gym” have an audience?