A lab for theoretical plurality
Hold more than one theory at a time.
Almost any dataset can be explained by several different theories. The usual practice is to pick one, fit it and report the answer as if the others had never been in the running.
I think we're entering a period where that stops being good enough, and where the people who do well are the ones who can keep several theories in play and weigh them properly. TheorySpace is where I work on the ideas and the tools for doing that.
The argument
Plurality is the normal case
A dataset is what's left behind after some process has run, and there's nearly always more than one process that could have left it. The candidate theories agree where you have data. They part ways where you don't, and that's usually the place you need an answer.
Bayes is how you keep the books
Bayesian inference was built for this. You put belief on the theories as well as on their parameters, let the data move that belief around, and carry whatever doubt is left into the prediction. If you aren't sure which theory is right, the forecast should say so.
Why it matters now
A new theory used to be expensive: one analyst, one model, weeks of work. An AI system can now draft ten candidate models in an afternoon, each with working code and a confident write-up. Producing theories is no longer the hard part. Comparing them, weighting them and deciding what to do when they disagree is.
It isn't easy
The true theory usually isn't on your list. Priors over theories are hard to defend, and the weights can swing on choices nobody looked at. The computation is heavy, and the person waiting for the result wants one number. I don't have clean answers to any of this, which is why I'm calling it a lab.
What this space is for
Ideas
Notes on what it means to work with several theories at once, and where the approach breaks down.
Tools
Methods tried on public data with the code posted, including the parts that didn't work.
Tutorials
Worked examples you can run yourself, starting with hierarchical models.
I'm Jay Sheldon. I trained in mathematics and I build Bayesian models for a living. This site is where I do the research side of that in the open.
Where I'm applying it
Bayesian models of choice and preference
Hierarchical Bayes for conjoint and MaxDiff, and the demand simulators built on top of them. This is my day job, so it's where I have the most opinions.
Analytic agents and their evaluation
Agents that pull evidence together, suggest hypotheses and run the analysis. What I mostly want to know is how you'd check their work.
Causal inference
Effects from observational data, with the assumptions written down and a look at how hard the answer leans on each one.
Adaptive experiments
Bandit and sequential designs next to plain balanced ones. Adaptive designs save sample, and I'd like to know what that costs you when it's time to estimate something.
Latest work
Hierarchical Bayesian models in PyMC, from specification to diagnostics
Partial pooling on a public dataset. Covers prior choice, sampling diagnostics, posterior predictive checks and what the model cannot tell you.
Applied work
I also do this for a living. I take on consulting projects, model reviews, agent builds and research collaborations, plus the occasional contract or fractional role. Paid work runs through Significance.ai, my consulting practice.