Ep.40 When should we trust expert opinions about risk?

EPISODE NOTES To frame our conversation, we use one of Drew’s papers to discuss this issue. This paper, Forecasts or Fortune-Telling,was borne out of deep frustration.

Topics:

The two questions the paper sought to answer. What we mean by “expertise”. Forecasting. Determining the value of a given expert. Biases in reporting and researching. Super-forecasting. Wisdom of crowds. Better ways to get better answers. Why mathematical models aren’t as helpful as we think. Practical takeaways.

Quotes:

“Is it best to grab ten oncologists and take the average of their opinions?”

“But there is this possibility that there are some people who are better at managing their own cognitive biases than others. And it’s not to do with domain expertise, it’s to do with a particular set of skills that they call ‘super-forecasting’.”

“As far as I understand it, most organizations do not use complicated ways of combining expert opinions.”