You've probably heard of evidence based policy. It's very in vogue these days with politicians... especially ones on the right. (Well, maybe not in America.) What you might not know is that academic policy analysts and senior technical advisers aren't fans of the phrase. To their view evidence based policy is a redundant expression: policy is always about evidence. Which rather begs the question of what policy is.
Policy is a slightly difficult term to nail down. Often we think of party manifestos as policies (elections should be about policy not personality) whereas we can quite reasonably say that policy is what governments do. I would suggest that policy is any conceivable intersection of politics with practical implementation, whether or not it's something "X wants to do" or "X is doing". Obviously the major concern is always going to be whether or not a policy should be done (is it broken?) and also whether or not it'll work (will it fix it?). A policy will answer both of those. The former is politics with evidence, the latter is a predictive exercise. The problem that I want to talk about is that most people don't know how to work with evidence. (Which therefore allows "evidence based policy" to masquerade as a statement which has meaning.)
Take, for example, the gender pay gap. This is a very commonly cited "problem". The political reasoning is that equal work should get equal pay. The thing is that this is usually measured in terms of average (or median) hourly earnings of men minus the same for women. That makes a lot of sense, right? Er, no. Not at all.
Anyone who's ever heard of regression will know that a regression model is the way to approach this exercise. But a regression model does not give a nice soundbite. And because most people don't know what regression is, no-one who wants to get anywhere is going to try and use a regression model. One, it takes longer to explain. Two, the populace are happy with the crude measure above.
The consequences of this are disastrous because it means the political reasoning we used to justify looking for a problem isn't relevant to our evidence. Hence we're not going to fix that. Even worse, this creates the wage gap myth myth because there are people who know (a) we haven't measured the wage gap but who then use that to, falsely, argue (b) therefore there's nothing broken.
So... what to do about this? Well, the answer is obvious... teach regression in schools... the concept, not the maths, anyway. But in a more general sense, the problem is that people are far too willing to accept movement from (a) to (b) as above. The reason being that people have very loose ideas about what burdens are.
There are people on NSG, for example, who think the burdens of arguments are subjective properties. They're not. If you're making a particular claim, it implies a particular burden. It might not be... probably is not... possible to objectively determine whether or not the burden of the claim is met by the evidence but the burden does have an objective existence conditional on the claim's existing. If we could get people to think more in terms of "if I'm saying X what do I require to be true" and "you're saying Y, therefore you need Z to be true" we'd be in a much better place. I suggest reframing subjects like English and Science to make this a more explicit part of what's going on.
So, what say ye, NSG? Are politicians able to get away with selling their party's positions as "evidence based policy" because their citizens are pretty poorly placed to evaluate policies? Could relatively minor changes to education have outsized impacts on the quality of policy debate?
