The problem with polling on policy
The problem with polling on policy
For example, in the last Opinium/Observer poll we asked about the government?s free schools policy and earlier we?ve asked about David Cameron?s marriage tax breaks, proposed changes to the Labour party?s links to the trade unions as well as various proposals from the ?Alternative Queen?s Speech? by a group of right wing Conservative MPs.
The difficulty with asking any questions on policy is that you need to explain it to respondents who, quite understandably, don?t spend their free time studying up on the finer details of education policy or the tax system.
This means that for the vast majority of respondents, what they think of a policy will depend on how you explain it to them (the exception is if you tell them who is proposing the policy in which case partisan allegiances tend to kick in and make the decision but that?s another blog post entirely).
Getting the explanation right therefore is hugely important but unfortunately it?s quite difficult to distil complex government policy into an explanation that presents enough information to allow respondents to make a fair decision while also being short enough that someone clicking through a survey will actually take the time to read and digest.
As an example, here?s the explanation we gave to respondents when asking them about ?free schools?:
?Like state / comprehensive schools ?free schools? are schools that are funded by the government but, unlike state / comprehensive schools, they are not under the control of the local authority. They can be set up by parents, teachers, charities or businesses and are free to attend.?
This got to the heart of the difference between free schools and normal state schools which is that they are publicly funded but more independent. People responded broadly positively to the idea with 44% calling them a good thing and 22% a bad thing.
However, it?s impossible in a short explanation to capture any of the debate or controversy surrounding free schools, such as claims that they divert money away from existing state schools or disproportionately benefitting children from wealthier backgrounds in the same way that independent schools do.
We did address one of the areas of controversy about free schools which is that they are able to hire teachers with fewer restrictions than other publicly funded schools, mirroring the independent school practice of hiring whoever they like. The explanation we gave was below:
?While teachers hired by state / comprehensive schools have to have a PGCE or equivalent teaching qualification, ?free schools? are free to hire whoever they choose, as private schools do. The advantage of this is that they can employ people who have more experience of the world outside teaching while the disadvantage is that it may risk children being taught by under qualified teachers. How concerned, if at all, are you that ?free schools? are able to hire teachers who may not have a PGCE or equivalent teaching qualification??
While this was all, to the best of our knowledge, completely accurate and attempted to show some of both the positives and negatives of recruiting teachers without traditional qualifications, the danger here is that the question itself may be giving the issue false prominence by taking one aspect of the debate out of context. Without taking sides it may be, for example, that poorly qualified teachers in free schools are a problem but so rare that in the grand scheme of things, this isn?t a major issue with the policy itself. Focusing a question on it though may make it seem like this is the defining feature of free schools which makes the policy itself questionable.
This is something that always happens most of the time when polling about complicated topics unless it?s part of a much larger study devoted to all aspects of that topic that can put everything in context.
I think these questions are still valuable in showing, at very least, how people respond to the concept or the basic idea behind a policy but it?s important to remember their limitations and think about what sort of information people are going to be basing their answers on when taking part in the survey.
