Letter: Be wary of ‘political’ survey questions

Survey is a worthy initiative but be wary of politically shaped questions and how the results get positioned

Last month one of those coincidences occurred that has interesting, if not immediately obvious implications.

Our council launched the municipality’s first-ever community satisfaction and priorities survey. And McGill University psychologist/neuroscientist Daniel J. Levitin released a new book bearing the delightfully subversive title, A Field Guide to Lies: Critical Thinking in the Information Age.

The connection? Well, Levitin describes his book as helping us “spot problems with the facts you encounter…that may lead you to draw the wrong conclusions,” and “sift through the various claims we hear and to recognize when they contain misinformation, pseudo-facts, distortions, and outright lies.” The problems, he says, arise from a range of issues, including how data is both collected and reported. This is particularly critical when looking at “topics that are highly politicized.”

And that’s where the Oak Bay survey comes in. Most of the online survey comprises straightforward and useful questions about community concerns, and service and information satisfaction levels. It only runs into trouble when it veers into a set of questions that appear to have been driven primarily by political considerations.

As Levitin says, “how something is defined or categorized can make a big difference in the statistic you end up with.”

Take the question about creating a “dog park.” It doesn’t define what a “dog park” involves, and then suggests that it would likely entail purchasing property, without explaining why.

Or the questions about deer, which immediately plunge into whether there’s an “overpopulation,” without defining that term, or first exploring community values around wildlife. (Full disclosure, I’m married to the president of the Urban Wildlife Stewardship Society, but not a member or participant).

If you answer “no,” you’re jumped over questions about your willingness to pay an increase in taxes to “reduce the deer population,” and whether you support a deer cull.

The latter question also hinges on a cull being the “only option available” (it isn’t) and being “humane” (the BCSPCA is already on record as “strongly” opposing Oak Bay’s approach to deer culls).

Given such shaky foundations, it’s hard to see how valid results could emerge or useful conclusions be drawn from these questions.

And then there’s the questions about balancing tax increases with services and the funding of future projects. Stripped of the choices and trade-offs that could be involved, they are devoid of real meaning.

Unfortunately, these suffer from the same kind of “political sculpting” as the 2014 Oak Bay amalgamation ballot question, where residents were asked, without any substantive community discussion or information, whether they would be willing to take a leap into “being amalgamated into a larger regional municipality.” While more than 60 per cent quite reasonably said “no” under those conditions, nearly 40 per cent said “yes.” That might suggest there’s a high level of interest in further exploring the issue, rather than that it has now been “settled.”

Overall, a community satisfaction and priorities survey is a worthy initiative and council deserves kudos for taking a step toward greater accountability to its electors. Oak Bay residents, however, should be wary of the handful of politically shaped questions it contains, and how the results get positioned and used.

As Levitin says, “If a statistic is composed of a series of poorly defined measures, guesses, misunderstandings, oversimplifications, mismeasurements, or flawed estimates, the resulting conclusions will be flawed.”

Bruce Kilpatrick

Oak Bay

Bruce Kilpatrick is a communications professional and 35-year Oak Bay resident. He’s been a volunteer board member for community organizations and a youth soccer and baseball coach.