Chances are you’ve heard the phrase on television many times, perhaps even made fun of it once or twice. But what does it really mean?
The phrase usually goes something like this:
“Last night’s political poll showed that 46% of people are in favor of John Smith. This poll is accurate within plus or minus 3 points, 19 times out of 20.”
Can you write a list question that includes every possible option? A question as simple as “What is your favorite color” can’t include just the six colors of the rainbow – what about brown, black, and grey? And to tread into more dangerous ground, what about pink, auburn, crimson, burgundy, carmine, and fuchsia? Even questions with a straightforward yes/no answer aren’t that straightforward – should marijuana use for any purpose be legal? What happens when we don’t include every possibly option in answer list?
One of the dreaded problems of writing a survey is ensuring that the answer options are as clear and precise as possible.When answer options aren’t clear enough, responders can’t tell which is the best answer for them. But, if the answer options are too precise, it can be difficult to even interpret what the answer options are.
I know, we’re all pressed for time and doing things differently, even just slightly differently, can take a lot of time. But what if you could make just one small change. Just one.
Leading questions can make your head spin. They make you feel like you must answer the question in a specific way, even when it’s not the answer you might normally choose. Leading questions can often be found in political polls and surveys as candidates try to find, or create, data that supports their campaign goals. But created data isn’t valid data.
Are you tired? Hungry? Bored? Disenchanted? Are your colleagues bugging you with unending emails, phone calls, and questions about where to get falafels for lunch? Are you finding it difficult to concentrate on one task because you have so many other important and unimportant things knocking on your door? If so, welcome to the world of survey responding, a world where surveys are just one of the 17 other things you are being asked to do at exactly the same time.
Double barreled questions are quite common in surveys. These types of questions incorporate two separate topics into one question. When people try to answer the question, they can become confused and unable to answer honestly, particularly if they have different opinions about the two items.
In most cases, the problem is easy to resolve. Simply identify the two parts of the question and create two questions. Even if you think the two parts are perfectly related to each other and no one should have different opinions about the two parts, it is still important to split the question apart.
In the market research industry, there are two common ways to share survey results. One of them is average scores and the other is box scores. Researchers use the term box score so often that we rarely notice we’re doing it and that not everyone know what they’re all about. So here is your guide to what they are and why we use them!
Since we were knee high to a grasshopper, we’ve been taught do as we’re told, make people feel good, don’t upset the boat, and get along with everyone. In other words, agree with people. In simple terms acquiescence is agreeing with something – agreeing with your boss, your mom, your kids, or even a survey question. The problem for researchers is acquiescence bias – saying that you agree with something when you truly don’t agree with it. Much of the time, this people don’t even realize they have this bias.
Why did Steve Jobs say he didn’t need market research? Could it be he was too embarrassed to confess he did use it? Perhaps the surveys he’d seen had leading questions, typographical errors, overly lengthy questions and answers, and a host of other obvious problems?