Archive for the ‘Dial-out Surveys’ Category

Change Question Phrasing with Variations

Wednesday, April 21st, 2010

We are pleased to add an easy-to-use interface for variations. These allow you to randomly rotate between different phrasings of the same question. This technique is used to address potential ordering bias by putting different options first, or for other methodological reasons.

variation

Our interface makes it easy to add a random variation. When editing your poll, just click on the new “variation” button, and a dialog pops up letting you provide an alternate phrasing. You may map between the variation’s options to the original answers (e.g. pressing “1″ in the original survey means “520 Bridge”, whereas pressing “2″ in a random variation might mean “520 bridge”). When you call in to record your survey, we will prompt you to record a variation just like any other question, e.g. “Please record question 2 with variation 1 at the beep..”

When someone takes a poll, we automatically pick between the original question and any random variations. For example, if a question has two random variations, one third of people will hear the original verison; one third will hear variation #1, and one third will hear variation #2. Later, when you dump your raw results, we will tell you who heard which variation.

New Knobs for Fine Grained Control on Running Your Polls

Wednesday, March 3rd, 2010

Our customers love the power to design and run polls exactly the way they want. So today we’re excited to announce several new features that provide more control over running polls:

  1. Control callbacks
    If we’re not able to reach someone (there is an answering machine, busy signal, or no answer), what should we do? We used to only give you two options: give up, or try every hour up to 3 times. But we learned this is sometimes not enough; sometimes clients want to retry only one time; or clients want to retry at the same time tomorrow. You now pick how many retry attempts (1-5), and the time between retries (from 30 minutes to one day). Use this to maximize the value you get from your phone list (if they aren’t answering now, they might a little bit later).
  2. Set how quickly we make calls
    By default, we run as fast as possible, making tens of thousands of calls an hour. But suppose you want to call a list of 1,000 people between 6-8pm; if you ran at full speed, you’d be done a few minutes past 6pm, which could introduce possible bias regarding who-is-at-home-when. Now, you can specify a rate like “make no more than 500 calls an hour”.
  3. Define what a “complete” is in more detail
    We’ve also tidied up the notion of whether a call counts as “complete” or not. You can now create a survey so that the call is counted as complete if the survey finishes “normally,” OR if a particular question is answered. For example, suppose you want to know the caller’s stance on a particular issue, but want to weed out answers from people who are not registered voters. You would ask “are you a registered voter, yes or no?” And then you’d ask the registered voters what their stance is, and mark that question as being the survey-completion criteria. The non-registered voters would never get a chance to answer that question, and so their responses wouldn’t be counted as “complete” (though you can of course look at their answers, if you want to).

We hope that these new features will give you more control over the surveying methodology, and help you get the most out of your sample. Please let us know what you think, and tell us what other features you’d like to see (the features above started as a request from a customer!).

Top 10 Ways to Improve Your Poll’s Response Rate

Monday, February 22nd, 2010

Tight budget constraints limit the kind of research and engagement political campaigns (or non-profits, or unions) are able to do. The most common question we get is: how much will it cost me to run a poll with 500 or 1000 completed responses?

The answer: it depends. The cost is almost entirely based on the response rate - how many people complete your survey, vs. how many hang up or don’t answer. The higher your response rate, the fewer calls you have to make, so the less the cost.

Organizations contacting their members tend to get response rates between 10-25%. We have seen political polls to the general public get response rates in the 3-7% range, though some have gone as high as 10%.

So how do you boost the response rate and make every call count? It turns out to be more of an art than a science because it depends on so many human factors. Based on what has worked well for our clients, we have compiled a list of the top 10 tips for improving your response rate. If you have experience with these approaches, or suggestions for others, please share them in the comments.

  1. Use a local caller-id
    The caller-id is the first thing someone sees when you call them. If it’s from a local number, they are much more likely to pick up the phone. Precision Polling lets you select your own caller-id.
  2. Call in the early evening
    If they aren’t home, they aren’t going to answer. The best time to call on weekdays is 6-8:30pm. Friday and Saturday evenings aren’t great, though you can call at almost any time on Sunday.
  3. Keep your intro snappy
    Get straight to why you’re calling in the simplest words possible, or you may risk them hanging up. The best intros squeeze in what you are calling about; why the participant should care; who you are; and how long it will take. Of course, you have to also make sure that you meet any disclosure requirements (some states require an organization name and phone number, for example).
  4. Use a recognizable voice
    If polling the general public, try to get a celebrity to record your poll (e.g. TV anchor, football coach), or failing that, use a female voice with a neutral accent. If you’re polling your own members, ask someone that they will recognize to record the poll, like your organization’s president.
  5. Pump up the volume
    Be sure to test your poll to make sure it’s easy to hear. Recording with a loud and clear voice gives the best results.
  6. Keep your poll short
    Automated polls work best when you’re asking no more than 10 questions (2-3 minutes). Any longer, and people start to hang up. As a rough rule of thumb, for every 10 additional questions, your response rate will drop by 1%. One way around this is to structure your survey so the important questions are at the front; and then define the survey to be considered as “complete” if the respondent finishes these important questions, with the remaining questions as optional (though useful) data.
  7. Use simple sentences
    Simple language and short sentences make it easier to follow questions, especially when listening to them over the phone.
  8. Keep it relevant
    Some of the highest response rates we’ve seen are for targeted local issues (e.g. mayor of your town, or pollution in a small neighborhood), or widely discussed topics (e.g. healthcare). Introduce your survey in a way to make it relevant, perhaps by mentioning a hot-button issue you know the respondent will have an opinion on.
  9. Call likely voters
    If possible for your survey, call those people who are most likely to be actively engaged in the political process (look for good voting histories) because they generally are more willing to pick up the phone and share their opinions. Outside of politics, looks for other signs of engagement, like member attendance. Of course, only do this if it won’t bias your desired results.
  10. Speak their language
    We’ve had customers record polls in dozens of different languages: Spanish, Mandarin, Vietnamese, and Russian to name a few. You can do the same by recording questions in multiple languages and starting your poll by asking which language they prefer. The response rates tend to be much higher for non-English-speaking groups receiving a phone call in their native language than it is for English-speakers receiving a phone call in English.