Carol Phillips
Tuesday, December 14, 2004
Online panels are superior to telephone surveys. Those who are still doing surveys the old fashioned way are paying a high price, and not just in dollar terms. Telephone surveys have always been suspect from a representation perspective. Who really believed that they ever represented a 'random sample? Now with many respondents believing that do not call applies to polls as well as sales calls, 8% of the population completely disconnected from a landline, do not call lists now extending to cell phones, anyone who uses the phone for survey recruiting should beware.
That's not to say online panels are without their problems as well. I can speak from first hand experience that online panels are not all of equal quality. In general, I advise clients to stick withpanels that use email over Internet, have enough representation of young men and minorities to be useful and balance their samples mathematically to census figures. Beyond that some good indications of quality are response rates. Panels that are 'grabbing whoever they can and pelting them with questionnaires probably are going to have lower response rates. Another indicator is how respondents are compensated. You don't want professional respondents.
Finally ask whether the panel allow you to recruit qualitative research respondents from among survey respondents? This powerful technique is quickly becoming popular as a way to get extra insights about difficult to recruit for populations such as concept acceptors or super heavy users.
I am aware of just one panel that meets all of these criteria, BIGresearch. Unfortunately, their panel is not available for custom research unless you are already a subscriber to one of their syndicated panel products. If you decide to go with a self-serve panel such as those offered by Survey Savvy, Zoomerang, Harris or Greenfield, just be sure you know what you are getting. And (hint) be sure to check the back-end analytic software.
Carol
That's not to say online panels are without their problems as well. I can speak from first hand experience that online panels are not all of equal quality. In general, I advise clients to stick withpanels that use email over Internet, have enough representation of young men and minorities to be useful and balance their samples mathematically to census figures. Beyond that some good indications of quality are response rates. Panels that are 'grabbing whoever they can and pelting them with questionnaires probably are going to have lower response rates. Another indicator is how respondents are compensated. You don't want professional respondents.
Finally ask whether the panel allow you to recruit qualitative research respondents from among survey respondents? This powerful technique is quickly becoming popular as a way to get extra insights about difficult to recruit for populations such as concept acceptors or super heavy users.
I am aware of just one panel that meets all of these criteria, BIGresearch. Unfortunately, their panel is not available for custom research unless you are already a subscriber to one of their syndicated panel products. If you decide to go with a self-serve panel such as those offered by Survey Savvy, Zoomerang, Harris or Greenfield, just be sure you know what you are getting. And (hint) be sure to check the back-end analytic software.
Carol
Saturday, November 27, 2004
Closing of the Digital Divide
Years from now, 2004 will be seen as a watershed for the Internet. It will be known as the year that the demograhpic differences that defined the online world from the unconnected world literally vanished. Those of us who conduct market research online felt the change as it happened. In the old days of 2003, we recruited focus groups based on how long a respondent had been online, this no longer was important. Newbies and old hands responded the same. It seemed the internet population was more and more, like, well like us!
Now there is data to support this conclusion. According to Dr. Jeffrey Cole is Director of the USC Annenberg School Center for the Digital Future, the founder of the World Internet Project (reported first on iMedia) the differences between new and experienced Internet users have almost disappeared and Internet users look more like a subset of the US population as a whole.
In the past, the most experienced users connected over twice as long as the newcomers and were far more likely to be connected through a high-speed connection. Long-time users also connected from more places, both inside and outside the home. New users were much more likely to be looking at chat rooms, playing games and searching for entertainment information and searching for medical information. Experienced users spent much more time than novices buying online, doing work related to their jobs and looking at news online. Fours years ago the average new Internet users did not make an online purchase until they had been online between 18 and 24 months. Prospective shoppers four years ago also did not buy online because they feared the product would not be delivered or would be delivered damaged.
Sometime in late 2003 or early 2004 everything began to change. Now Although long-time users still connect longer, new users are only slightly more likely to be looking at chat rooms or playing games online, and they are Today, Internet users are no longer a subset of America as a wholejust about as likely to be looking at news, entertainment information or doing work related to their jobs. Today, new users buy online almost from the day they get connected. The 18- to 24-month lag period is gone. Both Internet users and non-users believe that prices are lower online and that the availability of products is greater.
The most likely cause four years ago for the vast differences in Internet use by experienced and new users was demographic differences. The earliest Internet users were much more likely to be white or Asian, highly educated, male and with higher incomes. They were also much more technologically inclined. Over the past four years, more and more of America has gone online, with the fastest growing groups being African-Americans and Latinos, females, lower income and those with less education. New users go online knowing what to expect from the Internet having, in many cases, been online before with a friend’s or relative’s connection. New users get down to business much faster than new users of several years ago, with less exploring and experimenting.
The implications - for polling, for research, for online commerce and more are enormous. But in the short-term, I am just happy to be able to answer the inevitable question, 'but is my audience online?'
Now there is data to support this conclusion. According to Dr. Jeffrey Cole is Director of the USC Annenberg School Center for the Digital Future, the founder of the World Internet Project (reported first on iMedia) the differences between new and experienced Internet users have almost disappeared and Internet users look more like a subset of the US population as a whole.
In the past, the most experienced users connected over twice as long as the newcomers and were far more likely to be connected through a high-speed connection. Long-time users also connected from more places, both inside and outside the home. New users were much more likely to be looking at chat rooms, playing games and searching for entertainment information and searching for medical information. Experienced users spent much more time than novices buying online, doing work related to their jobs and looking at news online. Fours years ago the average new Internet users did not make an online purchase until they had been online between 18 and 24 months. Prospective shoppers four years ago also did not buy online because they feared the product would not be delivered or would be delivered damaged.
Sometime in late 2003 or early 2004 everything began to change. Now Although long-time users still connect longer, new users are only slightly more likely to be looking at chat rooms or playing games online, and they are Today, Internet users are no longer a subset of America as a wholejust about as likely to be looking at news, entertainment information or doing work related to their jobs. Today, new users buy online almost from the day they get connected. The 18- to 24-month lag period is gone. Both Internet users and non-users believe that prices are lower online and that the availability of products is greater.
The most likely cause four years ago for the vast differences in Internet use by experienced and new users was demographic differences. The earliest Internet users were much more likely to be white or Asian, highly educated, male and with higher incomes. They were also much more technologically inclined. Over the past four years, more and more of America has gone online, with the fastest growing groups being African-Americans and Latinos, females, lower income and those with less education. New users go online knowing what to expect from the Internet having, in many cases, been online before with a friend’s or relative’s connection. New users get down to business much faster than new users of several years ago, with less exploring and experimenting.
The implications - for polling, for research, for online commerce and more are enormous. But in the short-term, I am just happy to be able to answer the inevitable question, 'but is my audience online?'
Digital World Goes Mainstream
The Best Data Wins
According to a Nov. 5 Wall Street Journal article, “Polls Taken Just Before Voting Hit Target On US Pres”, “elections are one of the best chances to measure polling results against reality.” That is precisely the reason as a market research professional I closely followed the polling results. My conclusions are 1) that online polling beats telephone polling and 2) sample size matters – a lot.
The most accurate predictions were all from online polls. Telephones, and the people who answer them, do not reflect the population at large as well as the Internet population. Within the online world, the polls that got it right had the biggest sample sizes. Only one poll got it exactly right – BIGresearch of Columbus, OH. BIGresearch’s October survey tapped over 8000 online respondents, balanced 14 ways against the Census population. The data said 51% Bush, 46% Kerry and 3% Nader among likely voters who are decided, a close match to the final actual result of 51% Bush, 48% Kerry and 1% Nader. (http://www.bigresearch.com/samples/BIG-president-final.pdf)
If the results of all the other polls had been aggregated to create a similar sample size, similar results would likely have emerged. Given this kind of validation, it is little wonder that corporate America is seeing a surge in large sample, online market research techniques. In times of uncertainty, the best data wins.
Carol
According to a Nov. 5 Wall Street Journal article, “Polls Taken Just Before Voting Hit Target On US Pres”, “elections are one of the best chances to measure polling results against reality.” That is precisely the reason as a market research professional I closely followed the polling results. My conclusions are 1) that online polling beats telephone polling and 2) sample size matters – a lot.
The most accurate predictions were all from online polls. Telephones, and the people who answer them, do not reflect the population at large as well as the Internet population. Within the online world, the polls that got it right had the biggest sample sizes. Only one poll got it exactly right – BIGresearch of Columbus, OH. BIGresearch’s October survey tapped over 8000 online respondents, balanced 14 ways against the Census population. The data said 51% Bush, 46% Kerry and 3% Nader among likely voters who are decided, a close match to the final actual result of 51% Bush, 48% Kerry and 1% Nader. (http://www.bigresearch.com/samples/BIG-president-final.pdf)
If the results of all the other polls had been aggregated to create a similar sample size, similar results would likely have emerged. Given this kind of validation, it is little wonder that corporate America is seeing a surge in large sample, online market research techniques. In times of uncertainty, the best data wins.
Carol
Subscribe to:
Posts (Atom)
