Different Research Methods Yield Markedly Different Results

Seeing is believing is especially true in market research. Ball State researchers have just provided solid evidence that observation research is superior to telephone or diary based methods for measuring consumer behavior.

Observation methods of measuring media usage show consumers significantly underreport their daily minutes of time by medium when questioned using diaries or phone survey.

Daily Minutes of Time Attributed To Media Via Each Method

Phone Diary Observation

Home Computer* 21 52 64
Online 29 57 78
Television 121 278 319
Books 18 17 36
Magazines 8 10 14
Radio 74 132 129
Newspapers 15 26 17

Source: Ball State University, Center for Media Design.

Not surprisingly, the least discrepancies were for media with the greatest social desirability (books, magazines) and therefore the least incentive to misrepresent the actual time spent. The implications for researchers are clear: beware! Even relatively innocuous questions such as how many minutes a day do you watch television should be regarded with suspicion and whenever possible, backed up by observation research.

We have seen evidence of social desirability repeatedly in our research. A recent project involved asking adults who in the household was responsible for financial decision-making. Both men and women answer in the affirmative. Further questioning reveal that men are more likely to THINK they are the decision-maker, while women actually handle more of the day-to-day financial tasks and decisions.