Friday, October 19, 2007

Surveys

Following comments regarding the GSS ownership survey and observations over time, one really has to wonder how accurate surveying a few thousand people (at most) is out of a population of 300 million.

On firearms, the GSS states that the numbers of owners are decreasing when training courses are filled to capacity across the nation, are constantly being expanded, and sales are booming,. while the Gallup poll has a difference of nearly 20 million more owners in comparison and a similar number of respondents.

Even if 80% plus of these sales are by previous owners, that still shows a steady increase in previous non-ownership and expanded collections.

On exit polls, the last few elections have emphasized how inaccurate exit polling has become(waits for the inevitable conspiracy theories).

So I have to ask: How many of us have been surveyed/polled or know someone that's been surveyed/polled over an issue?

3 comments:

Anonymous said...

I’m not a statistician, but I think you can get pretty reliable results (plus or minus an acceptable level of error) with fairly small samples. Of course, sample size is a separate issue from whether or not the poll or survey has been properly designed.

I think it’s important to note that most phone surveys (or exit polls) are short, because the interviewer knows we’ll only stick around for a few minutes. The more sophisticated polls are longer and structured to create a sense of trust so that you’ll answer personal questions at the end. They usually begin with fairly innocuous questions and conclude with requests for detailed demographic data (eg, household income).

Phone surveys don’t have the luxury to mimic this process, so they never create any trust. As a result, when they do ask questions on controversial topics, respondents are less likely to give accurate answers. It’s not just a question of guarding your own thoughts, but sometimes also a desire to please the interviewer (eg, be polite, avoid controversy, give the answer you think they want, etc), or to just end the phone call.

I get called all the time for state and local political campaigns, customer surveys, etc. Whenever I get overly intrusive questions I simply provide evasive answers or BS. I can think of lots of topics, like gun ownership, where others would respond the same way.

Anonymous said...

When you compare the surveys to the manufacturing and sales records, there's only one possible conclusion:

A lot of gun owners lie to survey takers about their gun ownership.

Kevin said...

A lot of gun owners lie to survey takers about their gun ownership.

That's the result of the decades-long slow-motion hate crime that's been waged against gun owners.

Best example: Emily Yoffe, NPR's "human guinea pig" related this snippet from her Slate piece, Guinea Get Your Gun:

"So anathema are guns among my friends that when one learned I was doing this piece, he opened his wallet, silently pulled out an NRA membership card, then (after I recovered from the sight) asked me not to spread it around lest his son be kicked out of nursery school."

You think people like this are going to answer a phone survey honestly?