Comments on: A Disturbance in the Flow

By: Sonny Gill

Sonny Gill — Fri, 23 Jan 2009 20:48:27 +0000

Ah, thank you! I had this same gripe the other day that I was venting on Twitter. There seems to be more and more sites/blogs that popup a message ‘asking’ you if you’d wanna sign up for their free newsletter. I think a site can still promote that but within the confines of the sidebar or where else it flows with the rest of the site. 9 times out of 10 I end up exiting the site that does this, regardless if they have good content or not. Hopefully they start realizing this.

By: Grace Christian Fellowship, an Auburn, CA foothill church

Grace Christian Fellowship, an Auburn, CA foothill church — Sun, 18 Jan 2009 03:26:17 +0000

[...] in the PR department respond. I have seen a couple examples of this. The most recent involves the use of a survey on the Quaker Oats website. I was attracted to this post because I have had experiences like this [...]

By: David Armano

David Armano — Thu, 15 Jan 2009 17:31:03 +0000

Andrea,

Thanks for the comment and sorry it took so long to reply. Good to know Quaker’s intent and nice to see you looked over the comments. There are few good ideas in the comments here so hope they are helpful.

By: Andrea K

Andrea K — Wed, 14 Jan 2009 22:09:35 +0000

We noticed your reference to the new Quaker Oats web site on your blog and we appreciate the feedback! The site is a service that Quaker provides to its consumers and you’re correct in that the intent of our survey is to learn from our consumers visiting the site about ways to improve it and the overall site experience. User data via a survey is key to that. To Chris’s point, we have adjusted our sampling rate to the minimum necessary and our goal is to take this information and make improvements to our website which consumers will hopefully acknowledge and appreciate. We have also integrated polls throughout our site that consumers can opt to participate in which are less disruptive to the entire experience. We’re always looking for new ways to collect this type of information to better serve our consumers and welcome any additional input.

By: Rich C.

Rich C. — Thu, 08 Jan 2009 15:50:55 +0000

I agree with Marc, keep it VERY short and sweet. Ask one question and give an opportunity to contribute a brief comment or answer… a sort of “micropoll”.

By: David Armano

David Armano — Wed, 07 Jan 2009 22:22:17 +0000

Scott, I was using Firefox on Mac.
Chris, I’ll ping you later this week or possibly next.

By: Scott

Scott — Wed, 07 Jan 2009 20:35:41 +0000

I’d be curious to know what browser you were you using when this came up like that.

By: chris sandoval

chris sandoval — Wed, 07 Jan 2009 18:23:34 +0000

1. the survey always measures the site experience.
2. there may be better models than the interruption model, it’s just the best we’ve found so far (and we’ve tried many described in other comments to this post). surveys that are unobtrusive don’t get responses. surveying only brand fans or only angry customers doesn’t have statistical validity which impairs decision-making.
3. there’s always room for improvement.

to back off just a bit, i think interruption is actually good design only if you get the context right. interrupt someone after they’ve been browsing content for awhile, might be OK. interrupt someone when they’re buying tickets for an almost sold-out show, probably not OK.

yes, we have a high loyalty factor but that doesn’t mean that our customers are obsequious. i have a theory that brands with high loyalty garner more vocal feedback because their fans have a personal relationship with the brand – think about when BMW redesigned their entire product line. their biggest brand fans were the most vocal by far, both positive and negative: http://en.wikipedia.org/wiki/Chris_Bangle#Design_characteristics

if you you want to know more about the survey, drop me an email and I’ll set up some time and show you how we use it.

By: David Armano

David Armano — Tue, 06 Jan 2009 23:02:32 +0000

Thanks for all the ideas here!

Chris, those are interesting stats. The survey questions are always about the site experience? Would you say that the interruption model is the best way to go. There’s no way to make it better?

I’m also wondering of the high loyalty sentiment that you enjoy with your customers could be a factor.

Thanks again for that information.

By: chris sandoval

chris sandoval — Tue, 06 Jan 2009 22:25:06 +0000

i know and use the same service that interrupted you. we use it to make our website better – their interruption method has a much much better take rate than anything else we’ve ever tried. additionally, we’ve seen literally less than 0.1% (1/10th of 1%) complaints submitted about the survey interruption.

if you adjust the sampling rate to the absolute minimum necessary AND you actually do something with the survey results the end-user ends up with a better experience overall, even if a few don’t care for the inconvenience. it’s a trade-off.

because the take rate is so high we can simultaneously reduce our sample size (ie, interrupt fewer people) AND get better, more actionable results.