The difficulty of simple design – part 1

The hardest part of being a designer is choosing what goes into the product.

Deciding what should and shouldn’t go in is actually a very difficult choice.  You don’t want it to be overcomplicated, but you want to have a competitive edge.  Sure, it seems easy — just throw out whatever people don’t need and put everything else in.  Unfortunately every user is different.  I read an article which claimed that people only used 20% of Microsoft Office features, and so 80% should be removed.  Unfortunately, each person uses a different 20% of the features.

So how do you decide what goes in without overloading your product?  The easiest features to be sure about are the ‘standard’ ones.  What are the must-have aspects of your product to make it work?  What does everyone love about your competitors?  Put these in!

The rest?  While usability testing will give you some idea of what people would want or like, a user saying they’ll use something in a usability session does not mean they will actually use it.  To decide what might be used, you will need to use some of your best judgement, some user feedback, but most of all, pick features which are ambiguous.  People will always do surprising things with your product.  How people customize and appropriate a system for their own use is called “articulation work” in design academia, and the more ambiguous you make your design, the more people can appropriate it in innovative and surprising ways.

I think Twitter is a great example of articulation work.  Ostensibly it’s just a status update system.  However, people use it for all sorts of things — microblogging, link sharing, ad-hoc meetings, connecting with corporations, getting the news, etc.  What facilitated this was a simple system with a few key features – such as the “@” and “#” operators, and a real time search.  From these basics, the community began using it in new and unexpected ways.

So when you’re trying to keep things simple and to decide “should I put this feature in?”, wonder “how might this be used in other ways?”  It’s much better to put in one feature which can be used in a multitude of ways, rather than overload on catering to everybody.

However, the question remains — how do you decide which treatment for a particular feature makes it in?  For example, say you allow a user to select something via a drop down or with radio buttons, and both test equally well.  How do you decide which to use without providing alternative methods of interaction?  I’ll cover that in part 2.

Why I’m passionate about usability

There was a great Schneier post about why he is so ‘into’ security, and how his mindset differs from so many other people.  I was thinking tonight about why I’m so passionate about user experience and how to improve its general approach.  I think what drives me boils down to the following:

  • Constantly imagining how other people might interpret something

Obviously it’s impossible to know exactly what other people think, so usually I ask them.  What did you think about that?  Why did you think that?  Internally I create a mental model on how people with different views might interpret things.  When I use something I can’t but help imagine how my grandmother might use it.  Would my Dad know what to do next?  How about my best mates?  Not only with design, but this extends to even just being in a group conversation.  As people are relating stories I’m wondering “how will other people here interpret that?”.  I find I can’t but help notice when there’s a gap there, and I often find myself interrupting two people who obviously don’t share the same understanding, “oh by the way John, I think what Fred really meant is this…”.

  • Personal frustration with the design of products

My wife hates this.  She noticed this started just after I began my PhD.  I’ve always been critical of my personal devices and sites I use, but after starting a PhD in human computer interaction I became hypercritical.  Used to be if I got stuck, I’d blame myself and look up the manual.  I’d like to think that I’m fairly savvy, and most times I find myself stuck, it’s usually a usability problem.  On a daily basis, my wife deals with a lot of my frustration.  The worst two designs for me at the moment for this are the Playstation 3 system UI (what were they thinking? The company that brought the simplicity and joy of the PSP interface took it and just broke it) and the new Google search interface (they crowding my results with multiple suggestions that I search for what now? And what are all these new buttons everywhere?  Way too much clutter).

  • A belief that delighting users is the best thing a company can do

As a product manager I understand that it is necessary to balance business requirements with usability.  However it is not good enough to say “well, this gives us x revenue, so even though it upsets the users, let’s keep it in there”.  What about the lost y revenue from the people who stop coming to your site?  By focussing on user experience above all else you give people a product that they keep coming back to.  Lost revenue streams can usually be replaced.  Obnoxious ads aren’t the be all and end all of making money on the internet.  Creating something that makes people tell their friends about how great it is (so long as you have a business plan on how to monetize the traffic) is the best possible thing you can do.  Companies such as Apple and Google show this again and again.  I still believe user experience (in balance) with business requirements is key.

  • Wondering “why is it done this way?” and “how can it be done better?”

Every time I use a new product I always like contemplating why is its design the way it is.  Why did the Peek email device forgo all other online activity?  Could its interface be better?  Why does the iPhone not support MMS and video?  Could their touch interface be done better?  I can never be satisfied when using a product, as I’m always asking why.

  • Embracing change

Getting used to a particular way of doing things is great, as it reduces cognitive load.  However it’s often not the best way of doing it, nor the most intuitive.  If you can find a balance of both, everyone wins.  New paradigms for interaction should be supported, although I’m always happy to let someone else push them on their own designs first, and make them a success first so that when I employ them people are used to them.

  • Respecting users

Above everything else, assuming your users are “dumb”, and that you should cater for the “lowest common denominator” is a bad idea.  Why?  Because they’re not dumb.  They’re not the lowest common denominator.  Different people have different needs, different mental models, and different approaches to completing a task.  Simplifying your design approach to thinking “our users are dumb, let’s make it easy for them” is not usability, and a trap for unwary players.  People are smarter than you think, and designers that find the best ways to make use of tacit skills in their users are those that succeed.

Overall I think usability is more a state of mind than a set of skills.  But it’s a hard state ofmind to become accustomed to.  I wrote my PhD about how to better integrate engineers to the design process and make them aware of usability concerns, and my answer was it’s hard (and “it depends”).  But being cognizant of the difficulties users face, and respecting them and trying to anticipate these difficulties (feel free to just talk to them!) will make your design not just better but more successful.

Ranking crowdsourced data with curves

I’m a big fan of user review sites.  I’ve been using TripAdvisor since at least 2002 to help plan my journeys, and Yelp is my new favourite site since moving to the US.

Crowdsourcing information is usually a pretty good way of doing things.  There’s been plenty of research which has shown that if you get a crowd of people together and have them, for instance, guess the total number of jelly beans in a jar, the average guess will be pretty close to the real number.  Translating this to something usable in everyday life, I’ve seen people have a lot of success with Amazon’s Mechanical Turk for aiding research, since you can use the sheer weight of numbers to smooth the data.

This is great for something tangible and objective.  However, as soon as you start throwing subjective data into the mix, results get a little skewed.

In particular I’ve noticed lately I can’t quite trust online reviews (from a wide variety of users) the way I used to.  I’m not sure why things have gotten so skewed — perhaps I’m more discerning now, or maybe there are more outliers.  Either way, I haven’t trusted the average rating on sites like Amazon, Yelp and TripAdvisor for a couple of years now.  Instead I read a sample of reviews and then go straight to the upper and lower bounds and try to get a sense of why people are voting in a particular fashion.  Is the product/place being reviewed being unfairly penalized or rewarded? (for example, a hotel might get a lot of 1 star reviews for high parking fees, but are great otherwise.  A restaurant might get a lot of 5 star reviews because they’re cheap and have a nice ambiance, even though the food stinks.)

One method I’ve found for making a better decision is to look at the shape of the score curve.  For example, here are the scores for the top 5 TripAdvisor hotels in San Francisco at the moment, order by TripAdvisor by their average score:

1

2

3

4

5

There aren’t any that really stick out there as being obviously different, but you can see that the fourth one gets a far higher ratio of 5 star ratings to 4 star ratings than the others. These differences become more pronounced the further down the overall list of hotels you go. Hotel A is rated as a better hotel than Hotel B which is ranked (according to its average) after it:

Hotel A:

Hotel B:

Whenever I see a curve difference like this though, I always go for the latter when booking.  Since I changed tactics, I have been having great hotel experiences.  To give this a quantifiable score to compare, I tried out the following formula:

If ((x star votes) – (x-1 star votes)) > 0 then

y = x

else

y = 5 – (x – 1)

(y*(5votes – 4votes) + y*(4votes – 3votes) + y*(3votes – 2votes) + y*(2votes – 1votes) + 1votes) / total votes

Using a formula such as this the new overall scores become:

(260 + 96 – 9 + 8 + 1) / 128 = 2.78125
(600 + 108 + 6 + 8 + 2) / 206 = 3.51456
(1550 + 548 + 54 + 18 + 5) / 699 = 3.11159
(425 + 28 + 15 + 0 + 1) / 119 = 3.94117
(1260 + 948 + 60 + 16 + 13) / 883 = 2.60136

Which as you can see results in a much better looking ranking for the curves:

4

2

3

1

5

If we apply the formula to the hotels A and B, we see the difference becomes more pronounced:

(-26 + 244 + 15 + 22 + 6) / 185 = 1.41081

(250 + 236 – 9 + 46 + 10) / 301 = 1.77076

Introducing a new overall score would help people pick better hotels and for the hotel owners to strive for higher ratings.  I’m also a big fan of the trending data that Yelp has added recently, using Patxi’s as an example:

Coming from someone who wrote his thesis on qualitative user feedback, this has been really interesting for me to look at how you can properly interpret large amounts of quantitative data involving subjective scores.

Update November 7 2008: Thanks to Eric Liu for pointing out some weighting issues depending on vote numbers. We’re brainstorming some new algorithms to account for these situations.

In the meantime, anyone from the myriad of Netflix people who have stopped by, feel free to contact me! tim@<this domain>.

Does anyone actually notice a website being ‘free’?

I know that ‘freemium‘ is a popular business model at the moment, but do everyday web users really notice this when they’re using a site?

Personally I always assume a site is free and am surprised/annoyed when it isn’t, unless it offers something very compelling or a freemium model (the best, in my opinion, is Flickr).  However a lot of sites put “free” in large type (including Trovix!) on their landing page.  The only rationales I can think of (beyond the rare case that most other examples of your product are paid) split into two camps:

1) If you require an account to use the site, some people might think they will be asked for credit card details during the sign up.  But how prevalent is this concern?  It’s never come up in any focus groups or user testing.  When directly asked, the people in my study groups were unanimous in the assumption it would be free.

2) It’s used as an attention grabbing bit of text that is intended to make the user feel like they are getting something (which should be a paid service) for free.  This just feels tacky, and depending on how it is done, can detract from the brand.

I was struck by Mint’s use of this tonight, which prompted this post:

Mint\'s logo with \'free\' in it

Perhaps it is a cultural thing — I have seen many products that label themselves as ‘free’ in a scammy way.  Regardless, unless everyone else is charging (or even if they are, like when Yahoo and Google introduced real time stock prices), it seems an unnecessary feature to draw attention to.

Why isn’t anyone talking about Google’s spam problem?

Sometimes when I’m searching for a particular product or service, the most frustrating thing is the way Google brings up mostly things to purchase.  For example, recently I knew that a direct flight between Sydney and New York used to exist, and wanted to find out some information about it.

These are the top results:

  1. Cheap Flights to New York , USA from Sydney

    Compare cheap flights to New York , USA from Sydney , Australia with Airfares Flights.
    www.airfaresflights.com.au/flights/Sydney/NewYork-NYC/ – 154k – CachedSimilar pagesNote this

  2. Cheap Flights to New York JFK , USA from Sydney

    Search Sydney to New York JFK Flights. Search More Options Below (searches may not include deals above). Sydney to New York JFK Air Travel
    www.airfaresflights.com.au/flights/Sydney/New_York-JFK/ – 50k – CachedSimilar pagesNote this

  3. Flights New+york Sydney

    Flight from New+york to Sydney from 453 euros. Search every airline including low cost. Return One way Direct flights only Main airports only
    www.edreams.com/flights/new+yorksydney/ – 21k – CachedSimilar pagesNote this

  4. Flights Sydney: Cheap flights to Sydney, eDreams

    Flights New YorkSydney, 1 November 2008, 453€, United Airlines, John F Kennedy Intl Airport, ….. Direct flights only Main airports only. About eDreams
    www.edreams.com/flights/flights-to-sydney/ – 68k – CachedSimilar pagesNote this

To cut through this I like to use Google Blog Search, where I can usually find some helpful blogger who is talking about the concept I’m interested in.  However sometimes, like this occasion where I’m essentially searching for old news, Google Blog Search fails to remove the cruft, as it tends to give an inordinate amount of weight to more recent posts:

European and Asian Stocks Fall Sharply

8 hours ago by Jack Kelly
Total, the French oil company, slipped 4.5 percent on heavy volume, as oil futures for November delivery on the New York Mercantile Exchange fell as much as $3.92 to $89.96 a barrel on the New York Mercantile Exchange.
CompliancEX – http://compliancex.typepad.com/compliancex/
[ More results from CompliancEX ]

Stocks Fall Sharply on Credit Concerns

5 hours ago by chrisy58
New York Times. http://www.nytimes.com/2008/10/07/business/07markets.html?_r=1&hp&oref=slogin. October 6, 2008. The selling on Wall Street began at the opening bell on Monday and only intensified as the morning went on.
Chrisy58’s Weblog – http://chrisy58.wordpress.com/

Credit Crisis Drives Stocks Down Sharply2 hours ago by Ricardo Valenzuela
Crude oil was trading just over $89 a barrel in New York after 2 pm. President Bush made an unscheduled stop on Monday morning to speak about the crisis with owners of small businesses in San Antonio — and the television cameras that
INTERMEX POWER – http://intermexfreemarket.blogspot.com/

As such, one of my most often used Google sites is Google Groups — its long-term archiving and generally high quality results combined with a more traditional Google search algorithm mean I can usually find very specific discussions I’m looking for.  For example, in the past I’ve used it for finding out people’s experience with getting a green card, sourcing older tech support, or travel experiences.  Lately though, Google Groups has been completely broken.  Here are the results for my search:

Direct Flights Nassau To New York Group: obs27rruhuvtene

auwwb…@dipnoi.az.pl obs27rruhuvtene *Direct Flights Nassau To New
York* <http://sebj2o8a3l.airlinetickets24.info/?i=30091017&data=airline+ticket>
Direct Flights Nassau To New York Flights New York Dublin <http://groups.google.
com/group/obs27rruhuvtene/web/flightsnewyork-dublin>, Flights Sydney To New York
Sep 16 by auwwb…@dipnoi.az.pl

Direct Flights New York Group: skfea3bmsss

mvzdilis…@anosmia.az.pl skfea3bmsss *Direct Flights New York* <http://0vcz7v.
airlinetickets24.info/?i=30091020&data=airline+ticket> Direct Flights New York
Larnaca cheap flights, Wholesale cheap airline tickets, Cheap flights from
cardiff to majorca, Airline tickets bid, Cheap flights from brisbane to sydney,
Sep 19 by mvzdilis…@anosmia.az.pl

Cheap Flights From Sydney To Aukland Group: Data Access rodeo1a

Amsterdam 59.00 New York 179.00 Bangkok 389.00 Caribbean 299.00 Sydney 639.00
Flights Sydney Auckland Compare and book Cheap FlightFlights Sydney Auckland
Sydney.Cheap Flights to Sydney Australia from UK Airports TRAVELBAGTravelbag
provides cheap flights to Sydney flights from London direct flights Sydney.to

Apr 24 by susannabatchelorb8b…@gmail.com

It seems like Google has given access to creating groups to anyone with a Google account.  This, combined with the long broken Google CAPTCHA means it is getting completely clogged up.  I went 250 pages in (which in itself surprised me, as you can only go 100 pages using normal Google search) and there it was still.

So I tried a few other searches I’ve done lately.

Roomba battery? Broken.

Olympic games? Partly broken.

Travel to Ghent? Broken.

This one probably isn’t fair.

Anyway, there certainly is a lot of crap in there.

I’ve been noticing how useless Groups has been for several months now and assumed Google would do the usual thing and fix it up pretty fast.  I just checked and the last mention of Groups on the official Google blog was in January 2007 when this feature was taken out of beta, meaning it’s a problem wasn’t fixed in the 18 months it was in beta.

I heard one of the reasons why PicasaWeb limits a user’s space (as opposed to Flickr’s unlimited paid service, a sore point I won’t discuss now) is to “prevent spam” — if they can choose to fundamentally cripple a product in their fight against spam, why not fix one that is broken by spam?

Update: Apparently even more spam issues at hand with Google Groups.

Cuil.com – the next Google?

From the way Mike Arrington carries on about it (oh and a bit more here too), you’d think so.  My initial impressions are quite different (to be fair, he does state “…it doesn’t appear to have the depth of results that Google has, despite their claims. And the results are not nearly as relevant“).

I started with the old vanity surfing evaluation.  I initially searched for plain old “cederman” (I’m number 2 on Google behind darn Lars-Erik at the moment!).

Searching for \

Very confusing list of results without any sort of rhyme or reason to them.  None of the vaunted contextual search options delineating between the few Cedermans on the web (there are only three or four of us, with three of us being published authors).  There are also some incredibly irrelevant results there, and why is an extremely old copy of my twitter page listed there, but not cederman.com?

“cederman-haysom”, “tim cederman” and “tim cederman-haysom” didn’t fare much better either.  Ouch, it’s not hard when there are literally only three cederman-haysoms in the world!

Anyway, whatever.  The true test of a search engine is looking for stuff that you actually want.  While TechCrunch did several broad searches (such as for “dog”, “apple”, and “france”), these are fairly rare in real world searches.  At the moment I’m planning a trip to Belgium, so I tried one of the cities I’m looking at accommodation for and trying to plan a tour of.

No results for Ghent on cuil

No results!  Search for the alternative spelling “Gent” only showed pages in Dutch.  The most curious aspect to this is that contextual search worked here.  The options shown up the top are valid Ghent related things you’d search for, and it’s actually really nice having them identified and clickable in this manner.  So why have “no results” for the main page?  Doesn’t make any sense at all…

Finally, what’s with the ordering of results?  At first I thought I had to read sideways, in rows essentially, but the results don’t line up.  If I have to read by column, then I need to scroll to the bottom, and then all the way back to the top.  Without a sense of relevance, it’s very disconcerting, particularly if you’re searching for something you don’t know much about.  Sure it might help provide some users with a shotgun spray of results, but I think there’s a good reason why Google’s layout still works best.

As an example, imagine someone looking to buy a new Roomba:

Results for searching for a cheap Roomba on Cuil

While Arrington also states “I want to reemphasize that Cuil is only an hour old at this point, Google has had a decade to perfect their search engine.”  This is disingenuous to say the least.  Cuil is certainly not an hour old, and Google was VERY impressive when it first launched.

Still, I’ll have a close eye on what they do next.

Spamming users

One of the great things about Australia is we have a very strong department in the government called the ACCC.  The Australian Competition and Consumer Commission does a great job of keeping things fair in Australia between businesses and consumers.  While this can be seen as hampering free trade and an open market, they actually do a great job of keeping a “treat people fairly” mentality prevalent, and in practice there is great competition in Australia.

The ACCC help support other branches of government such as the Australian Communications and Media Authority with things like the 2003 Spam Act.  As per the ACCC, “Under the Spam Act it is illegal to send, or cause to be sent, ‘unsolicited commercial electronic messages’ that have an Australian link.”

What this means is even if you have a prior business relationship, if you haven’t explicitly stated “send me emails about stuff”, businesses are in breach of the act if they send you anything to do with a commercial site at all.

Since moving to the US I’ve noticed that on almost every site I use, if I give my email address I can expect to start receiving a decent amount of crap from that company.  For a lot of businesses it ends up losing them income in the long run by alienating power users who would otherwise use word-of-mouth to promote that business.

Lately I’ve noticed something somewhat sinister.  I’ve been trying to unsubscribe from websites and regardless of what I do, I remain on the lists.  Sometimes it’s because the company obfuscates the removal process (hi Mint – by the way, thanks for sending super-confidential details via email without asking me first!  Shame your site is so pretty, so I forgive you), but I’ve seen several examples of late where the unsubscribe is just plain broken.

So let me name and shame some people.

The worst two:

Lee Jeans is a shocker.  Unsubscribe link that does nothing at all.  I had to add them to a deletion filter, as numerous emails to members of staff did nothing to resolve this.  Even mention of the Spam Act did nothing to help.

JobFox.  Ahh, JobFox.  I tried everything I could to unsubscribe from JobFox.  I edited all my preferences, I clicked on links, I emailed the helpdesk, and then I even emailed individual members of their team.  Nothing.  Also added to the deletion queue.

Then there are a whole bunch of smaller sites (Hi DavinciTeam).  Thankfully some startups at least listen when you write to them.  I got a very impressive response from Mixx via the Director of Product Management, Will Kern:

I wanted to let you know that this has been taken care of.  You will no longer receive marketing e-mails from Mixx.  Let me know if there is anything else I can help you with.

Me: Thanks very much – and thanks for letting me know too (and on a Saturday no less!).

Will: You are most welcome!  Saturday, Sunday, Tuesday, who keeps track ;-)

Very pleasant and prompt.  You guys are definitely back in my good books!

Workology also had a similar bug, but again, I got a prompt, helpful response which was great.

Finally, I wish I could remember the name of the site who had an unsubscribe link to nowhere.  Checking back a couple of weeks later and there was a page but with no options on it.

Update: Just remembered.  Stumbleupon.  I never did get a reply from your customer service team either, although thankfully the emails stopped.

All of this begs the question, why do so many companies have broken systems?  Is it a deliberate thing?  Is QA behind the ball?  Am I just unlucky?  Inquiring minds wish to know.

All I do know is it really hammers home just how underappreciated the asynchronous user experience is.  Incorrect or poorly timed emails, slow-to-arrive confirmations, sensitive information, spam, and poor control of all of this can have a huge effect on the user experience of the site.  While this part of design for a new application usually comes late in the process, it doesn’t mean it should be treated as an afterthought or not part of the user experience.

Why doesn’t Google Reader support comments or discussions?

There was an interesting article that got linked on Hacker News the other day the asked why comments on news stories couldn’t be drawn to a central location (I read way too many articles and the breadcrumbs in my brain aren’t taking me back to it – can anyone please send me it?). For example, if on my blog I discuss something on TechCrunch. People coming to this blog comment on the story here. The article was asking why can’t the comments from this blog be combined with those on TechCrunch?

Personally, I can’t think of anything worse. Have you tried following a thread on Fark? Slashdot? Slashdot in particular relies on moderation to stay readable, and even then it’s easy to get lost in the sea of discussion.

Personally, I like to know what my friends and associates think of something. There have been “virtual comment boards” that require a plug-in or external software that let you comment on any site you’re at. But that’s a complete pain in the arse. I want something seamless and fits with my existing habits.

Over the last 6 months all of my friends who are interested in reading and disseminating information online have switched to Google Reader for organising all their sites. I am very impressed with it so far, and am amazed at how long it took me to make the switch. I think I’ve narrowed down the two things that (for me) make it a killer app.

  1. The ability to very very easily add new feeds
  2. The simplicity of navigation and reading itself

But I’ve felt like something is missing. Then it clicked. The other thing I’ve really enjoyed doing on Google Reader is sharing articles. I know that my friends will read them, and even my blog readers can pick up on them. However I’ve often wanted to add some commentary or a note to the article when I share it. Speaking with my wife and brother, they have also mentioned wanting the same thing. I had thought that given my wife works at Google she might go hassle the Google Reader team, but no luck. Searching their blog turns up nothing about whether it has even been a considered feature.

Here’s hoping that they really do try and read all feedback on blogs, and add the one last feature that will make it a truly killer app.