The blog of all and sundry

Why do you Turk?

This is great.

I’ve been using Amazon Turk for a variety of different projects and I love it.  It’s fantastic to actually be able to put a face to the people who do such an amazing job in contributing.

Thanks Waxy.

What I’ve been reading lately

I’ve really enjoyed the following blogs of late:

Tepom.com – amateur personal finance advice from Scott Bliss.  Usually thoughtful and interesting posts for everyday financial issues.

Cake Wrecks – professional cakes gone wrong.  Oddly addictive.  I’m always surprised at the niches blogs can fill (see also Passive Aggressive Notes).

Smashing Magazine – superb site for all aspects of web design, with a bias towards graphic design (which I need a lot of help with.

Futuristic Play – Andrew Chen’s blog which I read for its excellent posts on viral marketing, product design and user experience.

FiveThirtyEight – Nate Silver’s incredibly addictive election predictor site.  Sadly I probably won’t be back for a few more years now…

Ranking crowdsourced data with curves

I’m a big fan of user review sites.  I’ve been using TripAdvisor since at least 2002 to help plan my journeys, and Yelp is my new favourite site since moving to the US.

Crowdsourcing information is usually a pretty good way of doing things.  There’s been plenty of research which has shown that if you get a crowd of people together and have them, for instance, guess the total number of jelly beans in a jar, the average guess will be pretty close to the real number.  Translating this to something usable in everyday life, I’ve seen people have a lot of success with Amazon’s Mechanical Turk for aiding research, since you can use the sheer weight of numbers to smooth the data.

This is great for something tangible and objective.  However, as soon as you start throwing subjective data into the mix, results get a little skewed.

In particular I’ve noticed lately I can’t quite trust online reviews (from a wide variety of users) the way I used to.  I’m not sure why things have gotten so skewed — perhaps I’m more discerning now, or maybe there are more outliers.  Either way, I haven’t trusted the average rating on sites like Amazon, Yelp and TripAdvisor for a couple of years now.  Instead I read a sample of reviews and then go straight to the upper and lower bounds and try to get a sense of why people are voting in a particular fashion.  Is the product/place being reviewed being unfairly penalized or rewarded? (for example, a hotel might get a lot of 1 star reviews for high parking fees, but are great otherwise.  A restaurant might get a lot of 5 star reviews because they’re cheap and have a nice ambiance, even though the food stinks.)

One method I’ve found for making a better decision is to look at the shape of the score curve.  For example, here are the scores for the top 5 TripAdvisor hotels in San Francisco at the moment, order by TripAdvisor by their average score:

1

2

3

4

5

There aren’t any that really stick out there as being obviously different, but you can see that the fourth one gets a far higher ratio of 5 star ratings to 4 star ratings than the others. These differences become more pronounced the further down the overall list of hotels you go. Hotel A is rated as a better hotel than Hotel B which is ranked (according to its average) after it:

Hotel A:

Hotel B:

Whenever I see a curve difference like this though, I always go for the latter when booking.  Since I changed tactics, I have been having great hotel experiences.  To give this a quantifiable score to compare, I tried out the following formula:

If ((x star votes) – (x-1 star votes)) > 0 then

y = x

else

y = 5 – (x – 1)

(y*(5votes – 4votes) + y*(4votes – 3votes) + y*(3votes – 2votes) + y*(2votes – 1votes) + 1votes) / total votes

Using a formula such as this the new overall scores become:

(260 + 96 – 9 + 8 + 1) / 128 = 2.78125
(600 + 108 + 6 + 8 + 2) / 206 = 3.51456
(1550 + 548 + 54 + 18 + 5) / 699 = 3.11159
(425 + 28 + 15 + 0 + 1) / 119 = 3.94117
(1260 + 948 + 60 + 16 + 13) / 883 = 2.60136

Which as you can see results in a much better looking ranking for the curves:

4

2

3

1

5

If we apply the formula to the hotels A and B, we see the difference becomes more pronounced:

(-26 + 244 + 15 + 22 + 6) / 185 = 1.41081

(250 + 236 – 9 + 46 + 10) / 301 = 1.77076

Introducing a new overall score would help people pick better hotels and for the hotel owners to strive for higher ratings.  I’m also a big fan of the trending data that Yelp has added recently, using Patxi’s as an example:

Coming from someone who wrote his thesis on qualitative user feedback, this has been really interesting for me to look at how you can properly interpret large amounts of quantitative data involving subjective scores.

Update November 7 2008: Thanks to Eric Liu for pointing out some weighting issues depending on vote numbers. We’re brainstorming some new algorithms to account for these situations.

In the meantime, anyone from the myriad of Netflix people who have stopped by, feel free to contact me! tim@<this domain>.

American food is killing me

I’m completely addicted now.  In-n-out.  Krispy Kreme.  Sibby’s Cupcakes.  Fred steaks.  And tonight’s indulgence?  Chicago style deep-dish pizza from Patxi’s.

We ordered a 12″, which weighs nearly 2kg (it is 2″ deep).  It takes 30 minutes to bake (compared to the usual 6 minutes or so for a thin crust pizza).  The pizza actually consists of two layers.  Ingredients are placed on a bottom layer of dough, another layer is added on top of that, and then the sauce is placed on top.  We actually ordered the “low fat” version originally, but the kindly gentleman who took our order swiftly talked us out of that.

Delicious.

Does anyone actually notice a website being ‘free’?

I know that ‘freemium‘ is a popular business model at the moment, but do everyday web users really notice this when they’re using a site?

Personally I always assume a site is free and am surprised/annoyed when it isn’t, unless it offers something very compelling or a freemium model (the best, in my opinion, is Flickr).  However a lot of sites put “free” in large type (including Trovix!) on their landing page.  The only rationales I can think of (beyond the rare case that most other examples of your product are paid) split into two camps:

1) If you require an account to use the site, some people might think they will be asked for credit card details during the sign up.  But how prevalent is this concern?  It’s never come up in any focus groups or user testing.  When directly asked, the people in my study groups were unanimous in the assumption it would be free.

2) It’s used as an attention grabbing bit of text that is intended to make the user feel like they are getting something (which should be a paid service) for free.  This just feels tacky, and depending on how it is done, can detract from the brand.

I was struck by Mint’s use of this tonight, which prompted this post:

Perhaps it is a cultural thing — I have seen many products that label themselves as ‘free’ in a scammy way.  Regardless, unless everyone else is charging (or even if they are, like when Yahoo and Google introduced real time stock prices), it seems an unnecessary feature to draw attention to.

Why isn’t anyone talking about Google’s spam problem?

Sometimes when I’m searching for a particular product or service, the most frustrating thing is the way Google brings up mostly things to purchase.  For example, recently I knew that a direct flight between Sydney and New York used to exist, and wanted to find out some information about it.

These are the top results:

1. Cheap Flights to New York , USA from Sydney

   Compare cheap flights to New York , USA from Sydney , Australia with Airfares Flights.
   www.airfaresflights.com.au/flights/Sydney/New-York-NYC/ – 154k – Cached – Similar pages – Note this

2. Cheap Flights to New York JFK , USA from Sydney

   Search Sydney to New York JFK Flights. Search More Options Below (searches may not include deals above). Sydney to New York JFK Air Travel …
   www.airfaresflights.com.au/flights/Sydney/New_York-JFK/ – 50k – Cached – Similar pages – Note this

3. Flights New+york Sydney

   Flight from New+york to Sydney from 453 euros. Search every airline including low cost. … Return One way Direct flights only Main airports only …
   www.edreams.com/flights/new+york-sydney/ – 21k – Cached – Similar pages – Note this

4. Flights Sydney: Cheap flights to Sydney, eDreams

   Flights New York – Sydney, 1 November 2008, 453€, United Airlines, John F Kennedy Intl Airport, ….. Direct flights only Main airports only. About eDreams …
   www.edreams.com/flights/flights-to-sydney/ – 68k – Cached – Similar pages – Note this

To cut through this I like to use Google Blog Search, where I can usually find some helpful blogger who is talking about the concept I’m interested in.  However sometimes, like this occasion where I’m essentially searching for old news, Google Blog Search fails to remove the cruft, as it tends to give an inordinate amount of weight to more recent posts:

European and Asian Stocks Fall Sharply

Stocks Fall Sharply on Credit Concerns

Credit Crisis Drives Stocks Down Sharply2 hours ago by Ricardo Valenzuela
Crude oil was trading just over $89 a barrel in New York after 2 pm. President Bush made an unscheduled stop on Monday morning to speak about the crisis with owners of small businesses in San Antonio — and the television cameras that …
INTERMEX POWER – http://intermexfreemarket.blogspot.com/

As such, one of my most often used Google sites is Google Groups — its long-term archiving and generally high quality results combined with a more traditional Google search algorithm mean I can usually find very specific discussions I’m looking for.  For example, in the past I’ve used it for finding out people’s experience with getting a green card, sourcing older tech support, or travel experiences.  Lately though, Google Groups has been completely broken.  Here are the results for my search:

Direct Flights Nassau To New York Group: obs27rruhuvtene

Direct Flights New York Group: skfea3bmsss

Cheap Flights From Sydney To Aukland Group: Data Access rodeo1a

It seems like Google has given access to creating groups to anyone with a Google account.  This, combined with the long broken Google CAPTCHA means it is getting completely clogged up.  I went 250 pages in (which in itself surprised me, as you can only go 100 pages using normal Google search) and there it was still.

So I tried a few other searches I’ve done lately.

Roomba battery? Broken.

Olympic games? Partly broken.

Travel to Ghent? Broken.

This one probably isn’t fair.

Anyway, there certainly is a lot of crap in there.

I’ve been noticing how useless Groups has been for several months now and assumed Google would do the usual thing and fix it up pretty fast.  I just checked and the last mention of Groups on the official Google blog was in January 2007 when this feature was taken out of beta, meaning it’s a problem wasn’t fixed in the 18 months it was in beta.

I heard one of the reasons why PicasaWeb limits a user’s space (as opposed to Flickr’s unlimited paid service, a sore point I won’t discuss now) is to “prevent spam” — if they can choose to fundamentally cripple a product in their fight against spam, why not fix one that is broken by spam?

Update: Apparently even more spam issues at hand with Google Groups.

Correlation != Causation

Or does it?

Thanks lee.

Random Wikipedia link of the day – the dancing epidemic of 1518

You can’t make this stuff up.

The outbreak began in July 1518, when a woman, Frau Troffea, began to dance fervently in a street in Strasbourg. This lasted somewhere between four to six days. Within a week, 34 others had joined, and within a month, there were around 400 dancers. Most of these people eventually died from heart attack, stroke, or exhaustion.

(cribbed from a New Scientist book review)

Daily Show takes on sham interviews

Fox News interview by Sean Hannity or infomercial?

Skype will never succeed as a replacement for your home phone…

…so long as they’re happy to just “block” people’s accounts and prevent them from using the service.

Last night I tried calling my wife and I kept getting “call failed”. So I tried from my computer, thinking my Skype phone was screwy. I got the dreaded “account blocked” message.

I rely exclusively on Skype for my phone. I have a San Francisco SkypeIn number, a Brisbane SkypeIn number, a Skype home phone and I also rely on Skype To Go to make international calls from my mobile phone. Aside from my T-mobile bill, they now get all of my telecommunications money.

Regardless of my long-term account which has always been dutifully topped up using their auto-billing, for a reason they never both communicating to me, they have completely blocked any internal or external calls aside from Skype to Skype.

The worst aspects to this are:

• No notification of the blocking event
• No explanation for the blocking
• No customer support to get the account unblocked

I had to file a support ticket requesting they unblock my account.  There are no standard mechanisms for doing this – I had to go to support and select “Other help”.  I emailed a few people at Skype directly, but haven’t heard back from them either.  Other people in the same predicament reported between 4-10 days to get their account unblocked.  In the meantime I am completely without home phone service or international calling.  I am surprised I hadn’t heard of this before it happened to me — I can only imagine as they keep doing this to people they will continue to drive away customers to other VOIP solutions or cell phones.

Update 09/25/08:

Well, first of all I found out there is a dedicate help section for unblocking your account, but it amounts to the same as selecting other help.

Secondly, thank you to Peter in the comments for offering to help out which is very nice indeed.  After filing two tickets, I got in contact with Peeter Mõtsküla who was gracious enough to contact support for me and help resolve the problem.

I also found out the reason my account was blocked.  A couple of months ago I had some fraudulent activity on one of my credit cards.  As such I was issued a new card, but I decided to close the account.  As such when my autobill went through it got rejected and Skype emailed me to let me know that on August 16th.  Either way I think this would have been triggered since the original credit card number was no longer valid.

The weirdest thing is that it wasn’t until September 18th that my account was blocked due to the credit card rejection.  No ‘invalid’ funds had been withdrawn, and everything was fine otherwise.  Pretty frustrating stuff, but good to know Skype has some great people working for them!