via Giles.

The combination of “seems impossible” and “strong network effects” is about as close as you can get to the magic formula for incredible, sustainable success, as with eBay, Wikipedia, and Google.

Trenchant analysis from Joel Spolsky.

Most frequently when I am searching, if I enter multiple terms I am implicitly requesting results for items that match all of the terms. This is the way that results from Google and Yahoo work. If you search for Repair Manual the results are pages that contain both the term repair and the term manual.¹

I’ve bolded and in the last sentence because that is the operative term. The results returned by Google and Yahoo are the logical conjunction (and) of the separate results for the individual terms. If I search for Repair and Manual, Google helpfully informs me, “The ‘AND’ operator is unnecessary — we include all search terms by default.”

The other commonly encountered method of combining results is disjunction. This search returns all pages that contain either repair or manual. There are obviously many more results in this case. For repair and manual Google returns not quite 2.5 M results, while for repair or manual Google returns more than 40 M results.

The problem that I encountered, was that many web sites silently ignore the de facto standard set by search engines and attempt disjuctive, or, searches. Instead of returning pages that contain repair and manual they return pages that contain repair or manual. I’m not exactly sure why so many e-commerce sites seem to do this, but it may have to do with available tools. I know that disjuctive search is the default for the Apache Lucerne search engine library, although Ferret, the Ruby search engine gem which was inspired by Lucerne, switches the default to conjunctive search.

Can you see where this is going?

I was attempting to repair a dishwasher that had stopped cleaning the dishes on the top rack. Some small piece of plastic – perhaps the inner liner of a yogurt container – had gotten loose in the dishwasher, been sucked into the pump and macerated. Small pieces of plastic had been blown through the entire circulatory system. I had cleaned much of it out, but it still wasn’t working and I wanted a closer look, but I wasn’t sure how to take some parts off without breaking them. All I needed was a Repair Manual for my washer.

A little Googling around led me to an appliance repair site. I located my make and model of dishwasher and searched for repair manual. When there was only one result returned, I wasn’t surprised. I only expected one.

Without any closer examination I added it to my cart along with the other part that I knew I needed.

Unfortunately, when the part arrived, it was accompanied not by a detailed Repair Manual but by the ludicrous User Manual that I already had.

I was a little annoyed with myself for not being more careful and a little annoyed at the merchant for giving me a misleading answer to my question, but the cost was minimal (perhaps I would have been more careful if it had been higher).

As I noted above, this experience has recurred several times since, minus the steps of adding an incorrect item to my shopping cart and buying it. I think merchants are making a mistake in ignoring the search engine standard and courting a possible liability.

Customers are dissatisfied by these unexpected results and if they make a purchase based on them, they are going to want their money back.

1. Both search engines do some other processing, so the returned pages might contain “manuals” instead of “manual” or even “fix-it” instead of “repair.”

Why is it that whenever I hear those words on an automated voicemail system, I react viscerally?

If my call were important to the recipient, there would be a live person saying, “Hello,” and asking me how they could help me. That’s what you do when a call is important. You answer the phone.

Instead the business is saying, “Our money is important to us,” “Your time is not important to us,” or worst, “Our money is more important to us than your time.”

After the long weekend and being away for a while before that, I had a few calls that that I needed to make yesterday. Some of these calls required further calls, so eventually I made to five different businesses. For each call, I had to endure some degree of voicemail, and heard the lying words about how important my call was.

I was primed when I heard the phrase again on the final call. I was already mentally composing the blog post that went along the above lines.

Unfortunately, it didn’t end there. This particular vendor had concocted a new trick for managing their voicemail queue. Instead of giving me the chance to either hold for the “next available operator” or leave a voice mail message for callback, they implemented a completely annoying system where they 1) kept me on hold for five minutes, 2) at the end of five minutes they told me that I had been waiting for five minutes and if no one had answered my call after an additional five minutes, I would be able to 3) leave a voicemail message and 4) they would return my call when they could.

Talk about annoying. I had to wait 10 minutes, on actual direct dial, working hours, long distance, just to leave a voice mail.

Why do I think that neither my call, my time, nor my money are actually that important to them?

[Update: September 6, 2007, 8:36pm] So the vendor calls me back and gets my voice mail (immediately.) What do they say, “We are sorry we missed you. Please try calling again.” So I call again. I wait another 10 minutes, to leave another message.

Can I get an “Arggggggh!”

Our desire to believe in an orderly universe leads us to interpret the uncertainty we feel about the future as nothing but a consequence of our current state of ignorance, to be dispelled by greater knowledge or better analysis. But even a modest amount of randomness can play havoc with our intuitions. Because it is always possible, after the fact, to come up with a story about why things worked out the way they did — that the first “Harry Potter” really was a brilliant book, even if the eight publishers who rejected it didn’t know that at the time — our belief in determinism is rarely shaken, no matter how often we are surprised. But just because we now know that something happened doesn’t imply that we could have known it was going to happen at the time, even in principle, because at the time, it wasn’t necessarily going to happen at all.

Just like development, the current state of an idea in popular culture depends on the history of the idea and the random events that impinged upon it.

Scoble’s objection is that search results don’t reflect his notions of what is important in the world.

“As long as I’m the #1 Robert and Robert Redford isn’t as high up as me then you know that this is actually a search problem.”

This could be done by Google, Yahoo, or MSN pretty easily, but it would be a bad idea. The problem is that on the web we have today, Robert Scoble is more important than Robert Redford.

If you look at the Google results for Scoble and Redford, Scoble has 3.9M listings, while Redford only has 3.1M.

As near as I can tell, and with apologies to Robert’s brother, almost all 3.6M of those hits on Scoble refer to the Microsoft Geek Blogger, while there are hits on Redford that don’t refer to Sundance. All of the first 30 hits on Scoble referred to Robert Scoble, only 11 of the first 30 of the Redford results refer to Robert Redford. I also counted the 970-1000 hits on the two terms at Google. These are the highest results that Google allows the user to see. One hundred percent (28/28) of these hits referred to Robert Scoble while 92% of the sample (24/26) refer to Robert Redford.

Your mileage on these results may vary because of where and when you are pulling your results, but they make the point that Scoble is vastly more represented on the web than Redford. (If it makes Scoble feel any better, all of the links to Robert Redford are complimentary or neutral, while lots of the Scoble links are, umhh, challenging, to put it politely.)

Could Google, MSN or Yahoo make an editorial decision to boost Redford over Scoble because more people are interested in the movie star than in the geek blogger? Sure they could. When I was doing Commerce Search at Inktomi we did this all the time. There were all sorts of custom tweaks for various domains. If someone searched on “mouse,” we didn’t return mouse pads, even though mouse pads are formally as relevant as mice. If someone wants mouse pads, they will search for “mouse pad.” This seemed to make sense at the time. We were only indexing products and we were trying to build a database that made it easy for consumers to identify, locate and buy those products. We were trying to get inside the consumer’s head and send them the results that they expected.

Based on the history, consumers didn’t like it. As we were losing traffic, Google’s neutral results (and simple interface) were attracting consumers and growing like crazy.

There are a number of reasons for this, but I think the most important was that when the results are tweaked, the user experience is not as good. The user develops a mental model of how the search engine works. Even though the first time they search, they may be annoyed not to get what they want, they learn when and how to modify their search terms to get the results that they seek. Tweaking the search results makes the responses to modifications non-linear, and harder for the consumer to predict.

Consumers realize the the search results apply to the web that exists and that this web is limited in a number of ways, but they want the results to accurately reflect that reality, not some editorial reality.

I think that these problems will settle out as more people, and more types of people get on the web and start blogging, or otherwise creating content that links out to the world. If Robert Redford had a blog, and every Robert Redford fan blogged their responses to his posts and, I think most importantly, the various film and fan sites were really part of the web – linking out, taking part in the conversation – then Redford would quickly pass Scoble.

Until that time I hope the search engines resist the call to tweak for expectations based on non-web popularity. Who searches on “Robert” anyway?

I like this quote:

The atmosphere of the average workplace is to productivity what flames painted on the side of a car are to speed.

I set out to pursue a simple business errand, but I ended up at the center of dozens of changing lives.

There were two offices involved in my odyssey. I had to file a business registration at the office of the treasurer and tax collector ($25 for a business with a payroll of $0) and then, with that paper in had, I had to file a Fictitious Business Name Statement at the county clerk. The whole errand was unexceptional. I had filled out the PDFs forms and printed them before I arrived. At both offices I took my number, waited my turn, paid my money, accepted my paperwork and moved on.

I was in and out in 45 minutes. But I had walked through the doors of change with my fellow San Franciscans.

There were some rumblings of this at the Office of the Treasurer. When I arrived there were 3 people talking with a receptionist and another 3 who had moved on to submitting their papers to a clerk. There was a separate section of the office devoted to paying taxes and some other things, so everyone near me was in some way working on starting a business. I had a degree of hope and expectancy that I saw reflected in all the other faces, a hope that was not dimmed by the dispassionate jading of the clerks.

None of this prepared me for the excitement of the County Clerks Office. There were about 8 ahead of me when I arrived at, so I had some time to sit and observe. A couple of people seem to be filling routine paperwork, but everyone else was in a couple, some couples alone, others with a single friend or companion, others with parents and uncles in tow.

These couples were getting married! Talk about stepping through a door.

I don’t know why I was so surprised. I received a marriage license and filed it at the County Clerk’s office, but that was when it was in a temporary location while City Hall was seismically retrofitted and refurbished. There may have been some couples who were going to get married immediately after receiving their licenses, but because of the openness of the new space, and the beauty of city hall, everything was more public.

The hope and happiness of the couples and their families was great to be around. I felt envious of the clerks for being able to participate in such joy each day.

If you need a quick pick-me-up, you could do worse than spending a half hour in the county clerks office.