Amazon Understands The Long Tail

If it wasn't clear before, Amazon has revealed itself as the once and future Web 2.0 company. Though normally much-ballyhooed as a Web commerce visionary from the original Web generation, Amazon has stayed successful by continued leveraging of The Long Tail (through servicing numerous micromarkets), providing its services as a platform for other businesses to build e-businesses, and by providing compelling participation mechanisms like reviews and their Listmania! functionality. Along the way, Amazon has accumulated millions of customers that it can flip into new users of its other services.

Now Amazon has released an online service called the Mechanical Turk, a completely unexpected and innovative technique for harnessing collective intelligence. Says Michael Arrington's TechCrunch, "the “machine” is a web service that Amazon is calling 'artificial artificial intelligence.' If you need a process completed that only humans can do given current technology (judgment calls, text drafting or editing, etc.), you can simply make a request to the service to complete the process. The machine will then complete the task with volunteers, and return the results to your software."

Several things make the Mechnical Turk both unusual and special. One is that the Mechanical Turk service is fundamentally open ended and not tied to any specific product type or service offering, as long as it can be quantified into something they call a HIT (human intelligence task). This makes the Turk a kind of meta long tail that has farther reach than a product specific long tail (see diagram). Two, it has a working revenue model from day one that includes micropayments to the completor of the HIT. This is a lot more than most Web 2.0 offerings can say. Three, Amazon has provided an entire development platform upon which to build Mechnical Turk functionality.

For one example of the possibilities, I once worked on a SOA project for a large insurance company where a lot of the data needed to be OCR'ed with near 100% accurancy before an expert system could process the document. One large struggle was finding and scheduling people in a cost effective manner to monitor the individual words or letters that the OCR system knew it couldn't process accurately. People had to sit all day and correct these as they popped up. Now this could be outsourced to the Mechnical Turk (with some associated potential privacy issues of course) or more likely, everything could be hosted on another site with the Turk's functionality mashed into it.

What's better, is that the Turk's functionality can scale dynamically given that it presumably has a large pool of people to engage. When one submitter has lots work, lots of people can be tapped almost instantly. When they don't have more work, those people can work on something else. There are lots of interesting efficiency and scale issues related to businesses and employment that are apparently solved by the Turk.

Amazon also has the user base it can use to reach economies of scale and push the Turk past the critical mass tipping point to make it successful almost from day one. Another important thing to take away is that Amazon is clearly engaging in radical innovation years after entering the market and isn't afraid to keep taking risks in the market either, which is unusual in itself for an aleady successful industry leader. And it also shows it fully "gets" the best practices in the Web 2.0 toolkit. I'm really excited by the possibilities here. Let's see what kind of mash-ups evolve from the functionality and user base of the Turk over the coming months.