Who is Mechanical Turk?
I’ve been reading a lot lately about Amazon Mechanical Turk. What’s that you ask? Good question.
Mturk, for short, is Amazon’s human-powered marketplace. Just like you can provision servers on demand, with Mturk you provision small work units that you pay people to complete. These are often small tasks, such as a web search or transcribing a few seconds of audio, and for the work that you do, you get paid, from a penny per unit on up. If you’re concerned about the quality of the responses you get, you can assign each task to multiple people, so it’s done two or three times and you can verify the quality of the response.
Here’s a quick video about the service from the user perspective.
Anyone can also submit work units to Amazon to be completed. Here are three really interesting blog posts about people’s positive experiences with Mturk.
NewsCred.com used the service to help categorize RSS feeds.
iamelgringo.com used the service to verify 6,000 business URLs and addresses. His project was completed in 5 days at a total project cost of $300.
Andy Baio used Mturk to transcribe an audio interview, breaking it down into small chunks for easy transcribing. Transcriping 36 minutes of audio took only 3 hours and cost $15.40.
Are there uses for this type of service in higher ed? It sounds like it could really help on grunt work or large sets of data that need verified or cleaned up.
If you enjoyed this post, please consider to leave a comment or subscribe to the feed and get future articles delivered to your feed reader.
Thanks for the summary about MTurk. This (and Amazon’s other services) were all the rage at the Web 2.0 Expo last week. It’s nice to see some real cost numbers – people are saying this stuff is cheap, but no one ever says how much it actually costs!
Did you come across anything that correlates data set size or complexity with turnaround time?
The service is useful, but you have to be careful about the quality of the work you are paying for. Click on my name to see the results when I paid MTurk workers to rewrite individual Bible verses in their own words. The results were very mixed. Some people were obviously highly educated while others did not understand basic punctuation. If you plan to use MTurk for any task that requires a certain degree of education and intelligence, you have to get comfortable creating qualification tests and grading workers. I found it useful to create an open test project and then qualify the best workers for my real projects. That way, I can pay more to each worker because I know I will get quality results.