Comments on: Who is Mechanical Turk? http://highedwebtech.com/2008/09/24/who-is-mechanical-turk/ Higher Ed Web Development Fri, 11 May 2012 11:36:29 +0000 hourly 1 By: Paul /2008/09/24/who-is-mechanical-turk/#comment-1290 Paul Sat, 01 Nov 2008 00:40:03 +0000 http://highedwebtech.com/?p=216#comment-1290 The service is useful, but you have to be careful about the quality of the work you are paying for. Click on my name to see the results when I paid MTurk workers to rewrite individual Bible verses in their own words. The results were very mixed. Some people were obviously highly educated while others did not understand basic punctuation. If you plan to use MTurk for any task that requires a certain degree of education and intelligence, you have to get comfortable creating qualification tests and grading workers. I found it useful to create an open test project and then qualify the best workers for my real projects. That way, I can pay more to each worker because I know I will get quality results. The service is useful, but you have to be careful about the quality of the work you are paying for. Click on my name to see the results when I paid MTurk workers to rewrite individual Bible verses in their own words. The results were very mixed. Some people were obviously highly educated while others did not understand basic punctuation. If you plan to use MTurk for any task that requires a certain degree of education and intelligence, you have to get comfortable creating qualification tests and grading workers. I found it useful to create an open test project and then qualify the best workers for my real projects. That way, I can pay more to each worker because I know I will get quality results.

]]> By: Matthew Levy /2008/09/24/who-is-mechanical-turk/#comment-1289 Matthew Levy Thu, 25 Sep 2008 15:38:35 +0000 http://highedwebtech.com/?p=216#comment-1289 Thanks for the summary about MTurk. This (and Amazon's other services) were all the rage at the Web 2.0 Expo last week. It's nice to see some real cost numbers - people are saying this stuff is cheap, but no one ever says how much it actually costs! Did you come across anything that correlates data set size or complexity with turnaround time? Thanks for the summary about MTurk. This (and Amazon’s other services) were all the rage at the Web 2.0 Expo last week. It’s nice to see some real cost numbers – people are saying this stuff is cheap, but no one ever says how much it actually costs!

Did you come across anything that correlates data set size or complexity with turnaround time?

]]>