Task Turking

From Random Hacks of Kindness
Jump to: navigation, search

This problem is for those who aren't necessarily coding directly: it's hacking a spec, though a good code hack could make huge progress. Update: Those who want to join a project to actually code this, can check TaskMeUp

Contents

[edit] Desired outcome

A design and architecture document and as much code as possible outlining the detailed scenarios, functionality, workflow, schema, API and even UX for a web site and service that tracks and facilitates job tasking to support job “turking” for systems like OpenStreetmap and also incident follow-up for systehttp://wiki.rhok.org/skins/common/images/button_italic.pngms like Ushahidi. It should also consider the pro/cons of this system vs Crowdflower.

Present some mockups and or functioning code.


[edit] Implementations

RHoK 2010 NYC: TaskMeUp

[edit] Over the top outcome

Make it work in code!

[edit] Scenarios

the system should ideally support the following “Haiti” like scenarios:

1. Open Streetmap turking: distribute/queue a number of parallel tasks across a set of users.

a. Background (not in problem): There’s an earthquake (think Haiti) and the Open Streetmap (OSM) community steps up and starts helping out. Volunteers from all over the world ask “How can I help”. They go to the OSM website and see a button that says “Help by digitizing a road, camp or other feature”.

b. In problem: The user is taken to a signup page where they enter some minimal set of details (email) and click “get started”. They then get a web-page with a map segment and a task description. They work on this and hit “submit” along with some comments. They then get to say “I’m done for now” or “give me another one”.

c. System requirements: a web service and set of API’s that OSM can build around to setup, prioritize, distribute/manage, and track users and their tasks and any validation steps required.

2. Ushahidi turking: a queue of translation tasks.

a. Background: user comes to Ushahidi site and sees a link “How can I help”.

b. In problem: This link takes them to a list of tasks [translate|geolocate|…]. They pick one and get their instructions on what do to translate/geocode ushahidi input: including the information/input, the output, and somewhere to provide a comment when they’re done.

c. System requirements: a set of web-services and API that Ushahidi can program against to enable this with their choice/queue of tasks.

3. Ushahidi report follow-up: tracking what happened after a report was submitted to Ushahidi (or similar system). Basically an issue-tracker.

a. Background: User comes to Ushahidi and sees a report “orphanage needs food”. In the report is field that lists “status” as [unclaimed|Mercy Corps is working on this]. If unclaimed there’s a button to “do something about this”, and if claimed there’s a link to “contact Mercy Corps”

b. In problem: user clicks “do something” and they get a form where they state who they are, their contact details and what they’re doing about it. They can come back at any time and add a status report or chance the status to [dealt with|no longer an issue|bad report|working on it|doing an assessment|validating|…]. Other users can add comments to this report.

c. System requirements: Ushahidi can specify in the system via the API’s or account setup what the options/states are, and call the API’s to show the current state of a report.


[edit] Background reading/research links:

  • PyBossa - PyBossa is an open source platform for crowd-sourcing online (volunteer) assistance to perform tasks that require human cognition, knowledge or intelligence (e.g. image classification, transcription, information location etc).
    • PyBossa was inspired by the BOSSA_ crowdsourcing engine but is written in python (hence the name!). It can be used for any distributed tasks application but was initially developed to help scientists and other researchers crowd-source human problem-solving skills!
  • The OpenStreetMap team had begun work (and have some code) on an OSM tasks webservice that enables the checkin-checkout of squares for distributed work
  • Ushahidi Haiti at Tufts hacked together a workflow to do distributed translation and geocoding of incoming SMS messages
  • Crowdflower is a great model follow for some of the tasks as it offers a fully featured commercial service with fulfillment by Amazon mechanical turk and others. Their API is at http://crowdflower.com/docs/api, and there's a ruby gem for the API interface at http://github.com/dolores/ruby-crowdflower
  • http://textonic.org/ is an open source tool for submitting SMS related tasks (tagging etc) to Amazon's Mechanical Turk
  • Amazon's mechanical turk has a fully documented API, and there are a bunch of FOSS libraries for interfacing with it https://www.mturk.com/mturk/welcome

[edit] Comments

[edit] --Gavintreadgold 04:19, 5 June 2010 (UTC)

  • Scenario (3) has a working prototype that uses Sahana Eden. The core Ushahidi and Sahana devs have been working on this since Haiti, and have been coordinating closely. See Sahana Eden demo Ticketing Module. This will also include 'closing-the-loop' whereby when a ticket is closed in Sahana, it will post back to the original Ushahidi messages indicating that this has been closed. There is still some work to do to test and refine it but it is mostly done. This work started during Haiti response, and continued recently at RELIEF 10-3.
  • Any ticketing system also needs a good group and user system to underpin it. In Sahana Eden we will be extending and linking the Volunteer Registry to the Ticketing Module to enable tasks to be assigned to groups etc. This also needs to be linked to a good ACL as sometimes assigned tasks should be limited to a single organisation or group.

[edit] --omdesign Dec 6, 2010

Congratulations! Great work at NYC RHOK. In looking at your interface two things come to mind. 1. I think it would be easier and more useful UX to have each button be the destination/submit. Prevent dropdown selectors; code every option as a self titled submit. In cases where more than one option is required, have a few 'submits' and an 'all done'. This would greatly facilitate fast 'Turking' and keep people going. 2. the 'take me to another' link (button) should be at cursor x,y or near where they clicked last, again, to reduce mouse travel.

These things may seem trivial but make the difference between me Turking five or fifty jobs in a session.

Along the lines of 'gaming' to keep it interesting... Would you consider an "8 bit" bar graph along the left margin showing current user's total turks as height and colorize it by Turks/minute?

  • This could be by IP rather than having to deal with a login.
  • As traffic increases, you could group IP's thus making it represent a 'regional effort'.
    • Three bars showing:
      • current 'highest'
      • yours
      • current 'lowest'

[edit] --DamonOehlman 14:40, 5 June 2010 (UTC)

We have some very rough code and some screen mockups for an possible design that would handle implement a AMT like system, but throw in a little Stack Overflow for flavour. We have a github repo and wiki started which is pretty rough at the moment, but may have more in there in 24 hours time :)

http://github.com/jaredwy/taskoverflow

Retrieved from "http://wiki.rhok.org/Task_Turking"
Personal tools