More Lessons Regarding Crowdsourcing

Give a deeper consideration of my trial crowdsourcing projects, I did a little more research on the topic and came across a post from the start of 2007 titled The "Dumbness of Crowds" by Kathy Sierra. The critical take-away for me was the need to differentiate consensus towards convergence from the collection of discrete and potentially useful information packets.

This point was driven home by a series of simple although poignant examples differentiating wisdom/dumbness, which I want to paraphrase so they stick:

• Collection of book reviews at Amazon / Wiki-based collaboration on a book
• Commented and tagged photos on Flickr / Collaborative editing of a photo
• Input and ideas from varied perspectives / blindly averaging multiple inputs
• Designing, voting and commenting on shirts at Threadless / community designed shirt
  

The premise is that individual contributions should be captured and preserved, not aggregated, converged or averaged towards consensus. This point is reiterated again and again in the post: "Art isn't made by committee", "Great design isn't made by consensus", "True wisdom isn't captured from a crowd".

The second point is that discrete contributions and convergence are two techniques used to address different problems. Crowd-based consensus may or may not be a good way of achieving a converged solution, but aggregation of contributions towards a converged solution is simply different to constructing a database of discrete human contributions.

Pigment clearly captures and maintains individual user contributions in terms of named colors, whereas the humanTSPsolver does not, aggregating used edges into an averaged incidence adjacency list. Pigment provides a system for building a human-powered database of discrete user contributions, whereas the humanTSPsolver seeks a crowdsourced converged solution.

This realisation (mistake?) suggests that it may be possible to phrase the TSP solving problem in such a way that the collection of discrete contributions from users can be used to solve instances. The current implementation jumps this question by assuming that an aggregate representation of contributions is the answer.