Patterning Collective Intelligence: Users, Aggregation, and Emergent Service

I am still thinking on collective intelligence and emergence. In particular I have been thinking about the pattern in the following terms:

• User Contribution: The user context and the specific actions they perform. Basically the selfish (user-centric) reason the user accesses the site. Contributions are clicks, votes, ratings, views, comments, of varying degrees from 'interested party' to 'drive-by'. Content does not have to be discrete and lightweight, it can be heavy such as photos, videos, specialised knowledge, etc.
  
• Contribution Aggregation: The collection of contributions from the users (such as the things listed above) in aggregate, and persisted in someway. Involves the selection of features or indicators that are relevant or interesting, as well as those need to facilitate user contexts.
  
• Emergent Service: Services offered, or products that exploit the aggregation of contributions from individual users. Typically involves the use of clever proprietary algorithms (special sauce).
  

The generality of this framework provides the flexibility to map just about any website.

A classical approach may be to provide a service that users want and in so doing persist things users need to use the service. Later on there is the realisation that the collected data in aggregation has value and views are created and exploited, such as selling to a marketing firm or something. The point here is that the service was contrived after the fact, not designed into the system from the beginning.

The total reverse is you start with an emergent product that can only be addressed through the aggregation of the input from users. In this case, it is typical for users to be enticed or bribed to contribute to the product (such as MTurk), or for contributions to be aggregated from drive-by users (lots of low fidelity contributions such as social news), or from larger ownership-type contributions (such as Wikipedia and open source). Naturally, this is the Web2.0 pattern of O'Reilly.

Interestingly, one may consider the coupling between the user-centric contributions and the resultant aggregate service. For example, in social news the contributions are typically lightweight, although there is a tight coupling between the service and the contributions. Specifically, the user is presented with the present state of the system on which that may provide user-centered contributions (comment, save, moderate, submit, etc). The effective time horizon of the contributions is near, although ultimately contribute to the corpus that in itself may be exploited (related articles, resubmissions, SEO, etc.).

An alternative example is MTurk in which the projects typically have a very low coupling between user contributions and emergent product, lowering desirability of contribution and introducing the need for reparations (its boring). I remember a story on GalaxyZoo (maybe NPR) which highlighted the tedium of the project faced by the student/s (?) who contributed clicks in the early prototype. BTW: This seems a natural case to push out a widget for blogs to encourage drive-by contributions.

Google is an excellent example of tight coupling (content of pages and links), and highlights the need in such cases to be careful of manipulations to the state of system. This too is a well known situation in social news, where 'hits equal money' provides powerful motivators to game the system. Flickr on the other hand provides a corpus of photos that may be used in all manner of services from search, to dynamic wall papers, to CC stock images. These services as well as others like 'interestingness' are all loosely coupled with user contributions, and as such there is little motivation to game the site (most stock photos? dominate the 'popular' feed? most comments?).

Examples I'm unsure about are those like youtube and slashdot in which ad hoc communities are created around shared experiences, interest, or opinion, and commonly around a user contribution (video, news). The emergent service may the location and participating in those micro-communities (social) which does not fit neatly into the pure data/service model above.

The final point I want to make is that we humans are really crap at realting the bottom-up effects with interesting emergent effects in the case of complex systems. For example, as a group of humans it took us a while to recognize and articulate something as simple (in principle) as evolution, although prediction of the process is really really hard unless severely constrained (in general). I think many of the popular emergent services out there are simple (linear), or contrived after the fact. I think the interesting emergent services are (or will come from) those that capture a lot from discrete user interactions, and do not constrain the way the aggregation of such information is exploited, or at least do not constrain too early. I guess this is the core principle of providing API's (outsourcing the creation of services), and the mantra of 'perpetual beta' toward optimising for the utility of (ability to monetise?) the emergent service.