Sunday, August 02, 2009

A case of knowledge corruption and its remedies

Information these days is flowing from all directions. Traditional sources like books, magazines, newspapers and articles have provided information for centuries. New forms like websites, blogs, emails and sms have emerged and newer ones are breeding. The delivery mechanism in itself has seen a tremendous technological enhancement ranging from text to on-demand services. Whether it is communicated officially or casually, is structured or unstructured, documented or loose, information is found in abundance these days.
Abundance is good and welcomed by all. Unlike commodities, information does not perish with consumption. It combines with the existing knowledge and gives new dimensions to think and work with. Hence, authenticity and correctness of information is of paramount importance for its recipients. Incorrect data cumulated over a period of time leads to knowledge corruption. This is a disaster in a world where knowledge is considered power.
Consider, for example, you receive an email containing statistics about the bandwidth consumption and the behavior pattern of an average Internet user. You find it interesting, believe it, and forward it to your friends and colleague. What you have just done is the beginning of knowledge corruption cycle, unless you were vigilant enough and verified the text you received.

This is a very trivial case with a low impact. Imagine the state of an organization or a team that is working on building a product and follows similar practices for its knowledge management.
In this connected world, the rate at which information can spread is exponential. The process of publishing is also very easy and quick. No certification or authenticity check, not even a simple sanity check is done or is in place. Anyone can publish and circulate anything and there is no accountability. There are no filters that can save you.
How can one ensure that he is not contributing in knowledge corruption? Here are a few simple points to keep in mind:
  • Always quote the source: This helps in building confidence. That’s how scientific journals work. Referencing is an important parameter used in Google’s PageRank algorithm.
  • Control excitement: Excitement leads you to skim information and this leads to half baked knowledge. Emotions make thinking bias. Don’t get carried away and don’t start cooking your own stories.
  • Verify before communicating any dubious information: If you are unsure about the things you are communicating, verify them. Use any search engine and keep searching unless you are convinced.
  • Do not make assumptions: If you are unsure about something, then tell that you don't know. Don't speak just for the sake of speaking.
  • Establish pattern once you have understood the information correctly, whenever feasible try to stay away from patterns during learning phase.
Role of technology and policies
Technology acts as an instrument for enforcing policies. Policies provide rules and filters for publishing information. Technology can also be used to check the authenticity. Here is how we can use a combination of technology and policies to ensure correctness:
  • Information publishing policy: Enforce policies that permit publishing only after authenticity is established. This might be difficult and time consuming for the world information, but is suited for use within organizations. How can this be done when we are not yet very close to machine learning and understanding? Till then, people will have to substitute for machines. This will enable a selected group of people to first validate that the information they are providing (existing information) is correct. Everything above it will be based on and will refer this verified information. How will we add a totally new knowledge base? And how do we add contradicting information (for example, people have different theories and beliefs which are completely opposite to each other)? What about thoughts and vague ideas? These can be categorized as personal opinions and best left to the readers’ perception. If followers increase beyond a certain number, a new religion is formed. This article should hence be categorized as personal beliefs.
  • Rating and collective intelligence: Fix ownership and build a rating system. People will have complete ownership of what they are publishing, circulating, or promoting. Trust-levels can be established by cashing on social networks. People can associate trust-levels to different entities. Say, for example, a person can set that he trusts an entity A (person, source, or information) 100% and trusts another entity B just 75%. The chain continues and the trust-levels are calculated for each entity by using the cumulative ratings. Two types of trust-levels can be established for each entity:
    • Global: This is the trust-level computed using the ratings provided by all the people who have rated the entity.
    • My circle: Trust-level computed using the ratings provided by only the people who are part of the reader’s circle.
    This is similar to other applications of collective intelligence that are already in use for features like “Find similar items” or “You might also like these”.


sunny said...

Hi Mayank, Liked your ideas and thought process. Very structured and focused writing. Keep sharing...

PK said...

Hi, Mayank. It is wonderful and thought provoking piece. I encourage you to publish in some computer journal. I also encourage you to keep on writing such articles.

Keep it up.


Anonymous said...

Thank you Mayank. Such articles must be promoted and broadcast. I promise to do my bit.

Anonymous said...
This comment has been removed by a blog administrator.