Why not to use search engines

During my college days I was not aware of search engine clientele. Consequently, the area/domain where Google search was being used was an unfamiliar territory. For me, it was a new search engine that was continuously crawling net and used the PageRank algorithm to bring better search results as compared to other technologies. I believed it was being used by casual surfers. I was still betting more on Semantic web (studying OWL was something new that I did at that time), but I do believe that the ultimate search engine will be based on algorithms that use Artificial Intelligence to understand text available on the Internet and provide accurate, meaningful, and context-sensitive results.

Few years back, I was shocked to find how widely search engines were being exploited by the developer community. I saw many people depending on search engines for code snippets. Personally, I started hating search engines for corrupting developers’ mindset and mounting reliance. Hate is no solution, so I embarked on a new project. The aim was straight “Get rid of developers”. Initially, the task looked simple -- write a program that fetches the correct code snippet from the net, filter the results based on the task at hand, and customize it depending on user’s input.

The results were not convincing, following were the primary causes:

1. Authenticity was not guaranteed
2. Incomplete results
3. Addressed either the simplest (the most common) or a single scenario
4. Incorrect
5. Not ready for direct consumption

Solution to this exists: trust services, semantic web, investment in organizing and building appropriate code snippets, ETL. But the scope is limited. Some IT firms maintain a database that can be queried for commonly used code, but the process is not fully automated. It is an aid, not a replacement.

Although, priorities changed and I completely forgot about this project, looks like I need to restart the engine :)
Intent was much bigger – give software creation directly in the hands of its user. The idea was to develop an application that given the requirements will break these down into atomic tasks, maintain a set of domain specific components, integrate components by guiding the flow of data and control from one component to other (glue), map task to functionality, and relies on net (shared repository) for missing code.
In short build an ecosystem that helps user build what he wants.

I believe in a continuous knowledge acquisition process. A business expands by investing in the development of new product lines, expanding the service areas and acquiring other companies. A developer grows by reading more and more (on a regular basis) and venturing into new domains. This is a slow process, but results in fewer hurdles, timely completion of tasks, confidence of going in the right direction and doing it right.

Search engine provides a quick solution, is addictive, but hollows the intellect. Areas worst hit are product quality and developer's confidence.

When to use search engines:

• for awareness on what’s already there, so that you don't reinvent the wheel
• to look what others are doing, so that you don't do a parallel development (did someone mention competition ;))
• to find online projects, so that you can contribute your expertise
• for new and upcoming technologies -- this is the right time to search for parallelism and cloud computing
• find recipe of paw bhaji :)
• to check whether your internet connection is working :)

Don’t hit the search engines to find answers within a domain where you are supposed to be an expert, use it to find knowledge banks that provide comprehensive theory coverage. Use it to find authentic source of information. There are plenty of these on the net. Here are some of the sites that I visit:

• http://www.msdn.com/
• http://www.sun.com/
• http://www.devx.com/
• http://www.developer.com/
• http://www.oreilly.com/
• http://www.computerworld.com/
• http://www.informationweek.com/

Look for magazines and journals that are published (some are given out for free)

• MSDN
• Dr. Dobb
• Architecture Journal
• CIO
• PCQuest

Subscribe to feeds and newsletters published by infosites. At times, you will be bogged down by the number of emails in your inbox, so choose judiciously.

And most importantly, never forget your textbooks.

So surf well, surf wise.