Down the memory lane (Family of students)

This was my BIT final semester project. I am happy to say that it used a REST like mechanism for transferring data between cooperating universities (well, this was back in the year 2001). I invented MAD-XML (Mayank’s Academic Data Exchange Markup Language) for transfering information between different parties. I was heavily inspired by W3C Web Service standards and was closely following Web service draft reports being published by W3C. This project also exposed an API (similar to the ones now exposed by Facebook, Cozi, etc) to allow universities to query each other’s servers. Here’s the text from the project report:

"The project aims at developing a unified academic data exchange framework. The idea is to allow students to access the vast pool of academic talent from a single point. It will also enable a registered student to access his academic records, notes, assignments, calendar, etc. irrespective of which university/institute he is enrolled with. The project allows a university to register with it. Once a university is registered, it gets access to a tremendous amount of data shared by all other universities. In response, the university promises to share its data pool. This allows a student sitting in any corner of the world to access resources and knowledge shared by, possibly, all the universities of the world. This framework will also make use of the existing infrastructure so that a minimal amount of rework is required from the university. Once the system is up and running, there are variant possibilities. A registered student will be able to manage his records, as well as share experiences and knowledge, and get guidance from others. The system is also extensible as it can be easily extended to provide new services.

I hope this academic revolution will strengthen concrescence amongst students as we move forward in developing this little planet (our beloved Earth), and making it a better place to live in."

Here is a screenshot of the opening page:

Introduction

With the rapid development of science and technology, the advancement in communication, and elimination of distance, it is now possible to instantly share our knowledge, talent and resources with anyone connected to this vast and ever expanding network. This network of computers should now be transformed into the network of people. A network that will bring a feeling of togetherness, a network that will allow people to communicate for social development, a network that lets people share their expertise. And what better place could it be to sow the seed directly into the people who will collaborate in the future for an ignited and a prosperous world.

"The Family of Students" is an endeavor in this direction. This effort when successful will lead to an ambience of cooperation and goodwill. Everyone will be benefited with the knowledge, talent, and resource pool provided in this harmonizing environment. This will lead to the cultivation of team sprit among the people. A culture, which should be the way of life. "The Family of Students", as the name suggests, tries to instill this culture into the students.

The project aims at serving all academic needs of a student. A student can register himself with the site only if he is enrolled with a university that is a member of the site. Once a student is registered with us, he has the following worlds open to him:

1. ODD Service: The Online Dictionary Database Service. This service allows the user to upload words and meanings to the server, or share from the existing pool. In the process student improves his vocabulary. Currently the system will provide support of English language only. The ODD Service is intelligent and hence aware of human's learning process. So it follows the iterative process of leaning and revising.
   
2. University Notes Service: This comprises of all the notes released by a university. All the students have access to the notes provided by a university.


3. myNotes: This allows the user to upload his notes to the server. The notes are embedded within the "MAD-XML's Notes language" and hence are available for exchange with other servers and services. A student may share his notes, or decide not to share. However, students are encouraged to share their notes.


4. Academic Calendar Service: This service is meant to be provided by the universities to its student. It contains complete information about the events, holidays, exams dates, etc.


5. myCalendar: It is a student calendar information holder, which among other things will also reflect and complement Academic Calendar Service.


6. Assignment Service: This contains the assignment given by a university for various courses it offers. A student has access to all the assignments provided by a university.


7. myAssignments: A student can store his assignments using this service. Various features will enable him to submit his assignments to the university, and possibly, in future, a software will check and award marks.


8. Academic Directory Service: This gives information about all the universities registered with the site. Also provide a search engine to search universities by their rank and location.


9. Marks Announcement Service: This service provides the marks announced by the university for any of its courses.


10. Admission Announcement Service: This gives a vast amount of information to the students about any admission notification released by the university. It also provides a search engine wherein a student can get a list of universities offering his desired course.


11. myBookmarks: This facility allows the user to bookmark important information like URL address, books, places, universities, etc.


12. myReminders: This service works in coordination with the myCalendar and Calendar Service to remind students of any upcoming event.

Framework

"The Family of Students" provides a framework that consists of a request-response cycle in the form of question and answer session. The "The Family of Students" server acts as a middleman between the student and the vast number of universities. A student submits his query to this server that it processes and a request is sent to various universities for information. Universities are required to comply with the interface, so that the two servers can understand each other and communicate in harmony.

All the files and services provided are prefixed with either a 'g' or 'l' to indicate their respective locations. 'g' stands for 'global' and specifies that the page is located at "The Family of Students" server. 'l' stands for 'local' and specifies that the page is located at a local server, i.e. a university server.

For example Authenticator.html is an HMTL page provided to the users to enter their credentials, which are submitted to the gAuthenticator.pl. gAuthenticator.pl in turn request the university server for a page names lAutheticator.pl to perform authentication.

MAD-XML: In order to accomplish all the services and features listed above, the project makes extensive use of the MAD-XML (Mayank's Academic Data Exchange Markup Language). This is a question and answer type language.

This is then transferred to the university server in a secure mode. The university server is expected to read the data, and provide an answer to our server.

Similarly various document type definitions (doctypes) have been developed for each service after extensive thought and with a view to allow further expansion. Doctypes are either prefixed with either a 'q' or an 'a' to mean a 'question' or an 'answer' respectively.

Design

The project makes extensive use of the object-oriented paradigm. This enables organized structure of the program along with high possibilities of code reuse. Following modules are provides:

1. Authenticator: to perform authentication
2. Validator: to validate the user-data for its integrity, and to block any attempt for hacking.
3. Odd: top level module to provide ODD Service
4. UniversityNotesService: top level module to provide university notes service
5. myNotes: top level module to allow management of student's personal notes
6. AcademicCalenderService: top level module to provide Academic Calender Service
7. myCalender: top level module to allow management of student's personal calender
8. AssignementService: top level module to provide Assignement Service
9. myAssignment: top level module to allow management of student's assignment
10. AcademicDirectoryService: top level module to provide Academic Directory Service
11. MarksAnnouncementService: top level module to provide Marks Announcement Service
12. AdmissionAnnouncementService: top level module to provide Admission Announcement Service
13. myBookMarks: top level module to allow management of student's bookmarks
14. myReminders: top level module to allow management of student's reminders

In addition to these modules, the tfs also provides two more modules. These are:

1. MjSession: This module is responsible for the managing client's state and is used extensively throughout the project.
2. MjCommunicator: This module is the back-bone of "tfs". This module is used to establish connection with the remote university server, fetch data, parse data, and then feed it to the local programs.

Down the memory lane (AutoArranger)

This was my first ActiveX control that I sold in 2001. The product saw an immediate death when Microsoft launched Visual Studio 2002/2003 which provided similar functionality built right into the IDE. Now I can proudly say that I was competing with Microsoft back in 2001-2002 ;)

AutoArranger aims at delivering accurate object alignment and placement. BTW, I also designed the product logo. Here is the description of the product with About screen:

AutoArranger is a programmer's utility ActiveX Control. It is a powerful tool that can control height, width and font size of an object and aims at delivering accurate object alignment to suite different screen resolutions and size. AutoArranger is a programmer friendly tool that allows the programmer to set properties by the click of the mouse using the intelligent property pages. When placed on a form, it automatically aligns controls to adjust under different screen resolutions and sizes. The object has been designed in Visual Basic 6.0 and can run on Windows environment (including Windows 95/98/NT/2000) or other platforms that support OLE automation.

Why not to use search engines

During my college days I was not aware of search engine clientele. Consequently, the area/domain where Google search was being used was an unfamiliar territory. For me, it was a new search engine that was continuously crawling net and used the PageRank algorithm to bring better search results as compared to other technologies. I believed it was being used by casual surfers. I was still betting more on Semantic web (studying OWL was something new that I did at that time), but I do believe that the ultimate search engine will be based on algorithms that use Artificial Intelligence to understand text available on the Internet and provide accurate, meaningful, and context-sensitive results.

Few years back, I was shocked to find how widely search engines were being exploited by the developer community. I saw many people depending on search engines for code snippets. Personally, I started hating search engines for corrupting developers’ mindset and mounting reliance. Hate is no solution, so I embarked on a new project. The aim was straight “Get rid of developers”. Initially, the task looked simple -- write a program that fetches the correct code snippet from the net, filter the results based on the task at hand, and customize it depending on user’s input.

The results were not convincing, following were the primary causes:

1. Authenticity was not guaranteed
2. Incomplete results
3. Addressed either the simplest (the most common) or a single scenario
4. Incorrect
5. Not ready for direct consumption

Solution to this exists: trust services, semantic web, investment in organizing and building appropriate code snippets, ETL. But the scope is limited. Some IT firms maintain a database that can be queried for commonly used code, but the process is not fully automated. It is an aid, not a replacement.

Although, priorities changed and I completely forgot about this project, looks like I need to restart the engine :)
Intent was much bigger – give software creation directly in the hands of its user. The idea was to develop an application that given the requirements will break these down into atomic tasks, maintain a set of domain specific components, integrate components by guiding the flow of data and control from one component to other (glue), map task to functionality, and relies on net (shared repository) for missing code.
In short build an ecosystem that helps user build what he wants.

I believe in a continuous knowledge acquisition process. A business expands by investing in the development of new product lines, expanding the service areas and acquiring other companies. A developer grows by reading more and more (on a regular basis) and venturing into new domains. This is a slow process, but results in fewer hurdles, timely completion of tasks, confidence of going in the right direction and doing it right.

Search engine provides a quick solution, is addictive, but hollows the intellect. Areas worst hit are product quality and developer's confidence.

When to use search engines:

• for awareness on what’s already there, so that you don't reinvent the wheel
• to look what others are doing, so that you don't do a parallel development (did someone mention competition ;))
• to find online projects, so that you can contribute your expertise
• for new and upcoming technologies -- this is the right time to search for parallelism and cloud computing
• find recipe of paw bhaji :)
• to check whether your internet connection is working :)

Don’t hit the search engines to find answers within a domain where you are supposed to be an expert, use it to find knowledge banks that provide comprehensive theory coverage. Use it to find authentic source of information. There are plenty of these on the net. Here are some of the sites that I visit:

• http://www.msdn.com/
• http://www.sun.com/
• http://www.devx.com/
• http://www.developer.com/
• http://www.oreilly.com/
• http://www.computerworld.com/
• http://www.informationweek.com/

Look for magazines and journals that are published (some are given out for free)

• MSDN
• Dr. Dobb
• Architecture Journal
• CIO
• PCQuest

Subscribe to feeds and newsletters published by infosites. At times, you will be bogged down by the number of emails in your inbox, so choose judiciously.

And most importantly, never forget your textbooks.

So surf well, surf wise.

Compromised technologies

We live in a world of compromises. And when compromise is true for life, it easily gets instilled in whatever we do. Our thoughts, actions and feelings – all have an impact. The effect is minimal if it stays within. The magnitude increases if it comes out in whatever form. The enormity is directly proportional to the number of people a person is linked to. And if the person is an inventor of something that has become a part of our life, any compromise that came along the invention path, will lead to a terrible blow when things don’t work as planned.

Can we not have shock free electricity, or airplanes that bounce off when they collide with building? The vehicles that we use, can these prevent accidents? Do we design homes and use the roads judicially? What about software that we make? When we invent, do we give ample thought to any misuse? If we could, thousands of lives would have been saved. Did we rush to bring the latest invention to the market without bothering about its impact? Is money and fame that important?

There are multiple factors that lead us to compromise. Pressure to delivery or achieve under constraints, competition that can kill the product or relation, politics to gain when something is lacking, tight deadlines that if not met can ruin the idea, envy, lack of knowledge and inadequate experience in handling and decision making. All these impact quality, a compromise that does not have immediate visible impact, but serves deadly in future.

Another question that comes to minds is: Is the compromise worth? The answer is a strict NO, this question itself should not have come had the world been a no-compromise place. Can we have no-compromise world? Yes, we can. Will I be considered arrogant and egoistic if I am a no-compromise-man? Some people might think so, but the clear answer is no.

How to get things done right:

1. Plan, plan and plan. Plan ahead.
2. Dry run the plan
3. Have a road map and milestones
4. Take your own time
5. Implement wisely
6. Be honest and raise any red flags
7. Think ahead and list impacts and misuse. Invent ways to prevent any
8. Design for enhancements for areas where extensions are possible
9. Test it in and out
10. Engineer recovery, protection and lock up
11. Have courage
12. Patience is the key
13. Study well with comprehensive theory coverage
14. Design for anyone and everyone
15. Have a clear idea of what you are doing and how it will change the world

You never know your invention might become a necessity tomorrow, so don’t compromise.

The future is smart clients

Technologies come and go. Some become part of our lives. Some don’t. Some are just fillers. Some are ahead of their times. While others, serve as a base for new ones.

Software developers have used and programmed many. Architectures have selected the best that suits the purpose. But everyone knows we are still working on and looking for new ones. As we get one working, our expectations increases and we want to build more advanced and suitable technologies. Frameworks gets deviced once we have a clear understanding of what we want to achieve, how we want others to use it, and how will it impact future developments.

With Internet, the world became a village. Today, with the coming of mobile and embedded devices the applications are on move. We want information not just on our desks, but anywhere and wherever we are. And it’s not just about being connected, we want our data even in offline or disconnected mode.

We mastered the ins and outs of stand-alone applications, spent a lot on the client-server environments, moved to web pages and web applications and then entered the world of services. All these technologies are now playing a very important role in what’s called as 'smart clients'. These are applications which run on multiple platforms, customized by role, both in online and offline modes, with capabilities to synchronize data with multiple devices.

Multiple platforms: Smart applications will work on our desktops, mobile phones, PDAs, Mobile Internet Devices (MID), web applications, widgets, and any embedded device that user has access to. All have different presentation modes, and functionality depends of device specifications. Some will just be read-only, while others provide full experience.

Customized by role: In the world of applications-on-move, not all devices have the capability or complexity to handle everything. More importantly not everyone requires ever feature of the application. Smart-clients will enable application customization so that only relevant portions/features are available. This not only makes the application less bulky on resource constrained devices, but also serves the security and resource control requirements.

Online/Offline modes: These intelligent applications can detect networks and switch modes accordingly. In online mode application is fully functional. Data caching enables offline mode. While in offline mode, the application simulates an online behavior. Some functionality is disabled to protect data ACIDity and protect the user against the usage of stale data for decision making. But the overall experience is same as online mode. Any data that is to be sent to the server is queued until connection is found, and the lowest cost connection is used for communication.

Synchronization: We now live in a world of one-person-multiple-devices. Smart clients are responsible for not just synchronizing data between the client and the server, but also within all the devices a person use. This enable data availability at all times.

One excellent source of information on smart clients is Microsoft's Smart-client architecture http://msdn.microsoft.com/en-us/library/ms998506.aspx

More coming in with details on each of the features of smart-applications… keep watching…

Components components all around ...

We are now moving towards more organized application development process. This has been made possible due to the presence of well designed frameworks which bring with them best-practices and turn recommendations into required features. Such a development happens when designers and developers focus on building blocks for the application in a well planned manner. These components are reusable object and interact with each other for implementing the desired functionality. XML is now serving the role of integrating these components, facilitating exchanges of information between components, providing validation and controlling application flow. There is a huge market for such components. Organizations are now focusing on different verticals but a lot needs to be done on the horizontal front as well. The good thing for computer people is that it requires less effort and things are more predictable in horizontals. This is still to be conquered and has lots of scope.

This new paradigm is also bringing with it a new software role – integrators. These will be the people who will study different frameworks and components available for there domain. Unlike software engineers, these are right people to build software for them. I believe that the person who knows what he needs from a computer program is the only right person to build it. And this is what used to happen when computers were new. If a physicist wanted to compute the result of some of his findings he would code it, he never went to a software engineers. But yes he needed the OS, libraries and languages built by software people.

Availability of GUI tools will make it very easy for integrators to built applications. AI will make things more manageable and predictable for them.

The role of software engineers will focus on building components for the verticals and horizontals. This will ease their lives too – instead of building large and complex systems, they will focus on a well defined goal.

Welcome to the world of organized software practices.

XML and type checking

A lot has been talked about XML – its benefits and how it simplifies application development and integration.

XML has brought a new application development. This facilitates declarative programming which was considered unique to SQL and other 4G languages.

This paradigm is now irradiating type checking and hence introducing loosely typed data usage. Its only when such applications are run, you see the crash screen..

Ecosystem providers should take this into account by following either of these approaches
1. Use DTD or schemas in addition to some validation mechanism to ensure type checking
2. Use XML only for integration with external system.

… more on this later …