Advertisements
Recently I did a small survey on testinggeek to find out whether testing community prefers opensource tools or commercial tools. After around one month, 81% participants voted for open source tools. I have been using open source testing tools for around four years now so I wasn't surprised with the result. But still, this result got me interested and made me think about why so many people prefer open source tools? What are / were the problems with vendor tools? How open source tools have affected testers and the way we work?
I am sure some respondant might have voted for open source because of moral reasons, but for me and propbably many others its the value that we have got from using open source tools. I have been using Firefox, Selenium, FitNesse, WATIR, Selendion, Concordion and many such tools and was benefited greatly with the rich feature set and support. I have used vendor tools like Silk Test, Rational Robot, Rational Functional Tester, Quality Centre etc in past and have first hand experience of pain / problems of using them.
Lets start with brief understanding of what is open source / free software? OSS (Open Source Software)/FS (Free Software) programs are programs whose licenses give users the freedom to run the program for any purpose, to study and modify the program, and to redistribute copies of either the original or modified program (without having to pay royalties to previous developers). It is important to understand that free does not refer to price, it refers to liberty. Free here refers to the the freedom to understand, discuss, repair, and modify the technological devices / softwares user own.
In this paper, author has done in-depth study and given quantitative reasons to support open source adaption. This paper give us very good understanding and (to some people) confidence that open source software works by capturing information on trend, reliability, performance, scalability and so on. Price might be a factor when adapting open source, but if this is the only reason it might be difficult to justify it sometimes. As mentioned earlier, Free in OSS is liberty, which allow users to have fundamental control and flexibility on the software, since they can modify and maintain their software to their liking. This is probably one of the most important reasons why open source works, in almost every sphere.
But what is making open source tools work specifically for testing community? Lets discuss various factors which are important for adopting any tool and compare open source and commercial tools against them. I am taking a narrow view and summarizing these factors for the automated test execution tools only, some factors listed below might or might not be relevant for other tools.
1. Automation Language - Language in which you write your automation has a big influence on how maintainable and robust your automation suite is. If you use right language ( depending on your context), chances of getting right support, technical know-how and libraries for typical task are much higher. Historically, vendors have always given their own specific language which was good for only a single tool. This IMO, was one of the major drawbacks of tool vendors. This situation is changing with RFT (Rational Functional Tester) supporting JAVA / .NET and TestComplete supporting many languages. OSS, scores really well on this front. Tools like Selenium support almost every major language, and because of their open nature, if support for any language is not present someone from community might develop that.
2. Responsiveness - It can be argued that support from tool vendors should be better because they are getting paid for it. In my experience though, support from tool vendors have fallen short of support that we recieve from motivated bunch of people working as a community on any open source project. These folks are probably much more responsive than various level of support sold by tool vendors. Tool vendores can probably guarentee that support will be available, but if you choose popular open source project, chances of getting right support will be higher. One important point to remember here is, with tool vendors support is demanded (because you paid them.) and with open source, support is requested (Because you need them).
3. Feedback Loop - I started working with test automation tools in 2001 and for initial 4-5 years they were more or less static. There were changes / improvements but more or less they were not radically different. One reason for this slow development was probably long feedback. Users and developers in these cases were two different entities and were oblivious of each other's pain. On the other hand, in OSS users and developers are same folks, so feedback loop is extremely fast. Thats the reason why tools like Selenium, WATIR etc have become so popular and feature rich in such a short time.
4. Short evaluation period - Usually if time is short for the evaluation process, wrong decisions will be made about the tool and eventually they will become shelfware. Good evaluation is essential, but most of the time evaluation period given by tool vendors is not good enough to findout if tool is the right tool for the application under test? OSS on the otherhand, give us long (evaluation) period so when we decide to use any tool, we make that dicision with good knowledge on limitation, capabilities and its applicability in the context of the application under test. This increases chances of succeeding with the tool and so its reputation.
5. Selling is driven by Marketing - Normally tools are sold by sales people to managers and not to users. Tools are sold with wrong practices like record / playback, quick automation and so on. This gives wrong impression to management in terms of what is good automation, how it should be approached and so on. Most of the time, practices like this result in unmaintainable automation suite which becomes useless very soon. Normally tools or testers are blamed in these cases, but most of time its not the tool or tester but approach in which tool was sold /used. Most of the time its the reputation of tool which is damaged in such instances.
6. Vendor locking - Historically, tools have always tried to lock users with their offerings. QTP is integrated with quality centre, Rational robot was integrated with test manager and so on. Thier internal formats, integration everything is coupled with other tools from their stack making it impossible to migrate from one tool to another without substential rework once you are trapped. OSS, on the other hand are like open book and hardly have any motive to lock users.
7. Lack of choices - There are not many tools from the vendors and consolidation in the tool market means choices will be even lesser in future. There are only 4 or 5 major players in the field which hardly gives you any choice. Lack of choice is bad, because it increases the possibility of monopoly and features are not delivered based on what user is expecting but based on what competitors are doing.
8. Community Feeling - Sure there are some products like Mac which can instill feeling of community in users. On the software front though, this behavior is more common for OS than vendors. This community acts as a support centre, as a platform to discuss problem and give direction in which development should be carried on. This community usually provide information to anyone who is seeking information, facing difficulty in implementing anything and so on, without any self interest. I have not done any formal study, but was never disappointed with the community of many OSS tools.
9. Specialization & Generalization - Main motive of tool vendors is to sell more products and make money. They increase their chances of sales by combining more features, providing support for many languages / platforms etc. Most of time these products end up in a very bulky product, which is not easy to operate / understand and reduces the speed at which tools can be adopted. OSS on the other hand, try to solve one problem and they try to do it well. Motive in OSS is not to sell more products, but to solve a specific problem and that takes them much closer to users than vendors.
10. Reduces difference between dev / test team - IMO, with the usage of OSS, testing community is much closer to development team than tools from vendor. OSS has allowed developers and testers to talk in the same language, use same set of tools and allowed tighter integration. They have also allowed testers to leverage the work from developers and understand their work more closely. In the old world of vendor tools, testers and developers were effectively in their own silos, OSS has helped in reducing that gap
These, IMO are the top 10 reasons why open source tools are more popular in testing community than tools from vendors. One interesting reason which I have ommited is emergence of web as a platform. Are you using OSS because of any other reason? Please leave your reasons in the comments.
