Advertisements
Software testing is a relatively new field and has changed considerably in past few years. It is not taught in many universities and when I moved from development to testing in 2001, I was confused about it. I tried to learn from internet, books, forums and was not impressed with the information I got. I even did my certification (CSTE, if you are interested) but that wasn't very useful either. During that time, I came across many interesting theories / concepts and after working in the industry, I know they are not true, and are myths. Unfortunately, some of these myths are still in practice and widespread.
Myths in software testing has done this field more harm than good. In this post, I will explore popular software testing myths, why they are myths and what wrong these myths are doing to our profession?
1. Testers are Gatekeepers Of Quality - Nothing should be released to production before test organization / testers give their approval.
In many organizations, tetser / test team fight for this right. It makes test team empowered and to be honest, when I started my career I did think this is right way. In reality, this view is extremely dangerous, for the team and product both. Test team is information provider and provides information to stakeholders. It is up to them to act on the information test team provides. When testers act as gatekeeper they become responsible for quality in the product. It gives them feeling that other than test team, no one else is concern about quality. It also increases pressure and sometime creates situation wherein testers are afraid to release product, because there might be one last remaining defect which is not uncovered.
2. Complete testing is possible - If you plan properly, it is possible to test software completely, identify and fix all the defects.
Many organizations feel that it is possible to test software completely and fix all the defects. Nothing can be further from truth. No matter how much you test, complete testing is just plain illusion. Applications are becoming more and more complex and possibility of testing all the features, under all the conditions is extremely rare. When management is trap in this belief, test team will become responsible for every defect. Also, if test team attempts to do complete testing, they will become bottleneck. In reality, almost all the products have defects. Only difference is what kind of defects they have and how frequent is their occurrence. If you try hard, I am sure you can find defects in almost any software you use. Complete testing is not solution for this.
3. Best practices - Improving quality is simple & straight forward, just follow the best practices.
Best practices, standards and processes are still a big myth. Not all the standards, processeses and best practices work all the time. Sometime they work and sometime they don't. There is nothing wrong in the practice as such, problem is in not identifying the context and problem before applying practices. Practices are practices, what makes them good or bad is whether they are applied after considering the context or not. Applying best practices is like applying hammer, if you do not consider size of the nail and try to use same hammer for all the nails, sometime it will work and some time it will not. When test team starts implementing industry's best practices without considering their project, timeline, skills, technology, environment, team structure and many other aspects, they get frustrated because they do not get results they expected.
4. Certifications will make you better tester - So go and get CSTE, ISTQB.... etc to become better tester / get promotion.
When I started my career as tester, I was in service industry and certifications were / are considered good. There was a valid reason for that, because if you need more clients than boasting about number of certified test professionals will increase their confidence. But from what I have seen, certification exams are very shallow in nature and does not reflect whether person who is getting certification is good tester or not. Certifications, in their current format can be acquired by anyone who is prepared to study for a couple of weeks and it is highly unlikely that someone will become good tester in couple of weeks time. Certifications in its current format have created unnecessary pressure in the testing community to get certified, just because of peer pressure and client demand rather than as a benchmark for knowledge.
5. Test Automation is Silver Bullet - If something can be automated and you can automate - automate it.
Now do not get me wrong, I am a big fan of automation , but only where it add value. I have seen many engineering hours wasted on developing automation or frameworks for automation which are hardly used. Automation, without considering its ROI and effectiveness is just waste of time. Dr. James in his recent post has highlighted it nicely and made a very good point that manual / automated testing should be considered only after good test design. This mentality of considering test automation as silver bullet, like many other myths is dangerous for our profession because of many reasons. Management sometime can become extremely focused on the automation rather than improving quality. Remember, more automation will not improve quality. Right level of automation combined with required exploratory testing and good test design will certainly improve it.
6. Testing is demonstration of Zero defect - Testing is completed for this product and test team has signed off the product. This product does not have any defect now.
Whoever claim this, is obviously wrong. It is impossible to claim that any product is defect free. Even after spending thousands of hours in testing, there will be one more combination which was not tested, one condition which was missed and for all we know that might surface in production environment. If as a tester / manager you believe that zero defect is a possibility, you will feel responsible for any defect which is uncovered in production. On the other hand, if you modify the sentence and say that for the combinations I have tried, environment and data I have used and scenarios I tested, there was no defect according to my understanding of the product.Also, goal of testing is to uncover defects. As a tester, you can only find defects, you can not claim that this product is defect free.
7. All measurements are useful - Keep track of number of test cases, how many of them are executed, how much automation is present, defect count.. and any other numbers you can think of.
When I started my career, we started preparing reports on the lines of - how many test cases are written, how many of them are executed, how many of them are automated. how many defects were found and so on. Week after week, we would send these reports without realizing that if additional information is not provided along with numbers, they does not convey any meaning. If these numbers become primary consideration for management, quality will suffer. For example. if number of defects are important test team will start filing each and every issue, if number of rejected defects / duplicate defects become important test team will start spending lot more time on defects before filing them or may be will not file at all. Any measurement program should be approached with caution and should always provide clear meaning / summary for all the numbers.
8. Stable requirement and documentation are essential for any project.. BTW development team is crap.
With Agile development, this myth is slowly going away and we have realized that changes are inevitable. rather than fighting changes, we now embrace them. It was different when I started and probably still in many organizations, changes are not welcome, requirements are similar to contractual obligation and documentation is the first thing test team ask for. Development and test team work in their own silos and communication between them is limited to finger pointing. It is impossible to have quality software coming out from such environment. Development and test team should work together to improve quality.
9. Limited time and resources are the main reasons for missed defects - As a test team, we are always pressed for time and hardly have any resources. We could have caught these defects, only if we had more time / resources.
I am sure many of us have heard this and some of us even raised this as an issue, including me. It is true that time and resources are limited, but how many times defects are missed because of unavailability of resources and how many time defects are missed because of not utilizing resources properly and faulty test strategy and design. I have seen it many times that resources spend time in creating work which is not adding value in any way. It could be in the form of writing detailed test plans, test cases, writing automation suite which becomes shelf-ware, munching numbers to prepare report for management and so on. Availability of time and resources is important, but also it is more important to have solid test strategy and design, prepared with application / project under test in mind.
10. Anyone can become tester, they should have ability to read and follow instructions, thats it.Testing is not a creative job and does not require special trainings or skills and thats why there are not many professional testers around.
This is one of the most damaging myth of all and to some extent this is because of some of the practices we have seen in our industry. Manual scripted testing is probably closest to unskilled job, which require minimal skill and probably very basic training. Everything else apart from that, right from test design, to test execution to automation is highly skilled and creative job and can be done effectively, only if you are skilled. Not considering testing as a skilled profession has done more harm to the testing community than any other myth. This myth is going away with the rise / recognition of testing as a separate skill, exploratory testing practices, Agile and sensible test automation but still there is a long way to go.
This was my list of myths and is by no means is a complete list. Do leave your comments if you have observed / come across myths which are not covered here or if you do not agree with anything I said.
