|
7 Pillars of Testing Wisdom
|
|
| Jeff Gainer |
As a software quality consultant, I spend my days listening, analyzing, teaching, and giving advice. But when the discussion turns to testing, my work turns from giving advice to debunking myths. So, I'll use this space to dispense with seven major myths about testing, and replace them with some common sense advice. I like to call these testing best practices the "Seven Pillars of .NET Testing Wisdom."
Pillar One: Testing should begin before coding. Don't relegate testing .NET applications to the last 10–20 percent of the development cycle, only to trim it back when the schedule begins to slip. Testing should take roughly the same amount of time as application development, and occur concurrently with it. Testing early and often provides the opportunity to correct otherwise costly mistakes.
Pillar Two: Test against measurable criteria. In a perfect world, testing begins at the requirements stage. As each requirement is defined, it should be evaluated: Is the requirement correct? Is it objective, measurable, and, therefore, testable? It's not enough, for instance, to specify that a certain action be "fast." You must establish performance criteria defining what "fast" means. Testing quality isn't a fuzzy concept of "goodness," "ease of use," or even "user-friendliness," but an objective measure against quantifiable criteria.
Pillar Three: Testing is a legitimate professional specialty. Testing isn't a training ground for new developers or a dumping ground for mediocre developers. Software testing is a discipline in its own right and requires unique skills, not the least of which is programming. Some of the best testers I've hired, trained, or worked alongside were programmers seeking a new challenge.
Pillar Four: Use real-world test environments. Although your team can create an application in a "clean" development environment, on a robust machine with all the essential libraries while running in administrator mode, it's unlikely it will run in such an environment. Although .NET promises to eliminate DLL conflicts, as a tester, I wouldn't count on it. Nor would I count on a reasonably fast Internet connection or a stable runtime environment. When testing, you need to consider the broad range of consumer configurations and other apps the client machine might have installed or be running. For browser applications, be sure to test a number of browser brands and versions running under different operating systems.
Pillar Five: Security concerns will continue to increase. Security violations and application failures are no longer a matter of inconvenience and time lost while bugs are fixed. Now that the Internet and increasingly cheaper hardware and software have placed computers into the mainstream of daily life, application failures, security violations, worms, and viruses have become major news items, and rightly so. With the availability of greater functionality and interconnectivity comes the responsibility of building security into the code at all levels of a system.
Pillar Six: Approach automated testing tools with caution. Establishing a successful automated testing project requires time, a long-term commitment, a dedicated budget, then still more time. Unless you're in it for the long haul, I wouldn't even consider automated testing. Automated testing tools won't magically create effective tests or correct slipped schedules. The fact is automating testing takes far more time than manual testing initially, and the benefits are cumulative and realized only over time.
Pillar Seven: Complexity is here to stay: It's the insidious reality of continually improved development tools and deployment environments: .NET will make testing more complex, not less. Although new technologies have made it easy to slap together a quick prototype prior to any intensive design, this convenience makes it equally easy to construct a poorly designed, fundamentally flawed system.
The solution is arming yourself with caution and skepticism. Test your prototypes and applications for basic flaws early on, particularly in the areas of security and performance. I don't believe anyone when he or she tells me we can test later, when more of the "real" application is built, and you shouldn't either.
About the Author
Jeff Gainer is a principal partner at Darwin Partners, where he specializes in software process improvement consulting. He has written management and technical articles for numerous publications including Enterprise Development and Visual Studio Magazine. You can reach him by e-mail at gainerj@jeffgainer.com or read his previously published work on the Web at www.jeffgainer.com.
Back to top
|