Advertisements
Many organizations are surprised to find that it is more expensive to do testing using tools. In order to gain benefits from testing tools, careful thought must be given for which tests you want to use tools and to the tool being chosen.
Anteater is a testing framework designed around Ant, from the Apache Jakarta Project. It is basically a set of Ant tasks for the functional testing of Web sites and Web services (functional testing being: hit a URL and ensure the response meets certain criteria). One can test HTTP parameters, response codes, XPath, regexp, and Relax NG expressions. Anteater also includes HTMLreporting (based on junitreport) and a hierarchical grouping system for quickly configuring large test scripts. When a Web request is received, Anteater can check the parameters of the request and send a response accordingly. This makes it useful for testing SOAP and XML applications.
The ability to wait for incoming HTTP messages is something unique to Anteater, which makes it especially useful when building tests for applications that use high level SOAP-based communication, like ebXML or BizTalk. Applications written using these protocols usually receive SOAP messages and send back a meaningless response. It is only later that they inform the client, using an HTTP request on the client, about the results of the processing. These are the so-called asynchronous SOAP messages, and are the heart of many high-level protocols based on SOAP or XML messages.
Written in Java, HttpUnit emulates the relevant portions of browser behavior, including form submission, Javascript, basic HTTP authentication cookies, and automatic page redirection, and allows Java test code to examine returned pages either as text, an XML DOM, or containers of forms, tables, and links.
jWebUnitis a Java framework which facilitates creation of acceptance tests for Web applications. It provides a high-level API for navigating a Web application combined with a set of assertions to verify the application's correctness. This includes navigation via links, form entry and submission, validation of table contents, and other typical business Web application features. It utilizes HttpUnit behind the scenes. The simple navigation methods and ready-to-use assertions allow for more rapid test creation than using only JUnit and HttpUnit.
BugKillais a tool set to create, maintain, execute, and analyze functional system tests of Web applications. Specification and execution of tests is automated for both the Web frontend and business logic layers. One goal is to integrate with existing frameworks and tools (an Eclipse Plugin exists)
The Grinder a Java load testing framework freely available under a BSD-style Open Source license, makes it easy to orchestrate the activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plugins. Most users of The Grinder do not write plugins themselves; they use one of the supplied plugins. The Grinder comes with a mature plugin for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded.
Jameleonis an automated testing tool that separates applications into features and allows those features to be tied together independently, creating test cases. These test cases can then be data-driven and executed against different environments. Jameleon breaks applications into features and allows testing at any level, simply by passing in different data for the same test. Because Jameleon is based on Java and XML, there is no need to learn a proprietary technology.
It's an acceptance testing tool for testing the functionality provided by applications, and currently supports the testing of Web applications. It differs from regular HttpUnit and jWebUnit in that it separates testing of features from the actual test cases themselves. If I understand it correctly, you write the feature tests separately and then script them together into a reusable test case. Incidentally, you can also make these test cases data-driven, which gives an easy way of running specific tests on specific environments.
The framework has a plugin architecture, allowing different functional testing tools to be used, and there is a plugin for testing Web applications using HttpUnit/jWebUnit. The test case scripting is done with XML and Jelly.
Jameleon combines XDoclet, Ant and Jelly to provide a potentially powerful framework for solid functional testing of your Webapp. It strikes a good balance between scripting and coding, and allows you to set up multiple inputs per test by providing input via CSV files. Along with the flexibility come a complexity and maintenance overhead, but you are getting your Webapp tested for you.
LogiTestis the core application in the LogiTest suite. LogiTest is designed to aid in the testing of Web site functionality. It currently supports HTTP and HTTPS protocols, GET and POST methods, multiple document views, custom headers, and more. The LogiTest application provides a simple graphical user interface for creating and playing back tests for testing Internet-based applications.
Solex is a set of Eclipse plugins providing non-regression and stress tests of Web application servers. Test scripts are recorded from Internet browsers, thanks to a built-in Web proxy. For some Web applications, a request depends on a previous server's response. To address such a requirement, Solex introduces the concept of extraction and replacement rules. An extraction rule tied to an HTTP message's content will bind an extracted value with a variable. A replacement rule will replace any part of an HTTP message with variable content.
The tool therefore provides an easy way to extract URL parameters, Header values, or any part of a request or a response, bind their values with variables, and then replace URL parameters, Header values, or any part of a request with the variable content. The user has the ability to add assertions for each response. Once a response has been received, all assertions of this response will be called to ensure that it is valid. If not, the playback process is stopped. Several kinds of rules and assertions are provided. The most complicated ones support regular expressions and XPath.
Tclwebtest is a tool for writing automated tests of Web applications in Tcl. It implements some basic HTML parsing functionality to provide comfortable commands for operations on the HTML elements (most importantly forms) of the result pages.
Tagunit is a framework through which custom tags can be tested inside the container and in isolation from the pages on which they will ultimately be used. In essence, it's a tag library for testing tags within JSP pages. This means that it is easy to unit test tags, including the content that they generate and the side effects that they have on the environment, such as the introduction of scripting variables, page context attributes, cookies, etc.
Web Form Flodder is a Java console utility that analyzes a Web page, completes any forms present on the page with reasonable data, and submits the data. It crawls links within the site in order to identify and flood additional forms that may be present. It is great for load testing of Web forms, checking that all links work and that forms submit correctly.
XmlTestSuite provides a powerful way to test Web applications. Writing tests requires only knowledge of HTML and XML. The authors want XmlTestSuite to be adopted by testers, business analysts, and Web developers who don't have a Java background. XmlTestSuite supports "test-driven development". It lets you separate page structure from tests and test data. It can also verify databases. It's like JWebUnit, but has simple XML test definitions and reusable pages. The problems with raw HTTPUnit or JWebUnit are that:
- It's very hard to get non-programmers to write tests.
- Tests are so ugly you can't read them. (Trust me; HttpUnit test classes are a nightmare to maintain.)
- Web tests generally change far more often than unit tests, and so need to be altered, but your refactoring won't change them automatically (i.e., changing a JSP in IDEA will not cascade to the test like changing a class will).
