Testing in the e-Commerce World

When hot Web sites go down, the results are immediate and the public is often unforgiving. For example, Charles Schwab estimates that it costs approximately $1,000,000 a minute for a system outage. Of course, this does not include the soft dollar costs associated with the loss of customer confidence, bad press and general disruption caused by such problems. As this example shows, Web-enabled businesses are open to the world so the implications for a system failure are vast. What can help prevent these problems? The answer: testing.

Web-enabled systems are developed in much the same way as traditional computer systems, with several important caveats. One, the time pressure to develop and implement e-commerce systems is tremendous since time is of the essence. Two, e-commerce systems are often developed without systems documentation so that software developers have more leeway in the creation of the system. Unfortunately, the lack of documentation may result in more confusion. Three, the technical configuration of the systems may be far more complicated than that of traditional mainframes. As a result, e-commerce systems may have many more different pieces that need to be both individually set-up and integrated into a unified system. Four, e-commerce is a 24 by 7 operation---the systems must be available every day, all day.

While the complexity of the technical issues and the speed with which work needs to be completed may appear overwhelming, testing can help ensure, if not guarantee system success. Testing involves a series of activities to test the functionality of the system, its ability to perform under high volumes, and the integrity and consistency of the overall design. Finally, there is a testing stage that integrates all of these different activities to ensure that the overall “sum of the parts” actually works.

Functional or software quality testing evaluates whether the software works according to the specifications of the business needs. Traditionally, software was developed according to a structured process during which the business users developed requirements documentation and reviewed prototype systems to determine if the system being developed met their needs. This was a structured, lengthy process and typically involved many user reviews and sign-offs. In the web world, this type of testing is even more complex since the systems are often “open.” An “open” system has more flexibility and also allows for a vast array of possible inputs, processes and outputs. Often, as a system becomes easier to use, it becomes more complex to test.

Another type of testing is performance or stress testing. Performance testing helps to prevent the “blow-up” of a web site when too many users try to use it at the same time; a problem that has plagued many an e-commerce business. Specifically, performance testing is the evaluation of a system’s performance and functionality as additional users access a web site simultaneously. Load testing helps to identify the number of concurrent users that can be efficiently handled under existing configurations, as well as aspects of the system’s architecture that may “break” under volume pressure. Once the “breaking points” are identified, the technical staff can take proactive steps to make the systems more robust. Of course, once these steps are taken, the performance tests should be repeated.

While functional and performance testing are critical for ensuring that an e-commerce system actually works, site integrity testing is important for ensuring the professional and consistent look and feel of a web-site. We have all experienced web sites that do not quite work, full of typos, inconsistent graphics and missing links. Site integrity testing tackles these issues by putting a web site through the paces. Site integrity testing acts as a thorough and rigorous editor who ensures that all of the I’s are dotted and t’s are crossed. In essence, site integrity testing is like putting the finishing touches on your package.

Evidently, these testing activities are quite complex and can require intensive resources. Testing tools have been around since the early days of legacy systems. Thorough testing may require the acquisition of a variety of tools and resources to use and interpret the tools. More recently, innovations in testing have begun to recreate testing to leverage the web’s capabilities and ease time pressures. For example, Superior IS recently launched a TSP, a testing service provider, to provide outsourcing testing services. Superior IS has refined a testing tool set that can tackle all aspects of testing, and accomplish all testing activities efficiently and cost-effectively. Some firms are providing new tool suites dedicated to web-based testing, while others are offering traditional outsource arrangements. Each of these alternatives has a laundry list of pros and cons. The decision regarding which type of testing tools and resources to chose depends largely on an e-commerce business’ organization, availability of resources and time constraints.