Testing Process
Biraj Rath, Author, NIIC (USA), Inc.
Raj Nath, Program Manager, Microsoft Corporation
Mukesh Agarwal, Program Manager, Microsoft Corporation
Jas Lamba, Program Manager, NIIC (USA), Inc.
George Gianopoulos, Reviewer, Microsoft Corporation
Laura Hargrave, Contributor, Microsoft Corporation
July 2000
Version 0.8
Summary: When setting up an e-commerce Web site on the Internet, rigorous testing is crucial to the implementation and maintenance of a reliable system that will create customer confidence. This article discusses the need for testing and the various types of tests that should be performed, including security testing, software and hardware reliability, and compatibility between all the elements of the system. (20 printed pages)
Contents
Introduction Setting Up an E-Commerce System The Need for Testing Types of Testing for E-Commerce Testing Web Tier Testing Middle Tier Testing Data Tier Other Tests Conclusion
Introduction
This paper discusses the essential testing stages that are conducted during the implementation of an e-commerce solution and its maintenance. It also covers specific types of e-commerce testing and defines their stages.
This paper is targeted to the developer and system architect. The reader should be familiar with Microsoft® Windows® DNA architecture and have a thorough knowledge of testing terminology, such as Black Box testing, White Box testing, Unit testing, and Functional testing.
Setting Up an E-Commerce System
E-commerce Web sites are not easy to set up. With a plethora of e-commerce solutions in the market, entrepreneurs have to make a few key decisions:
- The entrepreneur has to decide on the initial amount of investment required for an e-commerce Web site, as well as the volume of business of an e-commerce Web site over the Internet. Investment factors and business objectives dictate the type of software, database, or other applications that are required to set up the e-commerce Web site.
- There are specific elements involved in an e-commerce system. These elements range from domain name for the site to the merchant account for e-commerce transactions. Each of these elements requires a certain amount of scrutiny before setting up an e-commerce Web site.
- Before launching the e-commerce Web site on the Internet, it requires rigorous testing. Some of the important and common types of testing include security testing, software and hardware reliability, and compatibility between all the elements of the system.
The Need for Testing
Testing is the process of running a system with the intention of finding errors. Testing enhances the integrity of a system by detecting deviations in design and errors in the system. Testing aims at detecting error-prone areas. This helps in the prevention of errors in a system. Testing also adds value to the product by conforming to the user requirements.
Causes of Errors
The most common causes of errors in an e-commerce system are:
- Communication gaps between the developer and the business decision maker
- Time provided to a developer to complete the project
- Over commitment by the developer
- Insufficient testing and quality control
- Inadequate requirements gathering
- Introducing complex business processes into an e-commerce system
- Keeping pace with the fast changing e-commerce technology
Communication gaps between the developer and the business decision maker
A communication gap between the developer and the business decision maker is normally due to subtle differences between them. The differences can be classified into five broad areas:
- Thought processes
- Background and experience
- Interests
- Priorities
- Language
For example, an entrepreneur with a financial marketing background wants to set up an online share-selling site. The developer with an engineering background might be unaware of the intricate details of the financial market. Therefore, the developer may not know to incorporate those features that can add value to other site visitors into the site, such as the prices of mutual funds.
Time provided to a developer to complete the project
A common source of errors in projects comes from time constraints in delivering a product. At best, a project schedule provides an educated guess that is based on what is known at the time. At worst, a project schedule is a wish-derived estimate with no basis in reality.
Assuming the best, previously unknown problems may present themselves during development and testing. This can lead to problems maintaining the schedule. To keep to the schedule, features can be cut. To keep the features, the schedule can be slipped. Failing to adjust the feature set or schedule when problems are discovered can lead to rushed work and flawed systems.
Over commitment by the developer
High enthusiasm can lead to over commitment by the developer. In these situations, developers are usually unable to adhere to deadlines or quality due to the lack of resources or required skills on the team.
Insufficient testing and quality control
Insufficient testing is also a major source of breakdown of e-commerce systems during operations, as testing must be done during all phases of development.
Inadequate requirements gathering
A short time to market results in developers starting work on the Web site development without truly understanding the business and technical requirements. For example, the intricacies of state tax calculation can result in shipping costs that may not be completely understood and therefore calculated improperly. Also, developers may create client-side scripts using language that may not work on some client browsers.
Introducing complex business processes into an e-commerce system
Modeling business processes and human interaction into an e-commerce system can be a complex task. As more business processes are automated, the complexity of the system increases and leads to an increased chance of errors.
Keeping pace with the fast changing e-commerce technology
New technologies are constantly introduced. There may not be adequate time to develop expertise in the new technologies. This is a problem for two reasons. First, the technology may not be properly implemented. Second, the technology may not integrate well with the existing environment.
Objectives of Testing
Testing is essential because of:
- Software reliability
- Software quality
- System assurance
- Optimum performance and capacity utilization
- Price of non-conformance
Software reliability
E-commerce requires software that performs critical tasks, such as creating storefront and a shopping cart, collecting customer data, and providing the payment gateway. This software needs to function correctly.
Testing assures the organization of the quality and integrity of the e-commerce solution.
Software quality
Software quality is characterized by the correctness of program logic and implementation. It begins with testing the software during development.
The developer must test each module to make sure that it functions correctly at the time it is written or modified. Test values and boundary conditions must both be verified. Next, the module should undergo interface testing to check for functional errors. Only after the module works correctly can it be released for testing to the larger system. Early detection of errors saves rework and prevents a problem from becoming more complex in nature. As a result, error detection during the operation of a system incurs greater direct and indirect costs.
At a higher level, the interaction of individually correct components must be tested. For example, if a customer enters the details of their credit card payment and are disconnected before the order confirmation, the software must indicate the status of the transaction when the customer reconnects to the e-commerce site. If the software functions otherwise, it does not meet organization requirements.
Another instance of the quality of software is that of accurate tax and shipping calculations. Because all states have different tax systems and some of them are complex, it becomes difficult for the developer to integrate all the tax structures with multi-location shipping. This raises the complexity of the software and increases the chance of errors.
System assurance
The main purpose of system assurance is to deliver a quality product. Conformance to requirements increases the organization’s confidence in the system.
An e-commerce system deals with three parties: the bank, the transaction clearinghouse, and the customer. The interdependency of these three parties makes the process of buying and selling over the Internet more critical than in real life. If the faith of any of these parties dwindles in the e-commerce site, the entrepreneurs can lose a lot of money, as well as their reputation.
For example, in the case of a faulty e-commerce system, the credit card of the customer may be billed immediately for the complete order, when only a partial order has been filled. Testing must assure that partial order fulfillment and billing are done correctly.
Optimum performance and capacity utilization
Another purpose of testing is to ensure optimum performance and capacity utilization of e-commerce system components. The purpose of stress or capacity testing/planning is to make sure that the Web site is able to perform acceptably at peak usage.
For example, during the Christmas shopping season the Web site loads increase significantly. To handle this, the e-commerce solution must be able to handle the anticipated load with minimal degradation.
Price of non-conformance
The main purpose of testing is to detect errors and error-prone areas in a system. Testing must be thorough and well planned. A partially tested system is as bad as an untested system. And the price of an untested and under-tested system is high.
The following list suggests some of the potential fallouts of an untested or under-tested e-commerce system:
- Legal suits against the entrepreneur due to a faulty transaction system. This type of system may not have been tested for transaction software functionality. Losses include millions of dollars, not to mention the entrepreneur’s reputation.
- Loss of critical data from the database can result in untraceable transactions. This could again invite legal action and losses due to the digression of site visitors to competitors’ sites.
- Insecure transactions can cause losses to customers and may result in the withdrawal of certification by the security certification agency.
- System breakdown if the system has not been tested for load. A system breakdown results in loss of time due to restoration of service. Fixing the error can involve both direct and indirect costs.
Testing Team
The testing team needs to be carefully selected. An inexperienced testing team may not be able to test the system thoroughly.
A tester should be familiar with the technology, the business, and the customer requirements. Testing on Microsoft Windows DNA requires that the tester is proficient with the three layers of the architecture. They must be familiar with how the technologies interrelate.
For testing applications on the Web tier, the tester should have an understanding of the different browsers, such as Microsoft Internet Explorer, Netscape, AOL, and Microsoft WebTV®. The tester must also understand ASP, HTML, DHTML, Java, and VBScript because they are relevant to the Web tier. Testing applications on the middle tier requires the tester to be aware of business logic. This includes the Web server applications, tax and shipping calculations, discount, up-sell and down-sell mechanisms, and shopping cart functionalities.
The tester must be proficient with the database software, Microsoft SQL Server™ commands, and file and storage systems for testing applications on the data tier of Windows DNA.
For more information on staffing, see Identifying Staffing Roles for E-Commerce Operations.
Best Practices
While testing a system, follow these steps:
- Prepare comprehensive test plan specifications and test cases for each level of testing. Supplement these with the test data and test logs. Test plans for system testing may involve operators and test plans because acceptance testing involves customers.
- Design the test cases to test system restrictions, such as file and database size (stress testing).
- Develop the data to test specific cases. Copies of live files must not be used except for Acceptance testing.
- Do not use confidential data for testing without written authorization, especially in the case of Acceptance testing.
- Follow relevant standards.
- Perform Regression testing on each component of the system. This ensures that no anomalies have crept into the system because of the changes made to the system.
- Make sure to document and set up the test environment for each level in advance of testing. Test environments specify the preconditions required to perform the tests.
- Specify the intended test coverage as part of the test plan. Test coverage is the degree to which specific test cases address all specified requirements for a specific system or component.
White Box and Black Box testing techniques
| White Box Testing | Black Box Testing |
|---|---|
| Complete Path Testing | Equivalence Partitioning |
| Branch or Decision | Boundary Value Analysis |
| Condition Testing | Cause Effect Graphing |
| Data Flow Testing | Syntax Testing |
| Loop Testing |
When to Stop Testing
At the beginning of the testing process, plan on the amount of testing to be done. Base these plans on assumptions about the quality of the system to be tested.
Often, testing stops when the schedule runs out. A more efficient testing plan provides for a stop to testing when no further errors are found with existing tests, or when the number of errors found is below an acceptance threshold.
Challenges of E-Commerce Testing
E-commerce testing is not easy. With rapid changes in technology and improvement in hardware and software, the tester finds it difficult to standardize tools or techniques for e-commerce testing. The following are some of the challenges of e-commerce testing:
Rapid change of technology in e-commerce
Rapid change in e-commerce technologies keeps the developer and the tester on their toes. As newer hardware and software bring better functionality, their conditions for testing become different every time they change. This causes the tester to have to create new environments each time.
Varied customer profiles
Site visitors may vary from a beginning customer to a sophisticated customer. Therefore, the tester needs to simulate the actions of all kinds of customers to be sufficiently thorough in testing the e-commerce system. Simulating the actions poses great challenges to the tester.
Changes in the business environment, especially in terms of tax, shipping costs, multi-location delivery, and multi-location dispatch should also be simulated.
Creating a test environment for e-commerce
Creating a test environment for e-commerce applications is difficult because of the complexity of an e-commerce Web site and its interaction with the live world. This includes the Web clusters, middle-tier components, clustered database servers, firewalls, and so on. There is interaction with credit card companies, fulfillment houses, and customers. Therefore, simulating every possible action of the online customer is impossible because the tester cannot predict the actions of the site visitor. Also, the tester is unaware of the traffic intensity at peak times. Because the future of the site is unknown to the tester, the e-commerce Web site can be tested for certain standard functions but may not be tested for all contingencies.
Testing security
Hackers have no standard method of breaking into e-commerce sites, so there are no standard methods of security testing. Also, there are few tools available to test security aspects thoroughly.
Types of Testing for E-Commerce
The e-commerce testing process is divided according to the three tiers of Windows DNA architecture.
- Web Tier
- Web site content testing
- Web site testing
- Browser compatibility
- Middle Tier
- Software performance testing (business logic, tax, shipping calculations)
- Server load testing
- Data Tier
- Database testing
- Search options
- Query response time
- Data integrity
- Data validity
- Recovery testing
Specific tests that must be performed across the Windows DNA layers are:
- Security
- Regression testing
- User acceptance testing
- Unit testing
- System testing
Testing Web Tier
Web Site Content Testing
A Web site should be well planned, tested for errors, and finally presented to the online customers. Content testing is important to present a quality, error-free Web site.
Types of content testing
The site should be attractive and customer-friendly. It should have:
- Visual appeal
- No grammatical and spelling errors
- Reliable and consistent information
Visual appeal
The visual appearance of a Web site is important to maintain repeat visits. Although the home page of an e-commerce site is the "breadwinner," catalog pages cannot be ignored. Regardless of the developer’s choice for color, font, or graphics, the tester needs to test for the appearance of the site and bring out problem areas.
Tests required to check the visual appeal of a site are described below.
| What to Test | Environment | Tools/Technique |
|---|---|---|
| Fonts | User Environment | GUI Testing |
| Intensity of Colors | User Environment | GUI Testing |
| Graphics | Development Environment | GUI Testing |
| Audio and Video | Development Environment | GUI Testing |
| Grammar and Spelling | User/Development Environment | GUI/Proofreading/Spell check |
| Facts and Figures | User Environment | GUI Testing |
Browser compatibility for font style
There are a number of different fonts available on HTML editors these days. However, many of these fonts may not display on all browsers, especially on older versions. Or they may display as unreadable characters. Therefore, it’s important to test the browser for version compatibility.
Consistency of font size
Test for consistency of font size throughout the Web site. A body text font size of 10 to 14, and a heading font size of 18 to 24 are the norm.
Colors
Consider the combinations of foreground and background colors throughout the site. For example, it may be difficult to read yellow text on a white background.
Graphics
Fewer graphics on a Web page aid in faster downloads. As much as possible, thumbnails should replace photographs. Developers must test for download time of graphics-intensive pages.
Grammatical and spelling errors in the content
The home page requires special attention because it is the first page that the site visitor sees.
Use the spelling checker to check the spelling throughout the site. Sometimes there are errors that may not be checked by the spelling checker, such as “there” and “their.”
Finally, make sure to proofread the entire site to check the grammar.
Authenticity of facts provided
Verify all facts and figures that relate to products and services. The testing team can verify these with the legal, marketing, and business groups.
Web Site Testing
Proper functioning of hyperlinks
Hyperlinks in Web sites can be broken, missing, or improperly assigned. In all three cases, the site visitor is unable to navigate to the appropriate Web page. Therefore, check to make sure that all the links work properly.
| What to Test | Environment | Tools/Technique |
|---|---|---|
| Broken Links | User Environment | GUI Testing |
| Missing Links | User Environment | GUI Testing |
| Wrong Links | User Environment | GUI Testing |
Broken links
There may be times when the developer inadvertently changes the source folder of the graphics or the graphics file name while modifying the Web site. This causes a break in the link between the existing page and the linked page or the graphic. In this case, testing needs to done to detect broken links. For example, if a developer changes the name of the products page from "Product.htm" to "Products.htm," the link between the home page and the Products page breaks so that the Products page is no longer accessible from the home page.
Missing links
Links that have not yet been created are called “missing links.” For example, a developer might forget to link a button on the home page with the Products page. Consequently, site visitors would not be able to access the Products page from the home page.
Incorrect links
Incorrect links cause errors. These links can take a site visitor to the wrong page.
Testing hyperlinks helps to eliminate broken, missing, or wrongly assigned links. There is software available that helps to check broken and incorrect links. Missing links should be checked manually.
Also, there are Web sites that offer online testing for broken links at a nominal fee:
https://netmechanic.com
https://www.linkalarm.com/index.htmlhttps://www.Websitegarage.com
Browser Compatibility
After creating a Web page, browser compatibility testing begins. This is because text, graphics, or colors may appear differently on different browsers. To prevent these problems, developers choose software that is compatible with most of the popular browsers, such as Internet Explorer, Netscape, AOL, and WebTV.
The following table details the compatibility of browsers.
| Browser | Browser version |
ActiveX controls |
VBScript | JavaScript | Java applets |
Dynamic HTML |
Frames | CSS 1.0 | CSS 2.0 |
|---|---|---|---|---|---|---|---|---|---|
| Internet Explorer | 4.0 and later | Enabled | Enabled | Enabled | Enabled | Enabled | Enabled | Enabled | Enabled |
| Internet Explorer | 3.0 and later | Enabled | Enabled | Enabled | Enabled | Disabled | Enabled | Enabled | Disabled |
| Netscape Navigator | 4.0 and later | Disabled | Disabled | Enabled | Enabled | Enabled | Enabled | Enabled | Enabled |
| Netscape Navigator | 3.0 and later | Disabled | Disabled | Enabled | Enabled | Disabled | Enabled | Disabled | Disabled |
| Both Internet Explorer and Navigator | 4.0 and later | Disabled | Disabled | Enabled | Enabled | Enabled | Enabled | Enabled | Enabled |
| Both Internet Explorer and Navigator | 3.0 and later | Disabled | Disabled | Enabled | Enabled | Disabled | Enabled | Disabled | Disabled |
| Microsoft Web TV | Unavailable | Disabled | Disabled | Disabled | Disabled | Disabled | Disabled | Disabled | Disabled |
| What to Test | Environment | Tools/Technique |
|---|---|---|
| Font and Graphics position | User environment | GUI Testing |
| Resolution | User environment | GUI Testing |
| Support for different scripts and software | User environment | GUI Testing |
Examples of GUI tests
- Test each toolbar and menu item for navigation using the mouse and keyboard.
- Test window navigation using the mouse and keyboard.
- Test to make sure that proper format masks are used. For example, all drop-down boxes should be properly sorted. The date entry should also be properly formatted.
- Test that the colors, fonts, and font widths are to standard for the field prompts and displayed text.
- Test that the color of the field prompts and field background is to standard in read-only mode.
- Make sure that vertical scroll bars or horizontal scroll bars do not appear unless required.
- Test that the various controls on the window are aligned correctly.
- Make sure that the window is resizable.
- Check the spellings of all the text displayed in the window, such as the window caption, status bar options, field prompts, pop-up text, and error messages.
- Test that all character or alphanumeric fields are left-justified and that the numeric fields are right-justified.
- Check for the display of defaults if there are any.
- In case of multiple windows, check that they all have the same look and feel.
- Check that all shortcut keys are defined and work correctly.
- Check for the tab order. It should be from top left to bottom right. Also, the read-only/disabled fields should be avoided in the TAB sequence.
- Check that the cursor is positioned on the first input field when the window is opened.
- Make sure if any default button is specified, it should work properly.
- Check for proper functioning of ALT+TAB.
- Ensure that each menu command has an alternative hot key sequence and that it works correctly. (See Appendix B & Appendix C.)
- Check that there are no duplicate hot keys defined on the window.
- Validate the behavior of each control, such as push button, radio button, list box, and so on.
- Test to make sure that the window is modal. This will prevent the user from accessing other functions when this window is active.
- Test that multiple windows can be opened at the same time.
- Make sure that there is a Help menu.
- Check to make sure that the command buttons are grayed out when not in use.
Testing Middle Tier
Software Performance Testing
Software performance testing aims to ensure that the software performs in accordance with operational specifications for response time, processing costs, storage use, and printed output.
The data on software performance is gathered during:
- Current and expected normal transactions
- Current and expected peak transactions
- Minimal transaction volumes
All interfaces are fully tested. This includes verifying the facilities and equipment, and checking to make sure that the communication lines are performing satisfactorily.
| What to Test | Environment | Tools/Technique |
|---|---|---|
| Correct data capture | Development environment | Testing ASP, CGI scripts by Black Box and White Box technique, Boundary value analysis, and Equivalence partitioning. |
| Transactions completion | User/System test environment | Functional testing by simulating customer data. |
| Gateway software | Development environment | Functional testing by simulating customer data. |
| Tax and shipping calculations | User environment | Functional testing by random data, Boundary value analysis, and Equivalence partitioning. |
Correct data capture
Correct data capture refers to the use of CGI scripts or ASP to capture data from the Web client. This includes forms, credit card numbers, and payment details. Any error in capturing this data will result in incorrect processing of the customers’ orders.
Completeness of transaction
Transaction completeness is the most important aspect of e-commerce transactions. Any error in this phase of operation can invite legal action because the affected party may be at risk of losing money due to an incomplete transaction.
Gateway compatibility
The payment gateway consists of software installed on Web servers to facilitate payment transactions. The gateway software captures credit card details from the customer and then verifies the validity of the credit card with the transaction clearinghouse.
Gateways are complex because they can create compatibility problems. In turn, these problems make e-commerce transactions unreliable. So, the entrepreneur needs to consult experienced developers before investing in a payment gateway. Therefore, before launching the site, online pilot testing must be done to test the reliability of the gateway.
Tax and shipping calculations
Entrepreneurs have to grapple with multiple taxes and shipping rates. The problem becomes larger if the entrepreneur is catering to customers outside the country.
Some off-the-shelf software provides ready-made solutions to both problems simultaneously. This software is upgraded whenever tax structures change. However, the entrepreneur needs to regularly check with legal and tax consultants to keep track of tax and shipping rates. Testing needs to be done to ensure that the customer is charged the correct tax and shipping amount. Incorrect tax and shipping levies on the customer invariably result in losing customers to the competitors.
Server Load Testing
E-commerce sites that rely on a heavy volume of trading on the Internet need to make sure that their Web servers have a very high uptime. To prevent breakdown and to offload traffic from a server at peak time, entrepreneurs must invest in additional Web servers. The power of a Web server to handle a heavy load at peak hours depends on the network speed and the server’s processing power, memory, and storage space of the server. The hardware component of the Web server is most vulnerable at peak hours.
The number of simultaneous users that the server can successfully handle measures its capacity. Excessive load on the Web server causes it to degrade dramatically in performance until the load is reduced. The objective of this load testing is to determine an optimum number of simultaneous users.
Stress testing
Running the system in a high-stress mode creates high demands on resources and stress tests the system. Some systems are designed to handle a specified volume of load. For example, a bank transaction processing system may be designed to process up to 100 transactions per second; an operating system may be designed to handle up to 200 separate terminals. Tests must be designed to ensure that the system can process expected load. This usually involves planning a series of tests where the load is gradually increased to reflect the expected usage pattern.
Stress tests steadily increase the load on the system beyond the maximum design load until the system fails. This type of testing has a dual function:
- It tests the failure behavior of the system. Circumstances may arise through an unexpected combination of events where the load placed on the system exceeds the maximum anticipated load. Stress testing determines if overloading the system results in loss of data or user service.
- It stresses the system and may cause certain defects to come to light, which may not normally manifest the errors.
Stress testing is particularly relevant to an e-commerce system with Web databases. These systems often exhibit severe degradation when the network is swamped with operating system calls.
Load testing software
There is a lot of load-testing software.
The testing technique used by most software is to simulate multiple logons. After a series of these multiple logons, the software calculates the optimum load factor for the Web server. The Web server software is then configured using this test data. As a result, if the traffic increases beyond the load capacity of the Web server, the server stops entertaining further requests from online users. For more details on Web server load testing, see www.Webperfcenter.com.
The Microsoft® Web Application Stress (WAST) tool is designed to simulate multiple browsers that are requesting pages from a Web site. This tool can realistically simulate many requests with relatively few client machines. Make sure that you are using an adequate number of client machines.
Testing Data Tier
Database Testing
An e-commerce site typically stores catalogs, shopping baskets, user profiles, and order information in the database.
Location of a database
The database does not have to be on the same server on which the storefront is hosted. The database server can be separated from the Web server by a firewall. This adds complexity to the testing processes. Therefore, you do not have to perform accessibility, security testing, and performance testing.
Objectives of database testing
The purpose of database testing is to determine how well the database meets requirements. This is an ongoing process because no database is static. When a database is created, a mirror of the same database should be created and stored either on the same computer or another computer. The original database is left alone and its mirror image goes through the various tests. This process continues until the tests are successful so that the changes can be implemented in the original database.
Databases are tested for five reasons:
- Relevance of search results
- Query response time
- Data integrity
- Data validity
- Recovery
| What to Test | Environment | Tools/Technique |
|---|---|---|
| Relevance of search results | System test environment | Black Box and White Box technique |
| Query response time | System test environment | Syntax Testing /Functional Testing |
| Data integrity | Development environment | White Box Testing |
| Data validity | Development environment | White Box Testing |
Relevance of Database Search Results
The Search option is one of the most frequently used functions of online databases. Search results provide direct links to other pages, saving time and effort.
Many site visitors complain that Search results are not relevant or result in “wild goose chases.” Therefore, building relevance into database searches is an essential part of data handling.
Testing for Search relevance should be carried out by a team of people that are not a part of the development team. This team assumes the role of the online customer and tries out random Search options with different keywords. The Search results are recorded by the percentage of relevance to the keyword. At the end of the testing process, the team comes up with a series of recommendations. This can be incorporated into the database Search options.
Query Response Time
The query response time is essential in online transactions. The turnaround time for responding to queries in a database must be short. The results from this testing may help to identify problems, such as bottlenecks in the network, specific queries, the database structure, or the hardware.
Data Integrity
Important data stored in the database include the catalog, pricing, shipping tables, tax tables, order database, and customer information. Testing must verify the correctness of the stored data. Therefore, testing should be performed on a regular basis because data changes over time.
Examples of data integrity tests
- Test the creation, modification, and deletion of data in tables as specified in the functionality.
- Test to make sure that sets of radio buttons represent a fixed set of values. Check what happens when a blank value is retrieved from the database.
- Test that when a particular set of data is saved to the database, each value gets saved fully. In other words, the truncation of strings and rounding of numeric value does not occur.
- Test whether default values are saved in the database if the user input is not specified.
- Test the compatibility with old data. In addition, old hardware, versions of the operating system, and interfaces with other software need to be tested.
Data Validity
Errors caused due to incorrect data entry, called data validity errors, are probably the most common data-related errors. These errors are also the most difficult to detect in the system. These errors are typically caused when a large volume of data is entered in a short time frame. For example, $67 can be entered as $76 by mistake. The data entered is therefore invalid.
You can reduce data validity errors. Use the data validation rules in the data fields. For example, the date field in a database uses the MM/DD/YYYY format. A developer can incorporate a data validation rule, such that MM does not exceed 12, DD does not exceed 31.
In many cases, simple field validation rules are unable to detect data validity errors. Here, queries can be used to validate data fields. For example, a query can be written to compare the sum of the numbers in the database data field with the original sum of numbers from the source. A difference between the figures indicates an error in at least one data element.
Recovery Testing
Another test that is performed on database software is the Recovery test. This test involves forcing the system to fail in a variety of ways to ensure that:
- The system recovers from faults and resumes processing within a pre-defined period of time.
- The system is fault-tolerant, which means that processing faults do not halt the overall functioning of the system.
- Data recovery and restart are correct in case of auto-recovery. If recovery requires human intervention, then the mean time to repair the database is within pre-defined acceptable limits.
Other Tests
Security
Gaining the confidence of online customers is extremely important to e-commerce success. Building the confidence of online customers is not an easy task and requires a lot of time and effort. Therefore, entrepreneurs must plan confidence-building measures. Ensuring the security of transactions over the Internet ensures customer confidence.
The main technique in security testing is to attempt to violate built-in security controls. This technique ensures that the protection mechanisms in the system secure it from improper penetration.
The tester overwhelms the system by continuous requests, thereby denying service to others. The tester may purposely cause system errors to penetrate during recovery or may browse through insecure data to find the key to system entry.
There are two distinct areas of concern in e-commerce security: network security and payment transaction security. Types of security breaches in these areas are:
- Secrecy
- Authentication
- Non-repudiation
- Integrity control
Network security
Unauthorized users can wreak havoc on a Web site by accessing confidential information or by damaging the data on the server. This kind of security lapse is due to insufficient network security measures. The network operating system, together with the firewall, takes care of the security over the network.
The network operating system must be configured to allow only authentic users to access the network. Also, firewalls must be installed and configured. This ensures that the transfer of data is restricted from only one point on the network. This effectively prevents hackers from accessing the network.
For example, a hacker accesses the unsecured FTP port (Port 25) of a Web server. Using this port as an entry point to the network, the hacker can access data on the server. The hacker may also be able to access any machine connected to this server. Therefore, security testing will indicate these vulnerable areas and will also help to configure the network settings for better security.
Network security over the Internet is tested using programs. One such program for Microsoft Windows 2000 is the Kane Security Analyst (KSA) from Intrusion Detection Inc. KSA is a complete network-testing tool that also tests operating systems other than Windows 2000.
The KSA network security testing tool tests for:
- User rights
- Removable disk locations
- Strength of password policies
- Use of logon scripts and password expiration dates
- Storage of passwords in clear text or encrypted form
The KSA report manager generates several reports to check miscellaneous sets of security-related concerns. The software points out security loopholes only and does not trap unauthorized visitors.
Visit these links for more details:
Payment transaction security
Secure transactions create customer confidence. That’s because when customers purchase goods over the Internet, they can be apprehensive about giving credit card information. Therefore, security measures should be communicated to the customer.
Two things needed to be tested to ensure that the customer’s credit card information is safe. First, testing should ensure that the credit card information is transmitted and stored securely. Second, testing should verify that strong encryption software is used to store the credit card information, and only limited, authorized access is allowed to this information.
For more information on secure electronic transactions, see:
www.verisign.com
www.cylink.comwww.terisa.comwww.cybercash.comwww.checkfree.com
Acceptance Testing
Acceptance testing is performed on a collection of business functions in a production environment, and after the completion of Functional testing. This is the final stage in the testing process before the system is accepted for operational use. It involves testing the system with data supplied by the customer or the site visitor rather than the simulated data developed as part of the testing process.
Acceptance testing often reveals errors and omissions in the system requirements definition. The requirements may not reflect the actual facilities and performance required by the user. Acceptance testing may demonstrate that the system does not exhibit the anticipated performance and functionality. This test confirms that the system is ready for production.
Running a pilot for a select set of customers helps in Acceptance testing for an e-commerce site. A survey is conducted among these site visitors on different aspects of the Web site, such as user friendliness, convenience, visual appeal, relevance, and responsiveness.
Regression Testing
Regression testing refers to retesting previously tested components/functionality of the system to ensure that they function properly even after a change has been made to parts of the system.
As defects are discovered in a component, modifications should be made to correct them. This may require other components in the testing process to be retested.
Component system errors can present themselves later in the testing process. The process is iterative because information is fed back from later stages to earlier parts of the process. Repairing program defects may introduce new defects. Therefore, the testing process should be repeated after the system is modified.
Here are some guidelines to follow for Regression testing:
- Test any modifications to the system to ensure that no new problems are introduced and that the operational performance is not degraded due to the modifications.
- Any changes to the system after the completion of any phase of testing or after the final testing of the system must be subjected to a thorough Regression test. This is to ensure that the effects of the changes are transparent to other areas of the system and other systems that interface with the system.
- The project team must create test data based on predefined specifications. The original test data should come from other levels of testing and then it should be modified along with test cases.
Conclusion
Testing is an essential activity for e-commerce implementation. It ensures software reliability and system assurance. Each element involved in an e-commerce system goes through rigorous testing. This testing ensures a reliable e-commerce site that creates customer confidence.
The different types of testing are content testing, software and database testing, server load, user acceptance testing, and security testing. E-commerce testing is typically a process of iteration. After the developer fixes errors and bugs in the e-commerce system, the tester has to retest the system for any anomalous behavior due to these fixes.
The most crucial aspect of e-commerce testing is the test environment. E-commerce testing is challenging. Breaking up the testing tasks based on each of the tiers of the Windows DNA architecture helps to reduce the complexity of the testing task.
Acknowledgment
Dyson, Peter. Mastering Microsoft® Internet Information Server 4, Second Edition. Sybex, 1997.
Note This technical article is one in a series about applying Microsoft Enterprise Services frameworks to e-commerce solutions. E-Commerce White Paper Series contains a complete list, including descriptions, of all the articles in this series.