Automated Error Prevention (AEP) is a well-defined and highly accepted methodology for preventing errors. AEP helps you prevent errors in an automated fashion. This article describes how to apply AEP to WebSphere development.
So how does AEP fit into the software life cycle? If you look at the software life cycle illustrated in Figure 1, you will see that the life cycle circulates through a number of stages. At each stage, a variety of different error prevention practices work together to stop errors. These practices include coding standards enforcement, unit testing, integration testing, load testing, connectivity verification, and monitoring. When one of these practices exposes an error, that error is analyzed, then the process that allowed the error is modified so that it prevents the same type of error from recurring.
This creates a feedback loop in which each error found serves as a catalyst for process improvement and error prevention. For example, assume that load testing reveals a performance problem caused by a critical piece of Java code that uses String instead of StringBuffer for nonconstant strings. Using AEP, you would not only correct this specific piece of code, but also implement and automatically enforce a new coding standard that forbids use of that inefficient coding construct. In this way, the AEP feedback loop helps prevent an entire class of performance problems.
Do not use Beans.instantiate() to create new bean instances.
The complete list of standards, as well as the justification for each standard, is available at www-3.ibm.com/software/webservers/
appserv/ws_bestpractices.pdf.
Documenting Interfaces with Design by Contract
Another AEP practice, Design by Contract (DbC), prevents integration problems. Integration is an especially error-prone phase for business system projects. These projects typically involve a moderate amount of time developing code around the middleware - the heart of the system - then a great deal of time trying to ensure that the glue code, existing or purchased components, and other hardware and software interact successfully. Even if each part is solid in and of itself, problems in the interfaces between the components can cause system functionality problems - especially if the problem occurs in a critical or frequently used interface.
Design-by-contract involves formally documenting each unit's interface requirements and assumptions. When each unit's interfaces are clearly documented in a contract, developers are less likely to introduce errors by writing code that incorrectly interfaces with other units. In addition, if the code is compiled with a DbC-enabled compiler such as iContract or Parasoft's Jcontract, you receive automatic and immediate notification if interface problems arise during test executions. This allows you to perform an extensive amount of interface verification with a relatively minor investment of time and effort.
Figure 2 illustrates where contracts (represented by small squares) should be added to units (represented by circles). Ideally, contracts specify requirements that must be satisfied before the unit executes, after the unit executes, and (optionally) at various points during execution.
Building and Extending Test Cases
Another way to prevent errors in WebSphere is to create tests that expose errors at the unit, submodule, module, and system levels, and use those tests to build a comprehensive error prevention test suite. This test suite can automatically expose problems introduced by a system modification - even problems that are the result of strange, unexpected side effects.
One advantage of having such a test suite is that it allows you to expose each error as early as possible. Consequently, you can fix it when it is fastest, easiest, and cheapest to do so - as well as reduce the chance for it to cause additional bugs (from code interactions or code reuse).
An even greater benefit of this test suite is that it fuels the AEP feedback loop. Each time it exposes an error, it helps you identify an error-prone part of your development process. You can then prevent an entire class of errors by modifying the process so that it does not allow the same type of error to recur.
As Figure 3 shows, the error prevention test suite creation process can be a natural progression that begins with building unit tests, then extending those tests so that they verify the submodule, the module, and then the entire system. You start by creating test cases that verify a unit (for example, a Java class). Once the unit is verified, you have a set of test cases that essentially freezes the correct functionality of the unit; if a change introduces an error into the unit, the test cases will expose it. Next, you extend the unit test cases so that they verify the submodule, then the module, then the entire system. Because you are extending and reusing test cases, you create a test suite that will automatically alert you if a modification introduces a problem into any aspect of the system.
For a more detailed look at how to develop such a test suite, consider an example. Assume we are developing a business system that provides our partners with a way to view inventory and order supplies. The partners access the application through a browser, and then beans access the database to perform the requested operations and deliver the results to the partners' browsers.
The moment each bean is completed, we perform unit testing - including white-box and black-box testing - to verify that it is robust and correct. This involves creating a test harness and any necessary stubs for external resources, sending test inputs using a unit testing tool such as JUnit or Parasoft's Jtest, then verifying the outputs. This general unit testing process is illustrated in Figure 4. We perform this process for each bean we write. Because we have created test cases that freeze the correct behavior of each unit, we can instantly verify whether code modifications introduce unit-level problems by running the applicable tests. Thus, by collecting all of these test cases and running them on a regular basis (e.g., nightly), we establish the foundation of an error prevention test suite. In addition, the verified input/output relationships serve as known checkpoints that we can use to verify the success of our submodule, module, and system tests.
If any errors are exposed at the unit level, we not only correct the errors, but also use AEP to prevent classes of similar errors from recurring. For instance, if we found that an error occurred because we used assignment in an if statement condition when we should have used equality [i.e., we wrote if (a=b) when we should have written if (a==b)], we could create and enforce a coding standard that forbids the use of assignment in if statement conditions. We could then check this standard automatically as we write code to ensure that code containing this problem is never added to the source code repository and integrated into the system. This is just one of many possible ways that the AEP feedback loop can be used to prevent errors through process improvement.
We start extending the unit test cases when we have created enough beans to have a submodule. A submodule is a collection of units that are related, but not yet complete enough to be executed outside of a testing framework. As shown in Figure 5, the submodule in this example includes several related beans that are running on WebSphere, but are not yet accessible from a Web server. They will eventually connect to a database, but the database is not yet available.
To test this submodule, we combine and extend the available unit test cases. For instance, assume we have a set of unit test cases for Bean A, Bean B, and Bean C - three beans that work together. Each test case has a test input and an expected output. To build a submodule test case from available unit test cases, we use a tool such as JUnit to invoke Bean A with a unit test input, verify the output at Bean A's checkpoint, send that output to Bean B, verify the output at Bean B's checkpoint, send that output to Bean C, then verify the output at Bean C's checkpoint. Each resulting submodule test case is a combination of multiple unit test cases we previously created for each bean. We already know the correct reference values for each checkpoint as a result of our unit testing, so verifying the output at each step is simply a matter of checking whether the values achieved during the test execution match the expected values. In other words, we create new test cases by reusing and extending the checkpoints established during unit testing.
In addition, because the submodule is designed to send or retrieve data from the database, we want to verify whether that functionality operates correctly. The database is not yet available, but we can start verifying this functionality by configuring each submodule test case to write database calls to a file that serves as a database stub. This creates a new checkpoint that exposes incorrect database calls.
If submodule testing exposes an error, we correct the code, as well as apply AEP to determine which process allowed the error, then implement a new quality control measure that prevents the same type of error from recurring. For instance, assume that we find problems in the interface between units. If we did not already use design-by-contract to document and verify interfaces, we could improve our process by incorporating design-by-contract and then using automated tools to verify whether or not we were adding sufficient contracts to each unit. If we were already using design-by-contract, we could implement an additional process improvement measure, such as requiring additional verification checkpoints that compare actual data values to expected ones.
Once the database is connected to the submodule, we have a complete module and can start extending the submodule tests into module tests (see Figure 6). At this point, we have tests that span multiple units, and checkpoints at each unit as well as at the stubbed connection to the database. To extend these tests, we reuse the submodule tests that feed inputs through the units and check the values at the established checkpoints, and then we verify whether each of those tests performs the expected database operation. The database operation can be verified using SQL, a database verification tool such as DbUnit or Parasoft's DataRecon, or by checking the values collected by a JDBC sniffing tool.
If a module test exposes an error, we correct the code and implement AEP to prevent classes of similar errors from recurring. For instance, assume our tests reveal that application data is not being stored properly in the database. In this case, we would implement AEP by creating a range of test cases that verify whether code is correctly adding records to the database. We would also modify our testing process to require that each database-related test case explicitly verify whether the correct data is being stored in the database.
Once we can execute the system as a user would - in this case through a browser that interacts with our modules through a Web server - we can start extending our various module test cases into system test cases (see Figure 7). System-level tests should initiate and monitor simulated user transactions. These tests watch how a transaction travels through a system's interfaces, and then verify the result of those interactions. If the transaction functions correctly, all checkpoints will contain the expected values. Because each system-level transaction is a chain of complex interactions, I recommend that you focus on verifying one transaction at a time. A reasonable goal is to verify a representative sample of the most critical and popular user transactions, one at a time. At the least, your test suite should exercise every system entry point that the user interface provides and exercise a representative sample of the possible input types for each entry point.
To create system-level test cases that check the established checkpoints as well as the interface with the user, we extend the beginning and end of our module tests. First, we configure a test client to invoke the module with the previously tested inputs. The test client can be a Web testing tool or an actual user exercising the application from a browser. Because the test is using the previously tested inputs, we can verify data values throughout the system by checking whether the actual value at each checkpoint corresponds to the expected value.
In addition to verifying the existing checkpoints, we create additional checkpoints - one that verifies whether the module was invoked appropriately, and one that verifies the response returned to the client. For this application, verifying the response involves checking the page sent to the client after each test action. Verification might include content verification, code validation, accessibility verification, and so on. Tools such as Watchfire's Bobby, the W3C validators, and Parasoft's WebKing can automate this verification process.
By extending the unit test cases to span the entire system, we establish a test suite that remembers the results of every checkpoint we ever verified, and instantly notifies us when a system modification introduces a problem into any part of the system. Many methods of system testing focus solely on verifying whether a user action produced the expected result, but this is not always an accurate assessment of whether the desired operation executed correctly. Consider a case in which the user action is supposed to record user information into the database, then send the user a confirmation message. If our system tests check only whether the user action produced the expected confirmation page, we could miss critical problems such as an interface problem that prevents the data from being recorded in the database.
The test suite we created, with its extensive system of checkpoints, instead targets that problem as soon as it occurs. The problem can then be fixed immediately, while the modification is still fresh in the developer's mind and before the error has a chance to spawn additional errors as a result of code interactions and reuse. More important, the process that allowed the error could be immediately repaired to prevent the introduction of similar database-application interface problems throughout this project and future projects.
The same general strategy can be applied to create system-wide test suites for business systems with different architectures. For example, if your module interacts with a legacy system or other resource outside of the submodule, you would verify that resource as we verified the database in the example. If your system is deployed as a Web service, you would exercise it using a test client, verify internal checkpoints throughout the system, then verify the HTTP traffic response to check the result. The verification logistics vary based on system architecture, but the effect is the same: a comprehensive test suite that automatically exposes errors as soon as possible.
Conclusion
With a comprehensive test suite that immediately and automatically exposes errors, you can apply AEP to prevent errors in two ways:
