Test your code! Debug your code! Trace your code! We all hear about how we should or at least could perform these activities with our programs. And all too often the terms and usages seem to blur, and be confused. In reality, these three activities – test, debug, trace – are all quite different in nature and serve different purposes.
This blog entry offers my perspective on how they differ. I hope you find it useful and I look forward to your comments.
SF
Testing
When you test a program, you run test code that exercises the program. That test code then either tells you whether or not the test succeeded -- or you have to manually go through the results of the test code and deduce for yourself if it worked.
The more automated you can make the testing process the better. Here are the options I know of for automated testing in the world of PL/SQL:
- Quest Code Tester for Oracle: [disclaimer: I designed this tool, I led the development team, and wrote lots of its backend. It is also sold by my employer, Quest Software.] This tool offers the highest level of test automation in the Oracle tools market. Describe the tests you need to run in a graphical interface, and Quest Code Tester then generates the test code, and runs the test automatically reporting all the results. For more information: http://www.quest.com/code-tester-for-oracle
- utPLSQL: this open source framework is modeled after the Extreme Programming unit testing principles. Think "Junit for PL/SQL". I wrote it back in 1999 and 2000 (it has been enhanced further since then) and it has for years been just about your only option for automated unit testing. The problem with utPLSQL is that you still have to write the vast majority of the test code yourself. For more information: http://utplsql.sourceforge.net/
- PL/Unit: very similar to utPLSQL, PL/Unit predefines a backbone of testing assertion routines which programs may then code to, taking some of the burden off the individual developer. For more information: http://www.apollo-pro.com/help/pl_unit.htm
A test identifies a problem in your code, but it usually doesn't give you a whole lot more information than "You passed in 16 and should have gotten back 12. Instead you got 10." You then need to figure out what lines of code are causing that problem. Both tracing and debugging can help you do that.
Tracing
When you trace the execution of your program, you run that program and as it is running, it records or builds a trace of information about what was happening inside the program.
In Oracle PL/SQL, the most common (and also the most crude) type of tracing is a call to DBMS_OUTPUT.PUT_LINE, which sends information to the screen after the program has finished running.
It is not at all rare to come across PL/SQL applications that are littered with calls to DBMS_OUTPUT.PUT_LINE. I generally avoid ever making a direct call to this built-in. For example, in the Quest Code Tester backend I use a tracing utility adapted from the Quest CodeGen Utility qd_runtime package to allow the user to flexibly turn on and off tracing, and write information to the qu_log table. From there, I can extract the data to display on the screen, fill up a file, etc. Here is a simple example from the Quest Code Tester backend:
IF qu_runtime.trace_enabled
THEN
qu_runtime.trace ('get_attribute_ex datatype guid'
, ret_attr.test_element.data_type_guid
);
END IF;
Notice that we call the trace_enabled function to determine if tracing is on before calling the trace subprogram. Qu_runtime.trace will check to see if tracing is enabled but by that time it will have also evaluated the arguments. To achieve the lowest possible runtime overhead, we call trace_enabled, which simply checks the value of a Boolean flag.
We then allow the user to specify when to turn on tracing with a little GUI widget like this:
Conditional compilation, a feature added to PL/SQL in Oracle Database 10g Release 2, can come in very handy with tracing because you can use "CC" to automatically compile the trace logic out of your programs when you don't want it showing up and possibly slowing down the application.
Debugging
Debugging is not the same thing as tracing and it definitely should not be confused with testing. You will generally start a debugging session after your tests have identified a bug. You might have also turned on tracing to get a "dump" of information about the overall flow of the application leading up to the bug.
With this "raw data" (specific test case failure information and trace data), you then need to find the specific lines of code that are causing the bug, and also figure how to change those lines of code to fix the bug. Given the maturity of the PL/SQL IDE (integrated development environment) market, most serious editors including a visual source code debugger.
For example, in Toad, you click in the "gutter" of the editor next to the line on which you want execution to "break" or pause. You start up your program and it runs to the breakpoint. You can then examine the values of variables, change values (in some debuggers), then step through the code, line by line, examining the program flow and how data structures are affected.
PL/SQL source code debuggers are incredibly useful and some are very powerful flexible implementations. Make them a part of your toolset when searching out the causes of bugs in your code!
Summary
To sum it all up, when thinking about identifying and fixing bugs in your code, keep in mind the differences between testing, tracing and debugging:
- Test your code to identify bugs that occur under a certain set of circumstances (a "test case" with specific inputs to arguments in the program).
- Turn on tracing and obtain a usually substantial amount of "raw data" about what the program did as it ran.
- Use your debugger to isolate the specific lines of code that caused the bug (and correspond to the flow indicated by the trace data).
Then use your IDE to fix the bug, and start the cycle all over again!