The goal of debugging and performance optimization is to create code that runs as correctly and quickly as possible. Debugging is the process by which programming errors are found and corrected. Performance optimization is the analysis and improvement of the algorithms and methods implemented in the code.
In this session, we will use examples in either C, C++ or Fortran90.
Choose your preferred language of the three and download the files to
be used in this session by either clicking one of the following three
links: C version
, C++
version
, F90 version
, or by copying the relevant files on
ManeFrame with one of the following 3 commands:
$ cp /hpc/examples/workshops/hpc/session9_c.tgz .
$ cp /hpc/examples/workshops/hpc/session9_cxx.tgz .
$ cp /hpc/examples/workshops/hpc/session9_f90.tgz .
Unpack your tarball and enter the resulting directory.
In most compilers (including GNU and PGI), you can enable debugging
information through adding the -g
compiler flag. Add this flag to
the compilation commands in the Makefile
for the target
driver2.exe
, and then compile the executable,
$ make driver2.exe
Run the new executable. It should die with an error message about a segmentation violation (segmentation fault) or bus error, depending on the compiler/OS, e.g.
$ ./driver2.exe
Segmentation fault
There are many ways to track down this kind of error (e.g. adding print statements everywhere, staring intently hoping for an epiphany, randomly changing things to see what happens). In this session we will use the most efficient debugging approach, that of using a tool to track down the bug for us.
The tool we will use is the GNU debugger, which can be accessed
through running the faulty executable program from within the
debugging program itself. Load the executable into gdb
with the
command
$ gdb driver2.exe
At the gdb
prompt, type run
to start the executable. It will
automatically stop at the line where the segmentation fault occurs.
In another terminal window, you can type man gdb
to learn more
about how to use the debugger (or you can click here to view the gdb
man page on the web.
Perhaps the most valuable gdb command is print
that may be used
to see the internal value of a specified variable, e.g.
(gdb) print i
will print out the current value of the iteration variable i
).
The help
command inside of gdb
may be used to find out more
information on how to use the program itself.
The quit
command inside of gdb
will exit the debugger and
return you to the command line. Alternatively, you may just type
^d
([control]-[d]) to exit.
driver2.c
and tridiag_matvec.c
,
and see if you can find/fix the problem by using gdb
and print
statements as appropriate.driver2.cpp
and
tridiag_matvec.cpp
, and see if you can find/fix the problem by
using gdb
and print
statements as appropriate.driver2.f90
and
tridiag_matvec.f90
, and see if you can find/fix the problem by
using gdb
and print
statements as appropriate.A word of warning, the location of the segmentation fault or bus error is not always where the problem is located. Segmentation faults generally occur due to an attempt within the program to read to or write from an illegal memory location, i.e. a memory location that is not a part of a currently-available variable. Examples of bugs that can cause a seg-fault are iterating outside of the bounds of an array, or a mismatch between the arguments that a program uses to call a function and the arguments that the function expects to receive.
Note
Tips for tracking/fixing segmentation faults
Using a debugger:
determine exactly the line of code causing the fault,
if the fault is inside a loop, determine exactly which iteration of the loop is causing the fault,
use print statements in the debugger to see which variable is
uninitialized, e.g. to see if the array x
has entry i
you could use
(gdb) print x[i]
Once you identify the precise location of the segmentation fault, go back to see where the data is allocated. Was it allocated with a different size, shape or type? Was it not allocated at all?
If the data is allocated in a different manner than it is being used, determine which location needs fixing and try your best.
Upon finding and fixing the bug causing the segmentation fault, the correctly-executing program should write the following line:
2-norm of product = 1.414213562373E+00
(or something within roundoff error of this result), and it should
write the file r.txt
that contains the result of the matrix-vector
product. This output vector should contain all 0’s except for the
first and last entries, which should be 1.
There are many freely-available Linux debugging utilities in addition
to gdb. Most of these are
graphical (i.e. point-and-click), and in fact use gdb
under the
hood. Some of the more popular of these debuggers include: ddd, nemiver, eclipse, zerobugs, edb.
However, of this set the ManeFrame cluster currently only has gdb
installed (ask your system administrators for others you want/need).
Additionally, there are some highly advanced non-free
Linux debugging utilities available (all typically graphical),
including TotalView, DDT, idb (only works
with the Intel compilers), and PGI’s pgdbg (graphical) and pgdebug
(text version). Of these, the ManeFrame cluster has both pgdbg
and pgdebug
.
The usage of most of the above debuggers is similar to gdb
, except
that in graphical debuggers it can be easier to view the
data/instruction stack. The primary benefit of the non-free debuggers
is their support for debugging parallel jobs that use OpenMP,
MPI, or hybrid MPI/OpenMP computing approaches (see session 9). In
fact, some of these professional tools can even be used to debug code
running on GPU accelerators.
If you’re interested in learning more about these, I recommend that
you re-download the tarball for this session, load the pgi
module,
update the Makefile to use the -g
option along with the relevant
PGI compiler (pgcc
, pgc++
or pgfortran
), and launch the
job in the pgdbg
debugger like you did with gdb
:
$ pgdbg ./driver2.exe
Press the “play” button to start the executable running, and use the mouse to interact with the debugger as needed.
Note
SMU pays for a five-seat PGI license, meaning that only five distinct compilation/debugging processes with the PGI tools may be run simultaneously. Typically, five is much more than sufficient for a campus of our size, since users spend most of their time writing code, preparing input parameters and scripts for running simulations, or post-processing simulation data; the time spent actually compiling and using a debugger is minimal. However, if everyone in the workshop tries this simultaneously, we would obviously exceed the five “seats,” which is why this is left as a personal exercise.