How to Debug a C++ Crash in Linux

For anyone who has done C or C++ development in Linux, debugging crashes is a critical part of the job. In this post, I’ll start with the basics of getting stack traces using gdb and core files.
Debugging a Crash in gdb

When your program is crashing you need to know where the problem occurred.  Running your program interactively in a debugger, such as the GNU Debugger (gdb), will give you a stack trace when the crash occurs.  If you have compiled your program with debugging enabled (the -g compiler option), then you will get a meaningful stack trace.

Consider the following code that deliberately generates a segmentation fault:

#include <stdlib.h>
void causeAnError (char& a)
{
        char* p = NULL;
        a = *p;
}
int
main (int argc, char *argv[])
{
        char ch;
        causeAnError (ch);
        exit (0);
}

Compiled with “g++ -o debugTest main.cpp” gives the following output in gdb:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000400618 in causeAnError(char&) ()
(gdb) bt
#0  0x0000000000400618 in causeAnError(char&) ()
#1  0x000000000040063e in main ()<p>Program received signal SIGSEGV, Segmentation fault.</p>

That narrows it down to the function where the error occured, but doesn’t tell you the line number.  Adding the “-g” compiler option will not only give you the line number, but will even print out the offending line:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000400618 in causeAnError (a=@0x7fffffffe06f) at main.cpp:6
6  a = *p;
(gdb) bt
#0  0x0000000000400618 in causeAnError (a=@0x7fffffffe06f) at main.cpp:6
#1  0x000000000040063e in main (argc=1, argv=0x7fffffffe158) at main.cpp:13

Debugging a Crash Using Core Files
Now suppose you are testing your application on a different machine than where you write and compile your code.  If gdb is not available, you can configure your Linux system to generate a core file.  A core file is generated when a program terminates abnormally; it records the state of the registers and memory at the time of the termination.  Once you have a core file you can transfer it to your development machine and feed that into gdb. 

Most Linux systems have core file generation disabled by default.  To enable it, with no limit to the core file size, type “ulimit -c unlimited”.  When your program generates a segfault, it will produce a file called core in the same directory as your program.  On your development machine, run “gdb <program name> core”.  If the program was compiled with debug symbols, then the output will be similar to what you see when running the program interactively in gdb:

Core was generated by `./debugTest'.
Program terminated with signal 11, Segmentation fault.
#0  0x0000000000400618 in causeAnError (a=@0x7fffbb679def) at main.cpp:6
6  a = *p;