The LLDB test suite consists of Python scripts located under the test directory. Each script contains a number of test cases and is usually accompanied by a C (C++, ObjC, etc.) source file. Each test first compiles the source file and then uses LLDB to debug the resulting executable. The tests verify both the LLDB command line interface and the scripting API.
Running the full test suite
Windows Note: In the examples that follow, any invocations of
should be replaced with
python_d, the debug interpreter, when running the test
suite against a debug version of LLDB.
The easiest way to run the LLDB test suite is to use the check-lldb build target. By default, the check-lldb target builds the test programs with the same compiler that was used to build LLDB. To build the tests with a different compiler, you can set the LLDB_TEST_COMPILER CMake variable. It is possible to customize the architecture of the test binaries and compiler used by appending -A and -C options respectively to the CMake variable LLDB_TEST_USER_ARGS. For example, to test LLDB against 32-bit binaries built with a custom version of clang, do:
> cmake -DLLDB_TEST_ARGS="-A i386 -C /path/to/custom/clang" -G Ninja
> ninja check-lldb
Note that multiple -A and -C flags can be specified to LLDB_TEST_USER_ARGS.
Note that on NetBSD you must export LD_LIBRARY_PATH=$PWD/lib in your environment. This is due to lack of the $ORIGIN linker feature.
Running a specific test or set of tests
In addition to running all the LLDB test suites with the "check-lldb" CMake target above, it is possible to run individual LLDB tests. For example, to run the test cases defined in TestInferiorCrashing.py, run:
> cd $lldb/test
> python dotest.py --executable <path-to-lldb> -p TestInferiorCrashing.py ../packages/Python/lldbsuite/test
If the test is not specified by name (e.g. if you leave the
-p argument off), LLDB will run all tests in
> python dotest.py --executable <path-to-lldb> functionalities/data-formatter
Many more options that are available. To see a list of all of them, run:
> python dotest.py -h
The dotest.py script runs tests in parallel by default.
To disable the parallel test running feature, use the
--no-multiprocess flag. The number of
concurrent tests is controlled by
LLDB_TEST_THREADS environment variable
--threads command line parameter.
The default value is the number of CPU cores on your
The parallel test running feature will handle an
--test-subdir SUBDIR arg. When
specified, SUBDIR is relative to the root test directory
and will limit all parallel test running to that
subdirectory's tree of tests.
The parallel test runner will run all tests within a given directory serially, but will run multiple directories concurrently. Thus, as a test writer, we provide serialized test run semantics within a directory. Note child directories are considered entirely separate, so two child directories could be running in parallel with a parent directory.
Running the test-suite remotely
Running the test-suite remotely is similar to the process of running a local test suite, but there are two things to have in mind:
You must have the
lldb-serverrunning on the remote system, ready to accept multiple connections. For more information on how to setup remote debugging see the Remote debugging page.
You must tell the test-suite how to connect to the remote system. This is
achieved using the
dotest.py. These parameters correspond to the
platform connectLLDB commands. You will usually also need to specify the compiler and architecture for the remote system.
Currently, running the remote test suite is supported only with
dosep.py with a single thread), but we
expect this issue to be addressed in the near future.
Debugging test failures
On non-Windows platforms, you can use the
-d option to
dotest.py which will cause the script to wait
for a while until a debugger is attached.
On Windows, it is strongly recommended to use Python Tools for Visual Studio for debugging test failures. It can seamlessly step between native and managed code, which is very helpful when you need to step through the test itself, and then into the LLDB code that backs the operations the test is performing. A quick guide to getting started with PTVS is as follows:
- Install PTVS
Create a Visual Studio Project for the Python code.
- Go to File -> New -> Project -> Python -> From Existing Python Code.
llvm/tools/lldbas the directory containing the Python code.
When asked where to save the
.pyprojfile, choose the folder
llvm/tools/lldb/pyproj. This is a special folder that is ignored by the
.gitignorefile, since it is not checked in.
test/dotest.pyas the startup file
Make sure there is a Python Environment installed for your distribution. For example, if you installed Python to
C:\Python35, PTVS needs to know that this is the interpreter you want to use for running the test suite.
- Go to Tools -> Options -> Python Tools -> Environment Options
- Click Add Environment, and enter
Python 3.5 Debugfor the name. Fill out the values correctly.
Configure the project to use this debug interpreter.
- Right click the Project node in Solution Explorer
- In the
Generaltab, Make sure
Python 3.5 Debugis the selected Interpreter.
Debug/Search Paths, enter the path to your
Debug/Environment Variables, enter
VCINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\
If you want to enabled mixed mode debugging, check
Enable native code debugging(this slows down debugging, so enable it only on an as-needed basis.)
Set the command line for the test suite to run.
- Right click the project in solution explorer and choose the
- Enter the arguments to
dotest.py. Note you must add
Example command options:
# quiet mode
# Path to debug lldb.exe
# Directory to store log files
-u CXXFLAGS -u CFLAGS
# If a test crashes, show JIT debugging dialog.
# Path to release clang.exe
# Path to the particular test you want to debug.
# Root of test tree
# Required in order to be able to debug the test.
As copy-pastable command line:
-q --arch=i686 --executable D:/src/llvmbuild/ninja/bin/lldb.exe -s D:/src/llvmbuild/ninja/lldb-test-traces -u CXXFLAGS -u CFLAGS --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe -p TestPaths.py D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --no-multiprocess
- Right click the project in solution explorer and choose the