Manual Installation for Linux LLDB

Compilers

Install a clang compiler. You can choose to do this manually, but it's probably easier to just use your package manager. On Ubuntu based Linux systems:

sudo apt install clang-9

Ubuntu generally has several options for clang compilers, for instance on 19.10 there are clang-6, clang-7, clang-8, clang-9 and a default clang package. I elect to go with the newest available package. After installation:

export CC=clang-9
export CXX=clang++-9

Now in order to use various (optional) PETSc packages, we must also have an available fortran compiler. The de facto choice is gfortran:

sudo apt install gfortran
export FC=gfortran

Compiler flags

Now there are some subtleties here. At least on my system by default the clang compilers do not generate position-independent code or position-independent executables, e.g. by default clang is compiling as if with the -fno-PIC -fno-PIE flags. With these default settings, configuration of fortran support for MPICH will fail because of an inability to link objects generated by gfortran and objects generated by clang. To fix this we need to explicitly pass the -fPIC -fPIE flags:

export CPPFLAGS='-fPIC -fPIE'

At the time of writing, clang is known to have issues reading debug info from libstdc++. The most straightforward fix to this is to have clang institute its own debug information:

export CFLAGS='-fstandalone-debug'
export CXXFLAGS='-fstandalone-debug'

Note that we do not pass this option into CPPFLAGS because gfortran doesn't recognize the -fstandalone-debug option.

Build MPICH

With our compilers and compiler flags set, we can proceed to building our MPI compilers. The git command below assumes you have ssh keys setup on github. If you do not, then you can clone using the https protocol. Moreover, if you are building from the git sources, you will need autotools installed in order to run autogen.sh. You can install autotools using your package manager, or you can actually build autotools in libMesh, by cd'ing into the root of the libMesh source directory and executing ./bootstrap --build-autotools. You can also download an mpich tarball if you prefer to avoid the git and autotools stages.

cd $HOME
git clone git@github.com:pmodels/mpich.git
cd mpich
git submouled update --init --progress
./autogen.sh
mkdir build
cd build
../configure --prefix=$HOME/mpich/installed && make -j64 && make install

With the MPICH compilers built, we can export them to our PATH:

export PATH=$HOME/mpich/installed/bin:$PATH

Build PETSc

PETSc prefers to do most configuration detection on its own so we:

unset CC CXX FC CPPFLAGS CFLAGS CXXFLAGS

We are going to build PETSc using our MOOSE submodule script, so make sure you've cloned MOOSE. (See Cloning MOOSE)

cd $MOOSE_DIR
scripts/update_and_rebuild_petsc.sh --with-mpi-dir=$HOME/mpich/installed

Build libMesh

Before building libMesh, let's set compilers again. If you have ccache available in your environment:

export CC="ccache mpicc"
export CXX="ccache mpicxx"
export FC="ccache mpifort"

else:

export CC=mpicc
export CXX=mpicxx
export FC=mpifort

Now let's build!

scripts/update_and_rebuild_libmesh.sh --without-gdb-command

Build WASP

Building WASP is also necessary. With the compilers already set in the above step, we only need to run the following commands:

cd $MOOSE_DIR
scripts/update_and_rebuild_wasp.sh

Build MOOSE

I always prefer to make sure that all potentially stale object files have been removed from my MOOSE directory before trying to link with new libraries. If using git clean, make sure that you've checked-in any untracked files you want preserved. I'm also going to be passing TAGS as an exclusion pattern because I use etags a lot when developing.

git clean -fxd -e TAGS
cd test
METHOD=dbg make -j64

Installing and Using LLDB

This next set of instructions is very much Ubuntu 19.10 specific, but it may still be beneficial across platforms and across distribution versions. I built my stack using clang-9, so I'm going to use the corresponding lldb version:

sudo apt install lldb-9

If you try to use lldb-9 straight out of the box, you're probably going to see some troubling messages when you begin trying to debug:

$ lldb-9 -- ../../../moose_test-dbg -i simple_diffusion.i
(lldb) target create "../../../moose_test-dbg"
Current executable set to '../../../moose_test-dbg' (x86_64).
(lldb) settings set -- target.run-args  "-i" "simple_diffusion.i"
(lldb) b fe.C:138
Breakpoint 1: no locations (pending).
WARNING:  Unable to resolve breakpoint to any actual locations.
(lldb) r
Process 74552 launched: '/home/lindad/projects2/moose/test/moose_test-dbg' (x86_64)
61 locations added to breakpoint 1

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'lldb'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'run_one_line' is not defined
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'run_one_line' is not defined
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'run_one_line' is not defined
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'run_one_line' is not defined
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'run_one_line' is not defined
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'run_one_line' is not defined
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'run_one_line' is not defined

The python import errors are irritating, but more irritating perhaps is that if you attempt to print container information (like for a std::vector), you're going to get something ugly:

(lldb) p fe->_fe_map->JxW
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'run_one_line' is not defined
(std::vector<double, std::allocator<double> >) $1 = size=1 {
  std::_Vector_base<double, std::allocator<double> > = {
    _M_impl = {
      std::_Vector_base<double, std::allocator<double> >::_Vector_impl_data = {
        _M_start = 0x000000000069a500
        _M_finish = 0x000000000069a520
        _M_end_of_storage = 0x000000000069a520
      }
    }
  }
}

So there are several problems here stemming from lldb interactions with python. Let's debug. We can first check where lldb things its python information is going to come from:

$ lldb-9 -P
/usr/lib/x86_64-linux-gnu/python3.7/site-packages

$ file /usr/lib/x86_64-linux-gnu/python3.7/site-packages
/usr/lib/x86_64-linux-gnu/python3.7/site-packages: cannot open `/usr/lib/x86_64-linux-gnu/python3.7/site-packages' (No such file or directory)

$ sudo apt-cache depends lldb-9
lldb-9
  Depends: libc6
  Depends: libedit2
  Depends: libgcc1
  Depends: liblldb-9
  Depends: libllvm9
  Depends: libncurses6
  Depends: libstdc++6
  Depends: libtinfo6
  Depends: llvm-9-dev
  Depends: python-lldb-9
  Breaks: llvm-9-tools
  Replaces: llvm-9-tools

$ dpkg -L python-lldb-9
/.
/usr
/usr/lib
/usr/lib/llvm-9
/usr/lib/llvm-9/lib
/usr/lib/llvm-9/lib/python2.7
/usr/lib/llvm-9/lib/python2.7/site-packages
/usr/lib/llvm-9/lib/python2.7/site-packages/lldb
/usr/lib/python2.7
/usr/lib/python2.7/dist-packages
/usr/share
/usr/share/doc
/usr/share/doc/python-lldb-9
/usr/share/doc/python-lldb-9/copyright
/usr/lib/llvm-9/lib/python2.7/site-packages/lldb/_lldb.so
/usr/lib/llvm-9/lib/python2.7/site-packages/lldb/libLLVM-9.0.0.so.1
/usr/lib/llvm-9/lib/python2.7/site-packages/lldb/libLLVM-9.so.1
/usr/lib/python2.7/dist-packages/lldb
/usr/share/doc/python-lldb-9/NEWS.Debian.gz
/usr/share/doc/python-lldb-9/changelog.Debian.gz

Hmm, so the lldb-9 binary is expecting to execute python3 code, yet the apt package manager has installed a python2 support package. Let's install the correct python3 version:

$ sudo apt install python3-lldb-9

$ file /usr/lib/x86_64-linux-gnu/python3.7/site-packages
/usr/lib/x86_64-linux-gnu/python3.7/site-packages: cannot open `/usr/lib/x86_64-linux-gnu/python3.7/site-packages' (No such file or directory)

Wow, even after we installed the python3 lldb support package, we still have lldb looking for its python module in a directory that doesn't exist. Luckily, we have Stack Overflow. The python3-lldb-9 package actually installs its python bindings in /usr/lib/llvm-9/lib/python3.7/site-packages. So we need to create symlink from where the lldb-9 binary is looking for the module to where the module is actually located:

cd /usr/lib/x86_64-linux-gnu/
sudo mkdir python3.7
cd python3.7
sudo ln -sf /usr/lib/llvm-9/lib/python3.7/site-packages site-packages

Now what happens in our debugger?:

(lldb) p fe->_fe_map.JxW
(std::vector<double, std::allocator<double> >) $0 = size=4 {
  [0] = 0.0025000000000000005
  [1] = 0.0025000000000000005
  [2] = 0.0025000000000000005
  [3] = 0.0025000000000000005
}

Ah, beautiful code! As a final step, if for some reason you ever wanted to load the lldb module inside of a standard python3 interpreter, you need to perform one additional symlink because:

$ ipython3
Python 3.7.5 (default, Nov 20 2019, 09:21:52)
Type "copyright", "credits" or "license" for more information.

IPython 5.8.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import sys

In [2]: sys.path.append('/usr/lib/x86_64-linux-gnu/python3.7/site-packages')

In [3]: import lldb
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
/usr/lib/x86_64-linux-gnu/python3.7/site-packages/lldb/__init__.py in <module>()
     34     # _lldb should be a built-in module.
---> 35     import _lldb
     36 except ImportError:

ModuleNotFoundError: No module named '_lldb'

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
<ipython-input-3-937abf6852a0> in <module>()
----> 1 import lldb

/usr/lib/x86_64-linux-gnu/python3.7/site-packages/lldb/__init__.py in <module>()
     36 except ImportError:
     37     # Relative import should work if we are being loaded by Python.
---> 38     from . import _lldb
     39 try:
     40     _swig_property = property

ImportError: cannot import name '_lldb' from 'lldb' (/usr/lib/x86_64-linux-gnu/python3.7/site-packages/lldb/__init__.py)

The difference here is that the __init__.py script in the lldb python module does an absolute import when run from inside the lldb binary, but attempts a relative import when run from inside the python interpreter. The problem is that:

$ file /usr/lib/x86_64-linux-gnu/python3.7/site-packages/lldb/_lldb.so
/usr/lib/x86_64-linux-gnu/python3.7/site-packages/lldb/_lldb.so: cannot open `/usr/lib/x86_64-linux-gnu/python3.7/site-packages/lldb/_lldb.so' (No such file or directory)

So let's create the final additional symlink:

cd /usr/lib/llvm-9/lib/python3.7/site-packages/lldb
sudo ln -sf ../../../liblldb.so.1 _lldb.so

lldb import inside a standard python interpreter should now be successful:

$ ipython3
Python 3.7.5 (default, Nov 20 2019, 09:21:52)
Type "copyright", "credits" or "license" for more information.

IPython 5.8.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import sys

In [2]: sys.path.append('/usr/lib/x86_64-linux-gnu/python3.7/site-packages')

In [3]: import lldb

In [4]: