Building
========

First you have to install the gsl, nlopt, zlib, and hdf5 libraries. On Fedora
you can install these with the following command:

    $ yum install gsl gsl-devel NLopt NLopt-devel zlib zlib-devel hdf5 hdf5-devel

You can also install these packages yourself. For instructions, see below
(Installing GSL and Installing NLopt).

Then, to build everything you just type:

    $ make

Some of the test programs also require plot and gnuplot which can be installed
with:

    $ yum install plotutils gnuplot

In addition, the python scripts require numpy, scipy, yaml, h5py, matplotlib,
pandas, and nlopt which can be installed with the following command on Fedora
27:

    $ yum install python2-numpy python2-scipy PyYAML python2-h5py python2-matplotlib python2-pandas python-NLopt

Installing GSL
==============

To install GSL in your home directory, you can run the following commands:

    $ curl -O -L ftp://ftp.gnu.org/gnu/gsl/gsl-2.5.tar.gz
    $ tar -xzvf gsl-2.5.tar.gz
    $ cd gsl-2.5
    $ mkdir $HOME/local
    $ ./configure --prefix=$HOME/local
    $ make
    $ make install

Installing NLopt
================

    $ curl -O -L https://github.com/stevengj/nlopt/archive/v2.5.0.tar.gz
    $ tar -xzvf v2.5.0.tar.gz
    $ cd nlopt-2.5.0
    $ mkdir build
    $ cd build
    $ cmake -DCMAKE_INSTALL_PREFIX=$HOME/local ..
    $ make
    $ make install

To use gsl and nlopt installed locally, you will have to edit the Makefile and
add the following to the CFLAGS and LDLIBS variables:

    CFLAGS=-I$(HOME)/local/include
    LDLIBS=-L$(HOME)/local/lib -L$(HOME)/local/lib64

and also edit the -lnopt_cxx line to just be -lnlopt.

You will also need to edit ~/.bash_profile and add the following line:

    export LD_LIBRARY_PATH=$HOME/local/lib:$HOME/local/lib64

Installing NLopt for the grid
=============================

When running the fitter on the grid, it's necessary to compile NLopt as a
static library.

    $ curl -O -L https://github.com/stevengj/nlopt/archive/v2.6.1.tar.gz
    $ tar -xzvf v2.6.1.tar.gz
    $ cd nlopt-2.6.1
    $ mkdir build
    $ cd build
    $ cmake -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/g++ -DCMAKE_INSTALL_PREFIX=$HOME/local -DBUILD_SHARED_LIBS=OFF ..
    $ make
    $ make install

and then you have to edit the Makefile to add:

    CFLAGS=-fdiagnostics-color -O2 -Wall -g -DSWAP_BYTES -I/home/tlatorre/local/include
    LDLIBS=-fdiagnostics-color -lm -L/home/tlatorre/local/lib64 -l:libnlopt.a -L/home/tlatorre/local/lib -l:libgsl.a -l:libgslcblas.a -lstdc++

Fitting Events
==============

You can fit events from SNOMAN data structure files by running the following
command:

    $ ./src/fit [input file] -o [output file]

and then plot the fit results by using the plot script:

    $ ./utils/plot.py [output file]

The plot script relies on the MC information in the SNOMAN file, so make sure
that SNOMAN is set up to output MC info when you simulate events (see the next
section).

Simulating Events with SNOMAN
=============================

Since the fitter reads in SNOMAN data structures it is necessary to use a
command file which correctly specifies the output file format. To benchmark the
fit results it is also necessary to save the initial MC vertex information.

I have created a template command file in macros/mc_run_10000.cmd which
simulates run 10000 from the D2O phase. You can create a proper command file to
run SNOMAN with by running the snogen program which uses a simple templating
system to fill in some variables in this template. For example to simulate 100
1 GeV muons uniformly distributed throughout the AV and with isotropic
directions you can run:

    $ ./macros/snogen -p mu_minus -e 1000.0 -n 100 | /path/to/snoman.exe

See the comment at the top of ./macros/snogen for more information.

Debugging
=========

To debug memory issues you can compile everything with an address sanitizer.
Edit the Makefile and add the following flags:

    CFLAGS=-fsanitize=address
    LDLIBS=-fsanitize=address

Then, recompile:

    $ make clean
    $ make

and run the fitter. It should crash if there is a buffer overflow or use after
free bug.

You can also run the fitter with valgrind to search for memory leaks:

    $ valgrind --leak-check=full ./src/fit [input file]

Profiling
=========

To profile the code, you can use the linux perf command along with Brendan
Gregg's flamegraph software to produce a flame graph.

First run the fitter:

    $ ./src/fit [input file]

Then, identify the PID of the fit program and run:

    $ sudo perf record -F 99 -p [PID] --call-graph dwarf -- sleep 10

Then, you can see the results by running:

    $ perf report

or to create a flame graph (see https://github.com/brendangregg/FlameGraph):

    $ sudo chown [username]:[username] perf.data
    $ perf script > out.perf
    $ stack-collapse.pl out.perf > out.folded
    $ flamegraph.pl out.folded > kernel.svg