From d48484c29d09a944de6f9251a3c659e76279464e Mon Sep 17 00:00:00 2001 From: tlatorre Date: Mon, 5 Oct 2020 14:36:40 -0500 Subject: major updates to the chi2 analysis This commit fixes the chi2 analysis so that it is no longer biased. Previously, the chi2 analysis pull plots showed a consistent bias. At first, I thought this was due to the fact that the posterior wasn't gaussian, but even after switching to percentile plots based on the algorithm outlined in "Validating Bayesian Inference Algorithms with Simulation-Based Calibration", I was still seeing a bias. I finally tracked it down to the fact that I was applying the energy scale parameters to the data instead of the Monte Carlo. Therefore, in this commit I update the posterior to now apply the energy scale parameters to the Monte Carlo instead of the data. This has the slight disadvantage that the final histograms will be binned in the biased energy, but that's not really a big deal. In addition, this commit contains several other updates: - switch to plotting percentile plots based on the algorithm in "Validating Bayesian Inference Algorithms with Simulation-Based Calibration" instead of pull plots - apply both the energy scale and resolution at the individual particle level, i.e. there is no longer an energy resolution term for electron + muon fits - separate pull plots and coverage plots. Previously I was making both the p-value coverage plots and the pull plots at the same time. However, the pull plots shouldn't have anything to do with the GENIE weights whereas the p-value coverage plots should draw samples weighted by the GENIE weights. In addition, for the pull plots we draw new truth parameters on every iteration whereas for the p-value coverage plots we only draw them once. - switch to using KDEMove() for the MCMC since I think it samples multimodal distributions a lot better than the default emcee move. - I now correct for the reconstruction energy bias in plot-michel and plot-muons --- utils/plot-muons | 3 +++ 1 file changed, 3 insertions(+) (limited to 'utils/plot-muons') diff --git a/utils/plot-muons b/utils/plot-muons index 25e7d24..27d3423 100755 --- a/utils/plot-muons +++ b/utils/plot-muons @@ -75,6 +75,7 @@ if __name__ == '__main__': from sddm.plot_energy import * from sddm.plot import * from sddm import setup_matplotlib + from sddm.utils import correct_energy_bias parser = argparse.ArgumentParser("plot fit results") parser.add_argument("filenames", nargs='+', help="input files") @@ -93,7 +94,9 @@ if __name__ == '__main__': for run, df in rhdr.groupby('run'): evs.append(get_events(df.filename.values, merge_fits=True, nhit_thresh=args.nhit_thresh)) ev = pd.concat(evs) + ev = correct_energy_bias(ev) ev_mc = get_events(args.mc, merge_fits=True) + ev_mc = correct_energy_bias(ev_mc) # Drop events without fits ev = ev[~np.isnan(ev.fmin)] -- cgit