contur.factories package
Submodules
contur.factories.test_observable module
- class contur.factories.test_observable.Observable(ana_obj, xsec, nev, sm=None)[source]
Bases:
object
Processes and decorates
YODA.AnalysisObject
to a testable format- Parameters:
ana_obj (
YODA.AnalysisObject
) –YODA
AO to dress, containing signal info.xsec (
YODA.Scatter1D
) – _XSEC scatter recording generator cross section in YODA file (contained in all Rivet run outputs)nev (
YODA.Scatter1D
) – _EVTCOUNT scatter recording total generated events in YODA file (contained in all Rivet run outputs)sm (
SMPrediction
) – Standard Model prediction for this observable
- property data_scale
Scale factor applied to the refdata histogram/scatter
type (
float
)
- doPlot()[source]
Public member function to build yoda plot members for interactive runs
These are only for displaty, they are not used in any of the statistics calculations.
- get_sm_pval()[source]
Calculate the pvalue compatibility (using chi2 survival) for the SM prediction and this measurement
- property has_theory
Bool representing if a theory prediction was found for the input signal
type (
bool
)
- property likelihood
The instance of
Likelihood
derived from this histogramtype (
Likelihood
)
- property ref
Reference data, observed numbers input to test, scaled if required
type (
YODA.Scatter2D
)
- property refplot
Reference data for plotting
type (
YODA.Scatter2D
)
- property scaled
Bool representing if there is additional scaling applied on top of luminosity
type (
bool
)
- property signal_scale
Scale factor applied to the signal histogram/scatter, derived generally from input nEv and xs
type (
float
)
- property sigplot
Signal for plotting
type (
YODA.Scatter2D
)
- property stack_databg
Stacked, unscaled Signal+background for plotting (data as background)
type (
YODA.Scatter2D
)
- property stack_smbg
Stacked, unscaled Signal+background for plotting (SM as background)
type (
YODA.Scatter2D
)
- property thy
Reference SM theory data, scaled if required
type (
YODA.Scatter2D
)
- property thyplot
Theory for plotting
type (
YODA.Scatter2D
)
- class contur.factories.test_observable.ObservableValues(bin_widths=None, central_values=None, err_breakdown=None, covariance_matrix=None, diagonal_matrix=None, isref=False)[source]
Bases:
object
A book-keeping class to contain all the numerical info (central values, err_breakdown, covariance) for a given binned observable.
contur.factories.likelihood module
This module contains the implementation of the likelihood calculation, and various functions to manipulate test statistics.
Abstracted from the underlying YODA
objects, this module defines
two ways to construct likelihood functions from numerical types:
Likelihood
– The base likelihood building blocks, representing the information extracted from an underlying histogram of potentially correlated observables.
CombinedLikelihood
– A shell to combineLikelihood
blocks into a full likelihood, automatically encodes assumption that the included blocks are uncorrelated.
- class contur.factories.likelihood.CombinedLikelihood(stat_type='all')[source]
Bases:
Likelihood
Shell to combine
Likelihood
blocksMust specify a statistic type which selects which statistics will be combined.
This class is used to extract the relevant test statistic from each individual
Likelihood
and combine them This is initialised with no arguments as it is just a shell to combine the individual components, and automatically encodes the fact that each block is uncorrelated with each otherNote
Technically this could be constructed by building a
Likelihood
with a master covariance matrix made forming block diagonals with each individual component. Avoiding this is faster but less rigourous- add_likelihood(likelihood)[source]
Add a
Likelihood
block to this combination likelihood- Parameters:
likelihood (
Likelihood
) – Instance of computed Likelihood
- calc_cls()[source]
Call the calculation of the CLs confidence interval
Triggers the parent class calculation of the CLs interval based on the sum of test statistics added with the
add_likelihood()
method
- getCLs(type='internal')[source]
CLs hypothesis test value (ratio of p_s_b and p_b) For the CombinedLikelihood version, the input type is ignored and the internal value is always used. The “all” option is for when it is being used for a subpool.
type (
float
)
- get_spey_CLs(type='internal')[source]
CLs hypothesis test value (ratio of p_s_b and p_b) For the CombinedLikelihood version, the input type is ignored and the internal value is always used. The “all” option is for when it is being used for a subpool.
type (
float
)
- get_ts_b(type='internal')[source]
Test statistic of b only hypothesis For the CombinedLikelihood version, the input type is ignored and the internal value is always used.
type (
float
)
- class contur.factories.likelihood.Likelihood(calculate=False, ratio=False, profile=False, lumi=1.0, lumi_fb=1.0, sxsec=None, bxsec=None, tags='', sm_values=None, measured_values=None, bsm_values=None, expected_values=None)[source]
Bases:
object
Fundamental likelihood-block class and confidence-interval calculator
This class defines the structure of a series of observables to be constructed into a hypothesis test
- Keyword Arguments:
calculate (
bool
) – Perform the statistics calculation (otherwise the input variables are just set up)ratio (
bool
) – Flag if data is derived from a ratio measurement (not general! caveat emptor!)profile (
bool
) – Flag if data is derived from a profile histogram measurementlumi (
float
) – the integrated luminosity in the units of the measurement. used to calculate expected stat uncertainty on signallumi_fb (
float
) – the integrated luminosity for the measurement in fb. used to scale for HL-LHCsxsec (
float
) – Signal cross-section in picobarns, if non-nulltags (
string
) – names of the histograms this is associated withsm_values (
Observable_Values
) – All the numbers for the SM predictionmeasured_values (
Observable_Values
) – All the numbers for the measurementbsm_values (
Observable_Values
) – All the numbers for the signalexpected_values (
Observable_Values
) – The SM prediction with data uncertainties.
- get_sm_pval()[source]
Calculate the pvalue compatibility (using chi2 survival) for the SM prediction and this measurement
- property pools
Pool that the test belongs to
settable parameter
type (
string
)
- property subpools
Subpool the test belongs to
settable parameter
type (
string
)
- property tags
Name(s) of source histograms for this block
settable parameter
type (
string
)
- contur.factories.likelihood.build_full_likelihood(sorted_blocks, stat_type, omittedPool='')[source]
Function to build the full likelihood representing an entire
YODA
fileThis function takes the
sorted_likelihood_blocks
and combines them as statistically uncorrelated diagonal contributions to aCombinedLikelihood
instance which is stored as an attribute to this class aslikelihood
- Keyword Arguments:
omittedPools (
string
) – String of analysis pools to omit
- contur.factories.likelihood.combine_subpool_likelihoods(likelihood_blocks, omitted_pools='')[source]
build combined likelihoods for any active subpools, and add them to the list of likelihood blocks.
- contur.factories.likelihood.filter_nuisances(central_values, error_breakdown)[source]
This function should take a dict of error sources, apply a filter to remove those that are either uncorrelated or neglible, and return the filtered dict for use in NP fitting. Untested, since we don’t really do the fitting at present, so probably has issues.
- contur.factories.likelihood.likelihood_blocks_find_dominant_ts(likelihood_blocks, stat_type)[source]
Function that finds the chi-square test statistic that gives the maximum confidence level for each likelihood block for which we don’t have a valid covariance matrix (either the matrix has no invserse or has not been succesfully built)
- contur.factories.likelihood.likelihood_blocks_ts_to_cls(likelihood_blocks, stat_type)[source]
Function that calculates the confidence level for each likelihood block extracted from the
YODA
file using the signal and background test statistic for the block
- contur.factories.likelihood.pval_to_cls(pval_tuple)[source]
Function to calculate a cls when passed background and signal p values.
notes: we are not actually varying a parameter of interest (mu), just checking mu=0 vs mu=1
the tail we integrate to get a p-value depends on whether you’re looking for signal-like or background-like tails. For the signal-like p-value we integrate over all the probability density less signal-like than was observed, i.e. to the right of the observed test stat.
For the background-like p-value we should integrate over the less background-like stuff, i.e. from -infty to t_obs… which is 1 - the t-obs…infty integral.
So CLs is the ratio of the two right-going integrals, which is nice and simple and symmetric, but looks asymmetric when written in terms of the p-values because they contain complementary definitions of the integral limits
The code has implemented them both as right-going integrals, so does look symmetric, hence this comment to hopefully avoid future confusion.
- Parameters:
pval_tuple (
Tuple of floats
) – Tuple, first element p-value of signal hypothesis, second p-value of background- Returns:
CLs
float
– Confidence Interval in CLs formalism
- contur.factories.likelihood.sort_blocks(likelihood_blocks, stat_type, omitted_pools='')[source]
Function that sorts the list of likelihood blocks extracted from the
YODA
fileThis function implements the sorting algorithm to sort the list of all extracted
Likelihood
blocks in thelikelihood_blocks
list, storing the reduced list in thesorted_likelihood_blocks[stat_type]
list- Keyword Arguments:
omittedPools (
string
) – String of analysis pools to omitstat_type (
string
) – Which statisic (default, smbg, expected, hlexpected) to sort on.
- contur.factories.likelihood.ts_to_cls(ts_tuple_list, tags)[source]
Method to directly cast a list of tuples of test statistics (tuple contains background and signal test stats) into a list of CLs values
notes: we are not actually varying a parameter of interest (mu), just checking mu=0 vs mu=1
the tail we integrate to get a p-value depends on whether you’re looking for signal-like or background-like tails. For the signal-like p-value we integrate over all the probability density less signal-like than was observed, i.e. to the right of the observed test stat.
For the background-like p-value we should integrate over the less background-like stuff, i.e. from -infty to t_obs… which is 1 - the t-obs…infty integral.
So CLs is the ratio of the two right-going integrals, which is nice and simple and symmetric, but looks asymmetric when written in terms of the p-values because they contain complementary definitions of the integral limits
The code has implemented them both as right-going integrals, so does look symmetric, hence this comment to hopefully avoid future confusion.
- Parameters:
ts_tuple_list (
list
) – list of tuples of tests statistics (tuples of the form (test stat background, test stat background))- Returns:
CLs
list
– List of Confidence Intervals in CLs formalism
contur.factories.depot module
The Depot module contains the Depot class. This is intended to be the high level analysis control, most user access methods should be implemented at this level
- class contur.factories.depot.Depot[source]
Bases:
object
Parent analysis class to initialise
This can be initialised as a blank canvas, then the desired workflow is to add parameter space points to the Depot using the
add_point()
method. This appends each considered point to the objects internalpoints
. To get the point message from database rather than map file to the Depot using theadd_points_from_db()
method.Path for writing out objects is determined by cfg.plot_dir
- add_point(yodafile, param_dict)[source]
Add yoda file and the corresponding parameter point into the depot
- add_points_from_db(file_path, runpoint=None)[source]
Get the info of model points from the result database into the depot class
@TODO write a “get from DB” method for likelihood_point?
- property frame
A
pandas.DataFrame
representing the CLs interval for each point inpoints
type (
pandas.DataFrame
)
- merge(depot)[source]
Function to merge this conturDepot instance with another.
Points with identical parameters will be combined. If point from the input Depot is not present in this Depot, it will be added.
- Parameters:
depot (
contur.conturDepot
) – Additional instance to conturDepot to merge with this one
- property points
The master list of
LikelihoodPoint
instances added to the Depot instancetype (
list
[LikelihoodPoint
])
- resort_points()[source]
Function to trigger rerunning of the sorting algorithm on all items in the depot, typically if this list has been affected by a merge by a call to
contur.depot.merge()
- write(outDir, args, yodafile=None, include_dominant_pools=True, include_per_pool_cls=True)[source]
Function to write depot information to disk
write a results db files to outDir if cfg.mapfile is not None, also write out a “map” file containing the full pickle of this depot instance if cfg.csvfile is not None, also write out a csv file containing the data
- Parameters:
outDir (
string
) – String of filesystem location to write out the pickle of this instance to
contur.factories.yoda_factories module
The yoda_factories module contains three main components in the middle of the data flow, sitting between the high level steering
in contur.factories.Depot
class and the lower level statistics in the contur.factories.Likelihood
class
- class contur.factories.yoda_factories.YodaFactory(yodaFilePath)[source]
Bases:
object
Class controlling Conturs YODA file processing ability
This class is initialised from an os path to a
YODA
file and dresses it by iterating through each ao and wrapping that in an instance ofObservable
which encapsulates a YODA analysis object and derives the requiredLikelihood
block for it. This class then contains the aggregated information for all of these instances across the entireYODA
file.- Parameters:
yodaFilePath (
string
) – Validos.path
filesystem YODA file location
- contur.factories.yoda_factories.load_bg_data(path, sm_id='A', smtest=False)[source]
load the background (THY) and data (REF) for all the observables associated with this rivet analysis if smtest then read all SM predictions, if not then only read the default/selected one.
- contur.factories.yoda_factories.load_ref_ao(path, orig_ao, aos)[source]
Load the ao, with the path=path, into memory, as THY or REF object
- contur.factories.yoda_factories.load_ref_aos(f, analysis)[source]
Load the relevant analysis objects (REF or THY) from the file f.
- contur.factories.yoda_factories.load_sm_ao(path, orig_ao, sm)[source]
Load the ao, with the path=path, into memory, as THY or REF object
- contur.factories.yoda_factories.load_sm_aos(sm)[source]
Load the relevant analysis objects (REF or THY) from the file f.
- contur.factories.yoda_factories.root_n_errors(ao, is_evcount, nx=0.0, lumi=1.0, replace=False)[source]
Function to include root(number of expected events) errors in the uncertainties of 2D scatter.
The uncertainty based on the expected events for the relevant integrated luminosity. This is not about MC statistics!
The minimum uncertainty is one event… we are not doing proper low-stat treatment in tails, so this is a conservative fudge.
- Parameters:
ao – The
YODA
analysis object to be manipulated.nx – factor needed to convert to number of events for none-uniform bin widths (<0, not used, ==0, do nothing).
is_evcount – True is the plot is in event numbers. Otherwise assumed to be a differential cross section.
lumi – Integrated luminosity used to get event counts from differential cross sections
replace – If True replace the uncertainties. If False (default) add them in quadrature.
- Type:
YODA.AnalysisObject
- Type:
- Type:
boolean
- Type:
- Type: