varstool.sensitivity_analysis package

vars_functions

apply_unique(func: Callable, df: DataFrame, axis: int = 1, progress: bool = False) → DataFrame

Apply func to unique rows (axis=1) or columns (axis=0) of df in order to increase the efficiency of func evaluations.

Parameters

  • func (Callable) – the function of interest to be applied to df

  • df (array_like) – the Pandas DataFrame of interest

  • axis (int, optional) – 0 for index, 1 for columns, defaults to 1

  • progress (bool, optional) – False for hiding the progress bar, True for otherwise, defaults to False

Returns

applied_df – the returned dataframe with the func evaluations

Return type

array_like

bootstrapping(num_stars: int, pair_df: DataFrame, df: DataFrame, cov_section_all: DataFrame, bootstrap_size: int, bootstrap_ci: float, delta_h: float, ivars_scales: Tuple[float, ...], parameters: Dict[Union[str, int], Tuple[float, float]], st_factor_ranking: DataFrame, ivars_factor_ranking: DataFrame, grouping_flag: bool, num_grps: int, progress: bool = False) → Tuple

performs bootstrapping procedure to gather confidence interval limits on the variogram, Sobol, IVARS and VARS-ABE results, and the reliability estimates of the variogram, Sobol, IVARS, and VARS-ABE results. Also groups the Sobol and IVARS50 results using clustering analysis in a hierarchical order

Parameters

  • num_stars (int) – number of star points

  • pair_df (array_like) – Pandas DataFrame that contains the pairing results of the VARS analysis

  • df (array_like) – Pandas DataFrame containing the star_points and model results

  • cov_section_all (array_like) – Pandas DataFrame containing the sectional covariogram results

  • bootstrap_size (int) – the number of bootstrap samples that were taken

  • bootstrap_ci (float) – the confidence interval of the bootstrapping results (ex. 0.90)

  • delta_h (float) – resolution of star samples

  • ivars_scales (tuple) – tuple containing the scales used in IVARS calculation

  • parameters (dictionary) – dictionary containing parameter names and their attributes

  • st_factor_ranking (array_like) – Pandas DataFrame containing the Sobol factor ranking results

  • ivars_factor_ranking (array_like) – Pandas DataFrame containing the IVARS factor ranking results

  • grouping_flag (boolean) – true if grouping is being done, false otherwise

  • num_grps (int) – the number of groups the parameters are to be clustered into, None if group is to be chosen using elbow method

  • progress (boolean) – true if loading bar is to be shown, false otherwise

Returns

  • gammalb (array_like) – variogram upper bound bootstrapping results

    • gammaub (array_like) – variogram lower bound bootstrapping results

    • stlb (array_like) – Sobol lower bound bootstrapping results

    • stub (array_like) – Sobol upper bound bootstrapping results

    • ivarslb (array_like) – IVARS lower bound bootstrapping results

    • ivarsub (array_like) – IVARS upper bound bootstrapping results

    • rel_sobol_factor_ranking (array_like) – reliability estimates of Sobol results based on bootstrapping

    • rel_ivars_factor_ranking (array_like) – reliability estimates of IVARS results based on bootstrapping

    • ivars50_grp (array_like) – IVARS50 grouping results

    • sobol_grp (array_like) – Sobol grouping results

    • reli_sobol_grp (array_like) – reliability estimates of groups based on bootstrapping

    • reli_ivars50_grp (array_like) – reliability estimates of groups based on bootstrapping

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

cov_section(pair_cols: DataFrame, mu_star: DataFrame) → DataFrame

This function return the sectional covariogram of the pairs of function evaluations that resulted from each star point. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

  • pair_cols (array_like) – a Pandas Dataframe of paired values function evaluations

  • mu_star (array_like) – a Pandas DataFrame of mu star values that are calculated separately

Returns

cov_section_values – the sectional covariogram dataframe

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

covariogram(pair_cols: DataFrame, mu_overall: Series) → DataFrame

This function return the covariogram values derived from the pairs of function evaluations that each resulted from each star point. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

  • pair_cols (array_like) – a Pandas Dataframe of paired values function evaluations

  • mu_overall (array_like) – a Pandas Dataframe of overall mu calculated on all function evaluation values for each time-step

Returns

covariogram_values – the covariogram dataframe

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

e_covariogram(cov_section_all: DataFrame) → DataFrame

This function return the Expected value of covariogram values derived from the pairs of function evaluations that each resulted from each star point. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

cov_section_all (array_like) – a Pandas Dataframe of sectional covariograms

Returns

e_covariogram_values – the covariogram dataframe

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

elbow_method(z: array) → int

a method used to determine the number of clusters in the data being grouped

Parameters

z (array_like) – the results from linking the factors together using statpy’s linkage function

Returns

cutoff – the optimal number to use as a cutoff when clustering factors

Return type

int

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

factor_grouping(sens_idx: DataFrame, num_grp: Optional[int] = None) → DataFrame

Groups parameters based on how close in ‘distance’ they are. This is done using clustering in a hierarchical fashion. The user can specify the number of groups or have the optimal number chosen using the elbow method by not inputting any group number. Usually done with high parameter models.

Parameters

  • sens_idx (array_like) – the Pandas DataFrame containing the parameters to be grouped along with their values

  • num_grp (int) – the number of groups the parameters are to be clustered into, None if group is to be chosen using elbow method

Returns

  • optm_num_grp (int) – the optimal group number, (either user inputted group number, or calculated group number)

  • rank_grp (array_like) – the group number each parameter belongs to in their corresponding index.

  • clusters (array_like) – list of different cluster configurations used for reliability estimates

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

factor_ranking(factors: ndarray) → ndarray

Ranks factors based on their influence (how large or small results are) The lowest rank corresponds to the most influential (larger) factor

Parameters

factors (array_like) – an array like object that contains factors/parameters/variables of the sensitivity analysis problem

Returns

ranks – a numpy array containing the ranks of each factor in their corresponding index

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

grouping(result_bs_ivars_df: DataFrame, result_bs_sobol: DataFrame, result_bs_ivars_ranking: DataFrame, result_bs_sobol_ranking: DataFrame, num_grps: int, st_factor_ranking: DataFrame, ivars_factor_ranking: DataFrame, parameters: Dict[Union[str, int], Tuple[float, float]], bootstrap_size: int) → Tuple

Groups parameters based on how close in ‘distance’ they are. This is done using clustering in a hierarchical fashion. The user can specify the number of groups or have the optimal number chosen using the elbow method by not inputting any group number. Usually done with high parameter models. Also calculates the reliability estimates of the group when bootstrapping.

Parameters

  • result_bs_sobol_ranking (array_like) – Pandas DataFrame with the bootstrapping results of the Sobol factor rankings

  • result_bs_sobol (array_like) – Pandas DataFrame with the bootstrapping results of the Sobol results

  • result_bs_ivars_ranking (array_like) – Pandas DataFrame with the bootstrapping results of the IVARS factor rankings

  • result_bs_ivars_df (array_like) – Pandas DataFrame with the bootstrapping results of the IVARS results

  • num_grps (int) – the number of groups the parameters are to be clustered into, None if group is to be chosen using elbow method

  • st_factor_ranking (array_like) – Pandas DataFrame holding the original Sobol factor rankings

  • ivars_factor_ranking (array_like) – Pandas DataFrame holding the original IVARS factor rankings

  • parameters (dictionary) – dictionary holding the parameter names and their attributes

  • bootstrap_size (int) – the number of bootstrap samples that were taken

Returns

  • ivars50_grp (array_like) – ivars50 grouping results

  • sobol_grp (array_like) – sobol grouping results.

  • reli_sobol_grp (array_like) – reliability estimates of sobol groups using bootstrapping results

  • reli_ivars50_grp (array_like) – relibaility estimates of ivars50 groups using bootstrapping results

    References

  • ———-

  • .. [1] Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, – robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

  • .. [2] Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, – robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

ivars(variogram_array: DataFrame, scale: float, delta_h: float) → DataFrame

Generates Integrated Variogram Across a Range of Scales (IVARS) by approximating area using right trapezoids having width of delta_h and hights of variogram values. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

  • variogram_array (array_like) – a Pandas Dataframe of variogram values for each time-step

  • scale (float) – the scale for the IVARS evaluations

  • delta_h (float) – the resolution of star point generation

Returns

ivars_values – the Sobol Equivalent values

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

morris_eq(pair_cols: DataFrame) → DataFrame

This function return the Morris Equivalent values derived from the pairs of function evaluations that each resulted from each star point. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

pair_cols (array_like) – a Pandas Dataframe of paired values function evaluations

Returns

morris_eq_values – the morris dataframe

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

pairs_h(iterable: Iterable) → DataFrame

Give the pairs of numbers considering their differences.

Parameters

iterable (iterable) – an iterable object

Returns

pairs – the returned dataframe of paired values

Return type

array_like

scale(df: DataFrame, bounds: DataFrame, axis: int = 1) → DataFrame

This function scales the sampled matrix df to the `bounds` that is a defined via a dictionary with ub, lb keys; the values of the dictionary are lists of the upper and lower bounds of the parameters/variables/factors. if (axis = 1) then each row of df is selected, otherwise columns.

Parameters

  • df (array_like) – a dataframe of randomly sampled values

  • bounds (dict) – a lower and upper bounds to scale the values

  • axis (int, optional) – 0 for index, 1 for columns

Returns

df – the returned dataframe scaled using bounds

Return type

array_like

section_df(df: DataFrame, delta_h: float) → DataFrame

This function gets the paired values of each section based on index.

Parameters

  • df (array_like) – a dataframe of star points

  • delta_h (float) – resolution of star samples

Returns

sample – the paired values for each section of star points

Return type

array_like

sobol_eq(gamma: DataFrame, ecov: DataFrame, variance: Series, delta_h: float) → DataFrame

This function return the Sobol Equivalent values derived from the variogram (gamma), expected values of sectional covariograms (ecov), and overall variance (variance). This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

  • gamma (array_like) – a Pandas Dataframe of variogram values for each time-step

  • ecov (array_like) – a Pandas DataFrame of expected values of sectional covariograms for each time-step

  • variance (array_like) – variance of function evaluations over all time-steps

  • delta_h (float) – resolution of star samples

Returns

sobol_eq_values – the Sobol Equivalent values

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

variogram(pair_cols: DataFrame) → DataFrame

This function return the variogram calculated from the pairs of function evaluations that each resulted from each star point. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

pair_cols (array_like) – a Pandas Dataframe of paired values function evaluations

Returns

variogram_values – the variogram dataframe

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

gvars_functions

custom_distribution_probabilites(dist_sample_file: Optional[str], param)

finds empirical cdf for custom probability distribution and puts it in a dataframe.

Parameters

  • dist_sample_file (str) – string name of .csv file containing custom distribution data

  • param (String) – name of parameter

Returns

cdp – df containing custom distributions and empirical cdf

Return type

array_like

find_boundaries(parameters, dist_sample_file: Optional[str] = None)

finds maximum and minimum boundary of each parameter.

Parameters

  • parameters (Dictionary) – dictionary containing parameters names and attributes

  • dist_sample_file (str) – name of file containing distributions data

Returns

  • xmin (array_like) – the lower boundaries of each parameter

  • xmax (array_like) – the upper boundaries of each parameter

map_2_cornorm(parameters: Dict[Union[str, int], Tuple[Union[float, str]]], corr_mat: ndarray, progress: bool) → ndarray

Computes the fictive correlation matrix given a correlation matrix and its corresponding parameters using nataf transformation

Parameters

  • parameters (dictionary) – a dictionary containing parameter names and their attributes

  • corr_mat (np.ndarray) – correlation matrix

  • progress (boolean) – true if loading bar is to be shown, false otherwise

Returns

corr_n – the fictive correlation matrix

Return type

np.ndarray

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

3

Razavi, S., & Do, C. N. (2020). Correlation Effects? A Major but Often Neglected Component in Sensitivity and Uncertainty Analysis. Water Resources Research, 56(3). doi: /10.1029/2019WR025436

n2x_transform(norm_vectors: ndarray, parameters: Dict, dist_sample_file: Optional[str] = None) → ndarray

transforms multivariate normal samples into parameters original distributions

Parameters

  • norm_vectors (np.ndarray) – multivariate normal samples

  • parameters (dict) – a dictionary containing parameter information (name: bounds, distributions, etc.)

  • dist_sample_file (String) – name of file that contains custom distribution data, optional only for users with custom distributions

Returns

x – the transformed vectors

Return type

np.ndarray

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

3

Razavi, S., & Do, C. N. (2020). Correlation Effects? A Major but Often Neglected Component in Sensitivity and Uncertainty Analysis. Water Resources Research, 56(3). doi: /10.1029/2019WR025436

pairs_h(iterable: Iterable) → DataFrame

Give the pairs of numbers considering their differences.

Parameters

iterable (iterable) – an iterable object

Returns

pairs – the returned dataframe of paired values

Return type

array_like

reorder_pairs(pair_df: DataFrame, num_stars: int, parameters: Dict[Union[str, int], Tuple[Union[float, str]]], df: DataFrame, delta_h: float, report_verbose: bool, xmax: ndarray, xmin: ndarray, offline_mode: bool) → DataFrame

Calculates the differences(‘h’) between the pairings of the star points, and bins and reorders the pair dataframe according to the calculated ‘h’ values

Parameters

  • pair_df (pd.DataFrame) – Pandas DataFrame containing the paired star points values with the model outputs

  • num_stars (int) – number of star samples

  • parameters (dictionary) – dictionary containing parameter names and their attributes

  • df (pd.DataFrame) – Pandas DataFrame containing the star points

  • delta_h (float) – resolution of star samples

  • report_verbose (boolean) – if True will use a loading bar when generating stars, does nothing if False

  • xmax (arraylike) – array containing max boundary of each parameter

  • xmin (arraylike) – array containing min boundary of each parameter

  • offline_mode (boolean) – if True GVARS analysis is in offline mode, if False it is in online mode

Returns

pair_df – the returned dataframe of paired values

Return type

array_like

rn2rx(distpair_type: List, param1: List, param2: List, rnpair: float) → float

transforms value rn in a correlation matrix to values rx

Parameters

  • distpair_type (List) – a list containing parameter 1 and parameter 2’s distribution type

  • param1 (List) – a list containing statistical information about parameter 1

  • param2 (List) – a list containing statistical information about parameter 2

  • rnpair (float) – value containing rn from the correlation matrix

Returns

rx – the transformed rn value

Return type

float

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

3

Razavi, S., & Do, C. N. (2020). Correlation Effects? A Major but Often Neglected Component in Sensitivity and Uncertainty Analysis. Water Resources Research, 56(3). doi: /10.1029/2019WR025436

rx2rn(distpair_type: List, param1: List, param2: List, rxpair: float) → float

transforms value rx in a correlation matrix to value rn

Parameters

  • distpair_type (List) – a list containing parameter 1 and parameter 2’s distribution types

  • param1 (List) – a list containing statistical information about parameter 1

  • param2 (List) – a list containing statistical information about parameter 2

  • rxpair (float) – value containing rx from the correlation matrix

Returns

rn – the transformed rx value

Return type

float

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

3

Razavi, S., & Do, C. N. (2020). Correlation Effects? A Major but Often Neglected Component in Sensitivity and Uncertainty Analysis. Water Resources Research, 56(3). doi: /10.1029/2019WR025436

tsvars_functions

cov_section(pair_cols: DataFrame, mu_star: DataFrame) → Series

Returns the sectional covariogram of the pairs of function evaluations that resulted from each star point. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

  • pair_cols (array_like) – a Pandas Dataframe of paired values function evaluations

  • mu_star (array_like) – a Pandas DataFrame of mu star values that are calculated separately

Returns

cov_section_values – the sectional covariogram dataframe

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

covariogram(pair_cols: DataFrame, mu_overall: Series) → Series

Return the covariogram values derived from the pairs of function evaluations that each resulted from each star point. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

  • pair_cols (array_like) – a Pandas Dataframe of paired values function evaluations

  • mu_overall (array_like) – a Pandas Dataframe of overall mu calculated on all function evaluation values for each time-step

Returns

covariogram_values – the covariogram dataframe

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

e_covariogram(cov_section_all: DataFrame) → Series

Returns the Expected value of covariogram values derived from the pairs of function evaluations that each resulted from each star point. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

cov_section_all (array_like) – a Pandas Dataframe of sectional covariograms

Returns

e_covariogram_values – the covariogram dataframe

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

ivars(variogram_array: DataFrame, scale: float, delta_h: float) → DataFrame

Generates Integrated Variogram Across a Range of Scales (IVARS) by approximating area using right trapezoids having width of delta_h and hights of variogram values. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

  • variogram_array (array_like) – a Pandas Dataframe of variogram values for each time-step

  • scale (gloat) – the scale for the IVARS evaluations

  • delta_h (float) – the resolution of star point generation

Returns

ivars_values – the Sobol Equivalent values

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

morris_eq(pair_cols: DataFrame) → Tuple[Series, ...]

Return the Morris Equivalent values derived from the pairs of function evaluations that each resulted from each star point. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

pair_cols (array_like) – a Pandas Dataframe of paired values function evaluations

Returns

morris_eq_values – the morris dataframe

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

pairs_h(iterable) → DataFrame

Gives the pairs of numbers considering their differences.

Parameters

iterable (iterable) – an iterable object

Returns

pairs – the returned dataframe of paired values

Return type

array_like

scale(df: DataFrame, bounds: DataFrame, axis: int = 1) → DataFrame

Scales the sampled matrix df to the bounds that is a defined via a dictionary with ub, lb keys; the values of the dictionary are lists of the upper and lower bounds of the parameters/variables/factors. if (axis = 1) then each row of df is selected, otherwise columns.

Parameters

  • df (array_like) – a dataframe of randomly sampled values

  • bounds (dict) – a lower and upper bounds to scale the values

  • axis (0 for index, 1 for columns) –

Returns

df – the returned dataframe scaled using bounds

Return type

array_like

section_df(df: DataFrame, delta_h: float) → DataFrame

Gets the paired values of each section based on index.

Parameters

  • df (array_like) – a dataframe of star points

  • delta_h (array_like) – resolution of star samples

Returns

sample – the paired values for each section of star points

Return type

array_like

sobol_eq(gamma: DataFrame, ecov: DataFrame, variance: Series, delta_h: float) → Series

Returns the Sobol Equivalent values derived from the variogram (gamma), expected values of sectional covariograms (ecov), and overall variance (variance). This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

  • gamma (array_like) – a Pandas Dataframe of variogram values for each time-step

  • ecov (array_like) – a Pandas DataFrame of expected values of sectional covariograms for each time-step

  • variance (array_like) – variance of function evaluations over all time-steps

  • delta_h (float) – resolution of star samples

Returns

sobol_eq_values – the Sobol Equivalent values

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

variogram(pair_cols: DataFrame) → Series

Returns the variogram calculated from the pairs of function evaluations that each resulted from each star point. This function is specific for the time-series varying/aggregate of the VARS sensitivity analysis.

Parameters

pair_cols (array_like) – a Pandas Dataframe of paired values function evaluations

Returns

variogram_values – the variogram dataframe

Return type

array_like

References

1

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017558

2

Razavi, S., & Gupta, H. V. (2016). A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Application. Water Resources Research, 52(1), 423-439. doi: 10.1002/2015WR017559

dvars_functions

L_runner(phi: ndarray, X: ndarray, Y: ndarray, verbose: bool) → float

A wrapper function for calculating the negative log-likelihood cost.

Parameters

  • phi (numpy.ndarray) – The hyperparameters for the covariance function.

  • X (numpy.ndarray) – The state matrix for all the input variables percentiles.

  • Y (numpy.ndarray) – The state matrix for the output variables nums.

  • verbose (bool, default False) – whether to print diagnostic information or not

Returns

L – The negative log-likelihood cost.

Return type

float

calc_Gammaj(Hj: float, phij: float, variance: float, correlation_func_type: str = 'Linear') → float

Calculates the IVARS sensitivity index from a learned covariance function.

This integrates the directional variogram for a specific input variable using trapezoidal integration.

Parameters

  • Hj (float) – The fraction of the total parameter space to integrate over. Note that the linear correlation function only has one hyperparameter, so as the Reference notes it is unable to distinguish variogram effects at varying length scales. So, this should not be set to anything other than 1 in practice.

  • phij (float) – The learned hyperparameter for the covariance function between the output and input variable.

  • variance (float) – The variance of the output variable.

  • correlation_func_type (str) – determines the type of covariance function to use

Returns

Gammaj – The global sensitivity index for this output-input variable pair.

Return type

float

calc_L(phi: np.ndarray, X: np.ndarray, Y: np.ndarray) → float

Calculate the negative log-likelihood cost. Note that this is just-in-time compiled by numba for increased speed.

Parameters

  • phi (numpy.ndarray) – The hyperparameters for the covariance function.

  • X (numpy.ndarray) – The state matrix for all the input variables percentiles.

  • Y (numpy.ndarray) – The state matrix for the output variables nums.

Returns

L – The negative log-likelihood cost.

Return type

float

calc_R(phi: np.ndarray, X: np.ndarray) → np.ndarray

Calculate the correlation matrix between each of the input states. Note that this is just-in-time compiled by numba for increased speed.

Parameters

  • phi (numpy.ndarray) – The hyperparameters for the covariance function.

  • X (numpy.ndarray) – The state matrix for all the input variables percentiles.

Returns

R – The correlation matrix.

Return type

float

calc_Ruw(phi: np.ndarray, Xu: np.ndarray, Xw: np.ndarray) → float

Calculate the correlation between two input states. Note that this is just-in-time compiled by numba for increased speed.

Parameters

  • phi (numpy.ndarray) – The hyperparameters for the covariance function.

  • Xu (numpy.ndarray) – The u’th input state.

  • Xw (numpy.ndarray) – The w’th input state.

Returns

Ruw – The correlation between the two states.

Return type

float

calc_phi_opt(simulation_df: DataFrame, ninvars: int, nobvs: int, outvarname: str, phi_max: float = 1000000.0, phi0: float = 1.0, tol: float = 1e-06, verbose: bool = False) → ndarray

Calculate the optimal hyperparameters for the covariance functions between the output variable and each of the input variables via maximum likelihood estimation (MLE). MLE works by minimizing a negative log-likelihood function.

Parameters

  • simulation_df (pd.Dataframe) – The input simulation.

  • ninvars (int) – number of input variables

  • nobvs (int) – number of observations

  • outvarname (str) – The name of the output variable to calculate sensitivities for. Note that the output variable must be scalar.

  • tol (float, default 1e-6) – The convergence tolerance for scipy’s minimize function acting on the negative log likelihood function.

  • phi_max (float, default 1e6) – the upper bound value for the optimal phi value

  • phi0 (float, default 1) – the value where we start our search for the optimal phi.

  • verbose (bool, default False) – Whether to print diagnostic information.

Returns

phi_opt – The learned hyperparameters for the covariance functions.

Return type

numpy.ndarray

calc_rj(hj: float, phij: float, correlation_func_type: Optional[str] = 'linear') → float

The covariance function (also called a kernel). We currently use a linear kernel which has a single hyperparameter that must be learned. Note that this is just-in-time compiled by numba for increased speed.

Parameters

  • hj (float) – The distance between two state elements.

  • phij (float) – The hyperparameter for the function.

  • correlation_func_type (str) – determines the type of covariance function to be used

Returns

rj – The covariance.

Return type

float

calc_sensitivities(simulation_df: pd.DataFrame, outvarname: str, Hj: float = 0.5, phi_max: float = 1000000.0, phi0: float = 1.0, correlation_func_type: str = 'Linear', tol: float = 1e-06, verbose: bool = False) → tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]

Calculates the global sensitivity indices and ratios for a specific output variable to each of a simulation’s input variables.

This implements the D-VARS algorithm as dscribed in Reference [1]_ to calculate global sensitivity indices from a set of given data. The implementation largely follows the Reference’s supplementary notes, though Multi-SQP is replaced with scipy’s minimize function implementing L-BFGS-B. See also Reference [2]_ for some theoretical background on the VARS and IVARS methods of calculating sensitivity indices.

Parameters

  • simulation_df (pd.DataFrame) – The input simulation.

  • outvarname (str) – The name of the output variable to calculate sensitivities for. Note that the output variable must be scalar.

  • Hj (float, default: 1.0) – The fraction of the total parameter space to integrate over. Note that the linear correlation function only has one hyperparameter, so as the Reference notes it is unable to distinguish variogram effects at varying length scales. So, this should not be set to anything other than 1 in practice.

  • tol (float, default 1e-6) – The convergence tolerance for scipy’s minimize function acting on the negative log likelihood function.

  • phi_max (float, default 1e6) – the upper bound value for the optimal phi value

  • phi0 (float, default 1) – the value where we start our search for the optimal phi.

  • correlation_func_type (str) – determines the type of covariance function to use

  • verbose (bool, default False) – Whether to print diagnostic information.

Returns

  • sensitivities (numpy.ndarray) – The global sensitivity indices for the output variable for each of the sim’s input variables.

  • ratios (numpy.ndarray) – The global sensitivity ratios for the output variable for each of the sim’s input variables, essentially the fraction of each input variable’s ability to explain the output variance.

  • phi_opt (numpy.ndarray) – the optimal phi values used to calculate the sensitivity indices

  • variance (float) – the variance value of the outvar

References

1

Sheikholeslami, Razi, and Saman Razavi. “A fresh look at variography: measuring dependence and possible sensitivities across geophysical systems from any given data.” Geophysical Research Letters 47.20 (2020): e2020GL089829.

2

Razavi, Saman, and Hoshin V. Gupta. “A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory.” Water Resources Research 52.1 (2016): 423-439.