nannyml.sampling_error.multiclass_classification module

nannyml.sampling_error.multiclass_classification.accuracy_sampling_error(sampling_error_components: Tuple, data) float[source]

Calculate the accuracy sampling error for a chunk of data.

Parameters
  • sampling_error_components (a set of parameters that were derived from reference data.) –

  • data (the (analysis) data you want to calculate or estimate a metric for.) –

Returns

sampling_error

Return type

float

nannyml.sampling_error.multiclass_classification.accuracy_sampling_error_components(y_true_reference: List[pandas.core.series.Series], y_pred_reference: List[pandas.core.series.Series])[source]

Calculate the sampling error components on reference data.

The y_true_reference and y_pred_proba_reference lists represent the binarized target values and model probabilities. The order of the Series in both lists should both match the list of class labels present.

Parameters
  • y_true_reference (List[pd.Series]) – Target values for the reference dataset.

  • y_pred_reference (List[pd.Series]) – Prediction values for the reference dataset.

Returns

sampling_error_components

Return type

Tuple

nannyml.sampling_error.multiclass_classification.auroc_sampling_error(sampling_error_components, data) float[source]

Calculate the AUROC sampling error for a chunk of data.

Parameters
  • sampling_error_components (a set of parameters that were derived from reference data.) –

  • data (the (analysis) data you want to calculate or estimate a metric for.) –

Returns

sampling_error

Return type

float

nannyml.sampling_error.multiclass_classification.auroc_sampling_error_components(y_true_reference: List[pandas.core.series.Series], y_pred_proba_reference: List[pandas.core.series.Series])[source]

Calculate the sampling error components on reference data.

The y_true_reference and y_pred_proba_reference lists represent the binarized target values and model probabilities. The order of the Series in both lists should both match the list of class labels present.

Parameters
  • y_true_reference (List[pd.Series]) – Target values for the reference dataset.

  • y_pred_proba_reference (List[pd.Series]) – Prediction probability values for the reference dataset.

Returns

sampling_error_components

Return type

List[Tuple]

nannyml.sampling_error.multiclass_classification.f1_sampling_error(sampling_error_components: List[Tuple], data) float[source]

Calculate the F1 sampling error for a chunk of data.

Parameters
  • sampling_error_components (a set of parameters that were derived from reference data.) –

  • data (the (analysis) data you want to calculate or estimate a metric for.) –

Returns

sampling_error

Return type

float

nannyml.sampling_error.multiclass_classification.f1_sampling_error_components(y_true_reference: List[pandas.core.series.Series], y_pred_reference: List[pandas.core.series.Series])[source]

Calculate the sampling error components on reference data.

The y_true_reference and y_pred_proba_reference lists represent the binarized target values and model probabilities. The order of the Series in both lists should both match the list of class labels present.

Parameters
  • y_true_reference (List[pd.Series]) – Target values for the reference dataset.

  • y_pred_reference (List[pd.Series]) – Prediction values for the reference dataset.

Returns

sampling_error_components

Return type

List[Tuple]

nannyml.sampling_error.multiclass_classification.precision_sampling_error(sampling_error_components: List[Tuple], data) float[source]

Calculate the precision sampling error for a chunk of data.

Parameters
  • sampling_error_components (a set of parameters that were derived from reference data.) –

  • data (the (analysis) data you want to calculate or estimate a metric for.) –

Returns

sampling_error

Return type

float

nannyml.sampling_error.multiclass_classification.precision_sampling_error_components(y_true_reference: List[pandas.core.series.Series], y_pred_reference: List[pandas.core.series.Series])[source]

Calculate the sampling error components on reference data.

The y_true_reference and y_pred_proba_reference lists represent the binarized target values and model probabilities. The order of the Series in both lists should both match the list of class labels present.

Parameters
  • y_true_reference (List[pd.Series]) – Target values for the reference dataset.

  • y_pred_reference (List[pd.Series]) – Prediction values for the reference dataset.

Returns

sampling_error_components

Return type

List[Tuple]

nannyml.sampling_error.multiclass_classification.recall_sampling_error(sampling_error_components: List[Tuple], data) float[source]

Calculate the recall sampling error for a chunk of data.

Parameters
  • sampling_error_components (a set of parameters that were derived from reference data.) –

  • data (the (analysis) data you want to calculate or estimate a metric for.) –

Returns

sampling_error

Return type

float

nannyml.sampling_error.multiclass_classification.recall_sampling_error_components(y_true_reference: List[pandas.core.series.Series], y_pred_reference: List[pandas.core.series.Series])[source]

Calculate the sampling error components on reference data.

The y_true_reference and y_pred_proba_reference lists represent the binarized target values and model probabilities. The order of the Series in both lists should both match the list of class labels present.

Parameters
  • y_true_reference (List[pd.Series]) – Target values for the reference dataset.

  • y_pred_reference (List[pd.Series]) – Prediction values for the reference dataset.

Returns

sampling_error_components

Return type

List[Tuple]

nannyml.sampling_error.multiclass_classification.specificity_sampling_error(sampling_error_components: List[Tuple], data) float[source]

Calculate the specificity sampling error for a chunk of data.

Parameters
  • sampling_error_components (a set of parameters that were derived from reference data.) –

  • data (the (analysis) data you want to calculate or estimate a metric for.) –

Returns

sampling_error

Return type

float

nannyml.sampling_error.multiclass_classification.specificity_sampling_error_components(y_true_reference: List[pandas.core.series.Series], y_pred_reference: List[pandas.core.series.Series])[source]

Calculate the sampling error components on reference data.

The y_true_reference and y_pred_proba_reference lists represent the binarized target values and model probabilities. The order of the Series in both lists should both match the list of class labels present.

Parameters
  • y_true_reference (List[pd.Series]) – Target values for the reference dataset.

  • y_pred_reference (List[pd.Series]) – Prediction values for the reference dataset.

Returns

sampling_error_components

Return type

List[Tuple]