nannyml.performance_calculation.metrics.multiclass_classification module

Module containing metric utilities and implementations.

class nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationAUROC(y_true: str, y_pred: str, threshold: nannyml.thresholds.Threshold, y_pred_proba: Optional[Union[str, Dict[str, str]]] = None, **kwargs)

Bases: nannyml.performance_calculation.metrics.base.Metric

Area under Receiver Operating Curve metric.

Creates a new AUROC instance.

Parameters

• y_true (str) – The name of the column containing target values.

• y_pred (str) – The name of the column containing your model predictions.

• threshold (Threshold) – The Threshold instance that determines how the lower and upper threshold values will be calculated.

• y_pred_proba (Union[str, Dict[str, str]]) –

  Name(s) of the column(s) containing your model output.

  • For binary classification, pass a single string refering to the model output column.

  • For multiclass classification, pass a dictionary that maps a class string to the column name containing model outputs for that class.

class nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationAccuracy(y_true: str, y_pred: str, threshold: nannyml.thresholds.Threshold, y_pred_proba: Optional[Union[str, Dict[str, str]]] = None, **kwargs)

Bases: nannyml.performance_calculation.metrics.base.Metric

Accuracy metric.

Creates a new Accuracy instance.

Parameters

• y_true (str) – The name of the column containing target values.

• y_pred (str) – The name of the column containing your model predictions.

• threshold (Threshold) – The Threshold instance that determines how the lower and upper threshold values will be calculated.

• y_pred_proba (Union[str, Dict[str, str]]) –

  Name(s) of the column(s) containing your model output.

  • For binary classification, pass a single string refering to the model output column.

  • For multiclass classification, pass a dictionary that maps a class string to the column name containing model outputs for that class.

class nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationF1(y_true: str, y_pred: str, threshold: nannyml.thresholds.Threshold, y_pred_proba: Optional[Union[str, Dict[str, str]]] = None, **kwargs)

Bases: nannyml.performance_calculation.metrics.base.Metric

F1 score metric.

Creates a new F1 instance.

Parameters

• y_true (str) – The name of the column containing target values.

• y_pred (str) – The name of the column containing your model predictions.

• threshold (Threshold) – The Threshold instance that determines how the lower and upper threshold values will be calculated.

• y_pred_proba (Union[str, Dict[str, str]]) –

  Name(s) of the column(s) containing your model output.

  • For binary classification, pass a single string refering to the model output column.

  • For multiclass classification, pass a dictionary that maps a class string to the column name containing model outputs for that class.

class nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationPrecision(y_true: str, y_pred: str, threshold: nannyml.thresholds.Threshold, y_pred_proba: Optional[Union[str, Dict[str, str]]] = None, **kwargs)

Bases: nannyml.performance_calculation.metrics.base.Metric

Precision metric.

Creates a new Precision instance.

Parameters

• y_true (str) – The name of the column containing target values.

• y_pred (str) – The name of the column containing your model predictions.

• threshold (Threshold) – The Threshold instance that determines how the lower and upper threshold values will be calculated.

• y_pred_proba (Union[str, Dict[str, str]]) –

  Name(s) of the column(s) containing your model output.

  • For binary classification, pass a single string refering to the model output column.

  • For multiclass classification, pass a dictionary that maps a class string to the column name containing model outputs for that class.

class nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationRecall(y_true: str, y_pred: str, threshold: nannyml.thresholds.Threshold, y_pred_proba: Optional[Union[str, Dict[str, str]]] = None, **kwargs)

Bases: nannyml.performance_calculation.metrics.base.Metric

Recall metric, also known as ‘sensitivity’.

Creates a new Recall instance.

Parameters

• y_true (str) – The name of the column containing target values.

• y_pred (str) – The name of the column containing your model predictions.

• threshold (Threshold) – The Threshold instance that determines how the lower and upper threshold values will be calculated.

• y_pred_proba (Union[str, Dict[str, str]]) –

  Name(s) of the column(s) containing your model output.

  • For binary classification, pass a single string refering to the model output column.

  • For multiclass classification, pass a dictionary that maps a class string to the column name containing model outputs for that class.

class nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationSpecificity(y_true: str, y_pred: str, threshold: nannyml.thresholds.Threshold, y_pred_proba: Optional[Union[str, Dict[str, str]]] = None, **kwargs)

Bases: nannyml.performance_calculation.metrics.base.Metric

Specificity metric.

Creates a new Specificity instance.

Parameters

• y_true (str) – The name of the column containing target values.

• y_pred (str) – The name of the column containing your model predictions.

• threshold (Threshold) – The Threshold instance that determines how the lower and upper threshold values will be calculated.

• y_pred_proba (Union[str, Dict[str, str]]) –

  Name(s) of the column(s) containing your model output.

  • For binary classification, pass a single string refering to the model output column.

  • For multiclass classification, pass a dictionary that maps a class string to the column name containing model outputs for that class.