nannyml.performance_estimation.confidence_based package

Submodules

Module contents

Package containing the Confidence Based Performance Estimator (CBPE).

Assuming properly calibrated probabilities, confusion matrix elements can be estimated and then used to calculate any performance metric. Given the assumptions are met, CBPE provides an unbiased estimation of the performance of the monitored model based on the monitored model’s outputs only (i.e. without access to targets).

For more information, check out the tutorial for binary classification, the tutorial for multiclass classification or the deep dive.