nannyml.drift.model_outputs.univariate.statistical.calculator module
Calculates drift for model predictions and model outputs using statistical tests.
- class nannyml.drift.model_outputs.univariate.statistical.calculator.StatisticalOutputDriftCalculator(y_pred: str, problem_type: Union[str, ProblemType], y_pred_proba: Optional[Union[str, Dict[str, str]]] = None, timestamp_column_name: Optional[str] = None, chunk_size: Optional[int] = None, chunk_number: Optional[int] = None, chunk_period: Optional[str] = None, chunker: Optional[Chunker] = None)[source]
Bases:
AbstractCalculator
Calculates drift for model predictions and model outputs using statistical tests.
Creates a new StatisticalOutputDriftCalculator.
- Parameters:
y_pred_proba (ModelOutputsType) – Name(s) of the column(s) containing your model output. Pass a single string when there is only a single model output column, e.g. in binary classification cases. Pass a dictionary when working with multiple output columns, e.g. in multiclass classification cases. The dictionary maps a class/label string to the column name containing model outputs for that class/label.
y_pred (str) – The name of the column containing your model predictions.
timestamp_column_name (str, default=None) – The name of the column containing the timestamp of the model prediction.
chunk_size (int, default=None) – Splits the data into chunks containing chunks_size observations. Only one of chunk_size, chunk_number or chunk_period should be given.
chunk_number (int, default=None) – Splits the data into chunk_number pieces. Only one of chunk_size, chunk_number or chunk_period should be given.
chunk_period (str, default=None) – Splits the data according to the given period. Only one of chunk_size, chunk_number or chunk_period should be given.
chunker (Chunker, default=None) – The Chunker used to split the data sets into a lists of chunks.
Examples
>>> import nannyml as nml >>> from IPython.display import display >>> reference_df = nml.load_synthetic_binary_classification_dataset()[0] >>> analysis_df = nml.load_synthetic_binary_classification_dataset()[1] >>> display(reference_df.head()) >>> calc = nml.StatisticalOutputDriftCalculator( ... y_pred='y_pred', ... y_pred_proba='y_pred_proba', ... timestamp_column_name='timestamp', ... problem_type='classification_binary' >>> ) >>> calc.fit(reference_df) >>> results = calc.calculate(analysis_df) >>> display(results.data) >>> score_drift_fig = results.plot(kind='score_drift', plot_reference=True) >>> score_drift_fig.show() >>> score_distribution_fig = results.plot(kind='score_distribution', plot_reference=True) >>> score_distribution_fig.show() >>> prediction_drift_fig = results.plot(kind='prediction_drift', plot_reference=True) >>> prediction_drift_fig.show() >>> prediction_distribution_fig = results.plot(kind='prediction_distribution', plot_reference=True) >>> prediction_distribution_fig.show()