nannyml.performance_calculation.result module

Module containing the results of performance calculations and associated plots.

class nannyml.performance_calculation.result.PerformanceCalculatorResult(performance_data: pandas.core.frame.DataFrame, model_metadata: nannyml.metadata.base.ModelMetadata)[source]

Bases: object

Contains the results of performance calculation and adds plotting functionality.

Creates a new PerformanceCalculatorResult instance.

Parameters

  • performance_data (pd.DataFrame) – The results of the performance calculation.

  • model_metadata – The metadata describing the monitored model.

plot(kind: str = 'performance', metric: Optional[Union[str, nannyml.performance_calculation.metrics.Metric]] = None, *args, **kwargs) → plotly.graph_objs._figure.Figure[source]

Render plots based on CBPE estimation results.

This function will return a plotly.graph_objects.Figure object. The following kinds of plots are available:

  • performance: a line plot rendering the estimated performance per Chunk after

    applying the calculate() method on a chunked dataset.

Parameters

  • kind (str, default='performance') – The kind of plot to render. Only the ‘performance’ plot is currently available.

  • metric (Union[str, Metric], default=None) – The name of the metric to plot. Value should be one of: - ‘roc_auc’ - ‘f1’ - ‘precision’ - ‘recall’ - ‘specificity’ - ‘accuracy’

Examples

>>> import nannyml.metadata.extraction
>>> import nannyml as nml
>>> ref_df, ana_df, _ = nml.load_synthetic_binary_classification_dataset()
>>> metadata = nannyml.metadata.extraction.extract_metadata(ref_df)
>>> calculator = nml.PerformanceCalculator(model_metadata=metadata, chunk_period='W')
>>> calculator.fit(ref_df)
>>> realized_performance = calculator.calculate(ana_df)
>>> # plot the calculated performance metrics
>>> for m in calculator.metrics:
>>>     realized_performance.plot(kind='performance', metric=m).show()