NannyML
v0.5.3

Contents:

  • Quickstart
    • What is NannyML?
    • Installing NannyML
    • Contents of the quickstart
    • Just the code
    • Walkthrough
      • Estimating Performance without Targets
      • Detecting Data Drift
    • Insights
    • What next
  • Tutorials
    • Data requirements
      • Data Periods
        • Reference Period
        • Analysis Period
      • Columns
        • Timestamp
        • Target
        • Features
      • Model Output columns
        • Predicted class probabilities
        • Prediction class labels
      • NannyML Functionality Requirements
      • What next
    • Estimating Performance
      • Why Perform Performance Estimation
      • Estimating Performance for Binary Classification
        • Just The Code
        • Walkthrough
        • Insights
        • What’s next
      • Estimating Performance for Multiclass Classification
        • Just The Code
        • Walkthrough
        • Insights
        • What’s next
      • Estimating Performance for Regression
    • Monitoring Realized Performance
      • Why Monitoring Realized Performance
      • Monitoring Realized Performance for Binary Classification
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
      • Monitoring Realized Performance for Multiclass Classification
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
      • Monitoring Realized Performance for Regression
    • Comparing Estimated and Realized Performance
    • Detecting Data Drift
      • Univariate Drift Detection
        • Why Perform Univariate Drift Detection
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
      • Multivariate Data Drift Detection
        • Why Perform Multivariate Drift Detection
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
      • Drift Detection for Model Outputs
        • Drift Detection for Binary Classification Model Outputs
          • Why Perform Drift Detection for Model Outputs
          • Just The Code
          • Walkthrough
          • Insights
          • What Next
        • Drift Detection for Multiclass Classification Model Outputs
          • Why Perform Drift Detection for Model Outputs
          • Just The Code
          • Walkthrough
          • What Next
      • Drift Detection for Model Targets
        • Drift Detection for Binary Classification Model Targets
          • Why Perform Drift Detection for Model Targets
          • Just The Code
          • Walkthrough
          • Insights
          • What Next
        • Drift Detection for Multiclass Classification Model Targets
          • Why Perform Drift Detection for Model Targets
          • Just The Code
          • Walkthrough
          • What Next
    • Adjusting Plots
    • Chunking
      • Why do we need chunks?
      • Walkthrough on creating chunks
        • Time-based chunking
        • Size-based chunking
        • Number-based chunking
        • Automatic chunking
      • Chunks on plots with results
  • How It Works
    • Confidence-based Performance Estimation (CBPE)
      • CBPE algorithm
        • Binary classification
        • Multiclass Classification
      • Assumptions and Limitations
      • Appendix: Probability calibration
    • Data Reconstruction with PCA
      • Limitations of Univariate Drift Detection
        • “Butterfly” Dataset
      • Data Reconstruction with PCA
      • Understanding Reconstruction Error with PCA
        • Reconstruction Error with PCA on the butterfly dataset
    • Chunking Considerations
      • Not Enough Chunks
      • Not Enough Observations in Chunk
      • Impact of Chunk Size on Reliability of Results
    • Estimation of Standard Error
      • Adapting Standard Error of the Mean Formula
      • Standard Error Estimation and Interpretation for NannyML features
        • Performance Estimation
        • Performance Monitoring
        • Multivariate Drift Detection with PCA
        • Univariate Drift Detection
      • Assumptions and Limitations
  • Examples
    • Binary Classification: California Housing Dataset
      • Load and prepare data
      • Performance Estimation
      • Comparison with the actual performance
      • Drift detection
  • Example Datasets
    • Synthetic Binary Classification Dataset
      • Problem Description
      • Dataset Description
    • Synthetic Multiclass Classification Dataset
      • Problem Description
      • Dataset Description
    • California Housing Dataset
      • Modifying California Housing Dataset
      • Enriching the data
      • Training a Machine Learning Model
      • Meeting NannyML Data Requirements
  • Glossary
  • CLI nml
    • Running the CLI
      • Installation
      • Configuration
    • Configuration file
      • Locations
      • Format
        • Input section
        • Output section
        • Column mapping section
        • Chunker section
        • Standalone parameters section
      • Templating paths
      • Examples
    • Command overview
      • run
        • Syntax
        • Options
        • Example
  • API reference
    • nannyml package
      • Subpackages
        • nannyml.cli package
          • Submodules
          • Module contents
        • nannyml.datasets package
          • Subpackages
          • Submodules
          • Module contents
        • nannyml.drift package
          • Subpackages
          • Submodules
          • Module contents
        • nannyml.io package
          • Submodules
          • Module contents
        • nannyml.performance_calculation package
          • Submodules
          • Module contents
        • nannyml.performance_estimation package
          • Subpackages
          • Module contents
        • nannyml.plots package
          • Submodules
          • Module contents
        • nannyml.sampling_error package
          • Submodules
          • Module contents
      • Submodules
        • nannyml.base module
        • nannyml.calibration module
        • nannyml.chunk module
        • nannyml.config module
        • nannyml.exceptions module
        • nannyml.runner module
      • Module contents
  • Contributing
    • Spread the word
    • Be a part of the team
    • Contribute to the codebase
      • Get started coding
      • Pull Request Guidelines
      • Tips
NannyML
  • »
  • Welcome to NannyML’s documentation!
  • Edit on GitHub

Welcome to NannyML’s documentation!

PyPi Supported versions coverage

Contents:

  • Quickstart
    • What is NannyML?
    • Installing NannyML
    • Contents of the quickstart
    • Just the code
    • Walkthrough
    • Insights
    • What next
  • Tutorials
    • Data requirements
    • Estimating Performance
    • Monitoring Realized Performance
    • Comparing Estimated and Realized Performance
    • Detecting Data Drift
    • Adjusting Plots
    • Chunking
  • How It Works
    • Confidence-based Performance Estimation (CBPE)
    • Data Reconstruction with PCA
    • Chunking Considerations
    • Estimation of Standard Error
  • Examples
    • Binary Classification: California Housing Dataset
  • Example Datasets
    • Synthetic Binary Classification Dataset
    • Synthetic Multiclass Classification Dataset
    • California Housing Dataset
  • Glossary
  • CLI nml
    • Running the CLI
    • Configuration file
    • Command overview
  • API reference
    • nannyml package
  • Contributing
    • Spread the word
    • Be a part of the team
    • Contribute to the codebase

Indices and tables

  • Index

  • Module Index

  • Search Page

Next

© Copyright 2022, NannyML. Revision 6a6f2bae.

Built with Sphinx using a theme provided by Read the Docs.