NannyML
main

Contents:

  • Quickstart
    • What is NannyML?
    • Installing NannyML
    • Contents of the quickstart
    • Just the code
    • Walkthrough
      • Estimating Performance without Targets
      • Detecting Data Drift
    • Insights
    • What next
  • Tutorials
    • Setting Up
      • Data requirements
        • Data Periods
        • Common columns
        • Binary classification columns
        • Multiclass classification columns
        • Regression columns
        • What next
      • Providing metadata
        • Why is providing metadata required?
        • Metadata for binary classification
        • Metadata for multiclass classification
        • Metadata for regression
        • Insights
        • What next
      • Chunking
        • Why do we need chunks?
        • Walkthrough on creating chunks
        • Chunks on plots with results
    • Estimating Performance
      • Estimating Performance for Binary Classification
        • Why Perform Performance Estimation
        • Insights
        • What’s next
      • Estimating Performance for Multiclass Classification
        • Why Perform Performance Estimation
        • Insights
        • What’s next
      • Estimating Performance for Multiclass Classification
    • Monitoring Realized Performance
      • Monitoring Realized Performance for Binary Classification
        • Why Monitor Realized Performance
        • Insights
        • What Next
      • Monitoring Realized Performance for Multiclass Classification
        • Why Monitor Realized Performance
        • Insights
        • What Next
      • Monitoring Realized Performance for Regression
    • Comparing Estimated and Realized Performance
    • Detecting Data Drift
      • Univariate Drift Detection
        • Why Perform Univariate Drift Detection
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
      • Multivariate Data Drift Detection
        • Why Perform Multivariate Drift Detection
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
      • Drift Detection for Model Outputs
        • Why Perform Drift Detection for Model Outputs
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
      • Drift Detection for Model Targets
        • Why Perform Drift Detection for Model Targets
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
    • Adjusting Plots
  • How It Works
    • Metadata extraction
      • Naming conventions
        • Common metadata columns
        • Binary classification columns
        • Multiclass classification columns
      • Feature type detection
    • Data Reconstruction with PCA
      • Limitations of Univariate Drift Detection
        • “Butterfly” Dataset
      • Data Reconstruction with PCA
      • Understanding Reconstruction Error with PCA
        • Reconstruction Error with PCA on the butterfly dataset
    • Confidence-based Performance Estimation (CBPE)
      • CBPE algorithm
        • Binary classification
        • Multiclass Classification
      • Assumptions and Limitations
      • Appendix: Probability calibration
    • Chunking data
      • Chunking considerations
        • Different periods within one chunk
        • Underpopulated chunks
        • Not enough chunks
      • Minimum chunk size
        • Minimum Chunk Size for Performance Estimation and Performance Monitoring
        • Minimum Chunk Size for Multivariate Drift
        • Minimum Chunk for Univariate Drift
  • Examples
    • Binary Classification: California Housing Dataset
      • Load and prepare data
      • Performance Estimation
      • Comparison with the actual performance
      • Drift detection
  • Troubleshooting
    • Dealing with a MissingMetadataException
      • The problem
      • The solution
      • Related reads
  • Example Datasets
    • Synthetic Binary Classification Dataset
      • Problem Description
      • Dataset Description
      • Metadata Extraction
    • Synthetic Multiclass Classification Dataset
      • Problem Description
      • Dataset Description
      • Metadata Extraction
    • California Housing Dataset
      • Modifying California Housing Dataset
      • Enriching the data
      • Training a Machine Learning Model
      • Meeting NannyML Data Requirements
  • Glossary
  • API reference
    • nannyml package
      • Subpackages
        • nannyml.datasets package
        • nannyml.drift package
        • nannyml.metadata package
        • nannyml.performance_calculation package
        • nannyml.performance_estimation package
        • nannyml.plots package
      • Submodules
        • nannyml.calibration module
        • nannyml.chunk module
        • nannyml.exceptions module
        • nannyml.preprocessing module
      • Module contents
  • Contributing
    • Spread the word
    • Be a part of the team
    • Contribute to the codebase
      • Get started coding
      • Pull Request Guidelines
      • Tips
NannyML
  • »
  • Tutorials
  • Edit on GitHub

Tutorials

  • Setting Up
    • Data requirements
    • Providing metadata
    • Chunking
  • Estimating Performance
    • Estimating Performance for Binary Classification
    • Estimating Performance for Multiclass Classification
    • Estimating Performance for Multiclass Classification
  • Monitoring Realized Performance
    • Monitoring Realized Performance for Binary Classification
    • Monitoring Realized Performance for Multiclass Classification
    • Monitoring Realized Performance for Regression
  • Comparing Estimated and Realized Performance
  • Detecting Data Drift
    • Univariate Drift Detection
    • Multivariate Data Drift Detection
    • Drift Detection for Model Outputs
    • Drift Detection for Model Targets
  • Adjusting Plots
Previous Next

© Copyright 2022, NannyML. Revision 72b1aad7.

Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: main
Versions
latest
stable
v0.4.1
v0.4.0
v0.3.2
v0.3.1
v0.3.0
v0.2.1
v0.2.0
main
Downloads
On Read the Docs
Project Home
Builds