NannyML
v0.5.0
Contents:
Quickstart
What is NannyML?
Installing NannyML
Contents of the quickstart
Just the code
Walkthrough
Estimating Performance without Targets
Detecting Data Drift
Insights
What next
Tutorials
Data requirements
Data Periods
Reference Period
Analysis Period
Columns
Timestamp
Target
Features
Model Output columns
Predicted class probabilities
Prediction class labels
NannyML Functionality Requirements
What next
Estimating Performance
Why Perform Performance Estimation
Estimating Performance for Binary Classification
Just The Code
Walkthrough
Insights
What’s next
Estimating Performance for Multiclass Classification
Just The Code
Walkthrough
Insights
What’s next
Estimating Performance for Regression
Monitoring Realized Performance
Why Monitoring Realized Performance
Monitoring Realized Performance for Binary Classification
Just The Code
Walkthrough
Insights
What Next
Monitoring Realized Performance for Multiclass Classification
Just The Code
Walkthrough
Insights
What Next
Monitoring Realized Performance for Regression
Comparing Estimated and Realized Performance
Detecting Data Drift
Univariate Drift Detection
Why Perform Univariate Drift Detection
Just The Code
Walkthrough
Insights
What Next
Multivariate Data Drift Detection
Why Perform Multivariate Drift Detection
Just The Code
Walkthrough
Insights
What Next
Drift Detection for Model Outputs
Drift Detection for Binary Classification Model Outputs
Why Perform Drift Detection for Model Outputs
Just The Code
Walkthrough
Insights
What Next
Drift Detection for Multiclass Classification Model Outputs
Why Perform Drift Detection for Model Outputs
Just The Code
Walkthrough
What Next
Drift Detection for Model Targets
Drift Detection for Binary Classification Model Targets
Why Perform Drift Detection for Model Targets
Just The Code
Walkthrough
Insights
What Next
Drift Detection for Multiclass Classification Model Targets
Why Perform Drift Detection for Model Targets
Just The Code
Walkthrough
What Next
Adjusting Plots
Chunking
Why do we need chunks?
Walkthrough on creating chunks
Time-based chunking
Size-based chunking
Number-based chunking
Automatic chunking
Chunks on plots with results
How It Works
Data Reconstruction with PCA
Limitations of Univariate Drift Detection
“Butterfly” Dataset
Data Reconstruction with PCA
Understanding Reconstruction Error with PCA
Reconstruction Error with PCA on the butterfly dataset
Confidence-based Performance Estimation (CBPE)
CBPE algorithm
Binary classification
Multiclass Classification
Assumptions and Limitations
Appendix: Probability calibration
Chunking data
Chunking considerations
Underpopulated chunks
Not enough chunks
Minimum chunk size
Minimum Chunk Size for Performance Estimation and Performance Monitoring
Minimum Chunk Size for Multivariate Drift
Minimum Chunk for Univariate Drift
Examples
Binary Classification: California Housing Dataset
Load and prepare data
Performance Estimation
Comparison with the actual performance
Drift detection
Example Datasets
Synthetic Binary Classification Dataset
Problem Description
Dataset Description
Synthetic Multiclass Classification Dataset
Problem Description
Dataset Description
California Housing Dataset
Modifying California Housing Dataset
Enriching the data
Training a Machine Learning Model
Meeting NannyML Data Requirements
Glossary
API reference
nannyml package
Subpackages
nannyml.datasets package
Subpackages
Submodules
Module contents
nannyml.drift package
Subpackages
Submodules
Module contents
nannyml.io package
Submodules
Module contents
nannyml.performance_calculation package
Submodules
Module contents
nannyml.performance_estimation package
Subpackages
Module contents
nannyml.plots package
Submodules
Module contents
Submodules
nannyml.base module
nannyml.calibration module
nannyml.chunk module
nannyml.exceptions module
nannyml.runner module
Module contents
Contributing
Spread the word
Be a part of the team
Contribute to the codebase
Get started coding
Pull Request Guidelines
Tips
NannyML
»
nannyml
»
nannyml package
»
nannyml.runner module
Edit on GitHub
nannyml.runner module
Read the Docs
v: v0.5.0
Versions
latest
stable
v0.5.0
v0.4.1
v0.4.0
v0.3.2
v0.3.1
v0.3.0
v0.2.1
v0.2.0
main
Downloads
On Read the Docs
Project Home
Builds