NannyML
stable

Contents:

  • Quickstart
    • What is NannyML?
    • Installing NannyML
    • Contents of the quickstart
    • Just the code
    • Walkthrough
      • Estimating Performance without Targets
      • Detecting Data Drift
    • Insights
    • What next
  • Tutorials
    • Data requirements
      • Data Periods
        • Reference Period
        • Analysis Period
      • Columns
        • Timestamp
        • Target
        • Features
      • Model Output columns
        • Predicted class probabilities
        • Prediction class labels
      • NannyML Functionality Requirements
      • What next
    • Estimating Performance
      • Why Perform Performance Estimation
      • Estimating Performance for Binary Classification
        • Just The Code
        • Walkthrough
        • Insights
        • What’s next
      • Estimating Performance for Multiclass Classification
        • Just The Code
        • Walkthrough
        • Insights
        • What’s next
      • Estimating Performance for Regression
        • Just The Code
        • Walkthrough
        • Insights
        • What’s next
    • Monitoring Realized Performance
      • Why Monitoring Realized Performance
      • Monitoring Realized Performance for Binary Classification
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
      • Monitoring Realized Performance for Multiclass Classification
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
      • Monitoring Realized Performance for Regression
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
    • Comparing Estimated and Realized Performance
    • Detecting Data Drift
      • Univariate Drift Detection
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
      • Multivariate Drift Detection
        • Why Perform Multivariate Drift Detection
        • Just The Code
        • Walkthrough
        • Insights
        • What Next
    • Working with results
      • What are NannyML Results?
      • Just the code
      • Walkthrough
    • Adjusting Plots
    • Chunking
      • Why do we need chunks?
      • Walkthrough on creating chunks
        • Time-based chunking
        • Size-based chunking
        • Number-based chunking
        • Automatic chunking
      • Chunks on plots with results
  • How It Works
    • Estimation of Performance of the Monitored Model
      • Confidence-based Performance Estimation (CBPE)
        • The Intuition
        • Implementation details
          • Binary classification
          • Multiclass Classification
        • Assumptions and Limitations
        • Appendix: Probability calibration
      • Direct Loss Estimation (DLE)
        • The Intuition
        • Implementation details
        • Assumptions and limitations
      • Other Approaches to Estimate Performance of Regression Models
        • Bayesian approaches
        • Conformalized Quantile Regression
        • Conclusions from Bayesian and Conformalized Quantile Regression approaches
    • Univariate Drift Detection
      • Methods for Continuous Features
        • Kolmogorov-Smirnov Test
        • Jensen-Shannon Distance
        • Wasserstein Distance
      • Methods for Categorical Variables
        • Chi-squared Test
        • Jensen-Shannon Distance
        • L-Infinity Distance
    • Data Reconstruction with PCA
      • Limitations of Univariate Drift Detection
        • “Butterfly” Dataset
      • Data Reconstruction with PCA
      • Understanding Reconstruction Error with PCA
        • Reconstruction Error with PCA on the butterfly dataset
    • Chunking Considerations
      • Not Enough Chunks
      • Not Enough Observations in Chunk
      • Impact of Chunk Size on Reliability of Results
    • Calculating Sampling Error
      • Defining Sampling Error from Standard Error of the Mean
      • Sampling Error Estimation and Interpretation for NannyML features
        • Performance Estimation
        • Performance Monitoring
        • Multivariate Drift Detection with PCA
        • Univariate Drift Detection
      • Assumptions and Limitations
  • Examples
    • Binary Classification: California Housing Dataset
      • Load and prepare data
      • Performance Estimation
      • Comparison with the actual performance
      • Drift detection
  • Example Datasets
    • Synthetic Binary Classification Dataset
      • Problem Description
      • Dataset Description
    • Synthetic Multiclass Classification Dataset
      • Problem Description
      • Dataset Description
    • California Housing Dataset
      • Modifying California Housing Dataset
      • Enriching the data
      • Training a Machine Learning Model
      • Meeting NannyML Data Requirements
    • Synthetic Regression Dataset
      • Problem Description
      • Dataset Description
  • Glossary
  • Command Line Interface (CLI)
    • Running the CLI
      • Installation
      • Configuration
    • Configuration file
      • Locations
      • Format
        • Input section
        • Output section
          • Writing to filesystem
          • Writing to a pickle file
          • Writing to a relational database
        • Column mapping section
        • Chunker section
        • Scheduling section
        • Standalone parameters section
      • Templating paths
      • Examples
    • Command overview
      • run
        • Syntax
        • Options
        • Example
  • Usage logging in NannyML
    • TLDR
    • What do we mean by usage statistics?
      • What about personal data
      • What about my dataset?
    • Why are we doing this?
      • Improving NannyML and prioritizing new features
      • Surviving as a company
    • How usage logging works
    • To opt in or not to opt in, that’s the question
    • How to disable usage logging
      • Setting the environment variable
      • Providing a .env file
      • Turning off user analytics in code
  • API reference
    • nannyml package
      • Subpackages
        • nannyml.cli package
          • Submodules
          • Module contents
        • nannyml.datasets package
          • Subpackages
          • Submodules
          • Module contents
        • nannyml.drift package
          • Subpackages
          • Submodules
          • Module contents
        • nannyml.io package
          • Subpackages
          • Submodules
          • Module contents
        • nannyml.performance_calculation package
          • Subpackages
          • Submodules
          • Module contents
        • nannyml.performance_estimation package
          • Subpackages
          • Module contents
        • nannyml.plots package
          • Submodules
          • Module contents
        • nannyml.sampling_error package
          • Submodules
          • Module contents
      • Submodules
        • nannyml.analytics module
          • SegmentUsageTracker
          • UsageEvent
          • UsageTracker
          • track()
        • nannyml.base module
          • AbstractCalculator
          • AbstractCalculatorResult
          • AbstractEstimator
          • AbstractEstimatorResult
        • nannyml.calibration module
          • Calibrator
          • CalibratorFactory
          • IsotonicCalibrator
          • NoopCalibrator
          • needs_calibration()
        • nannyml.chunk module
          • Chunk
          • Chunker
          • ChunkerFactory
          • CountBasedChunker
          • DefaultChunker
          • PeriodBasedChunker
          • SizeBasedChunker
        • nannyml.config module
          • ChunkerConfig
          • ColumnMapping
          • Config
          • CronSchedulingConfig
          • DatabaseWriterConfig
          • InputConfig
          • InputDataConfig
          • IntervalSchedulingConfig
          • PickleWriterConfig
          • RawFileWriterConfig
          • SchedulingConfig
          • TargetDataConfig
          • WriterConfig
          • get_config_path()
        • nannyml.exceptions module
          • CalculatorException
          • CalculatorNotFittedException
          • ChunkerException
          • EstimatorException
          • IOException
          • InvalidArgumentsException
          • InvalidReferenceDataException
          • MissingMetadataException
          • NotFittedException
          • ReaderException
          • WriterException
        • nannyml.runner module
          • run()
        • nannyml.usage_logging module
          • SegmentUsageTracker
          • UsageEvent
          • UsageLogger
          • disable_usage_logging()
          • enable_usage_logging()
          • get_logger()
          • log_usage()
      • Module contents
  • Contributing
    • Spread the word
    • Be a part of the team
    • Contribute to the codebase
      • Get started coding
      • Pull Request Guidelines
      • Tips
NannyML
  • nannyml
  • nannyml package
  • nannyml.plots package
  • Edit on GitHub

nannyml.plots package

Submodules

  • nannyml.plots.colors module
    • Colors
      • Colors.BLUE_SKY_CRAYOLA
      • Colors.GRAY
      • Colors.GRAY_DARK
      • Colors.GREEN_SEA
      • Colors.INDIGO_PERSIAN
      • Colors.LIGHT_GRAY
      • Colors.RED_IMPERIAL
      • Colors.SAFFRON

Module contents

Module containing plotting implementations.

Previous Next

© Copyright 2022, NannyML. Revision 22f24137.

Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: stable
Versions
latest
stable
v0.8.0
v0.7.0
v0.6.3
v0.6.2
v0.6.1
v0.6.0
v0.5.3
v0.5.2
v0.5.1
v0.5.0
v0.4.1
v0.4.0
v0.3.2
v0.3.1
v0.3.0
v0.2.1
v0.2.0
main
Downloads
On Read the Docs
Project Home
Builds