NannyML
stable
Contents:
Introduction
What is NannyML?
Key features
➖ Performance Estimation and Calculation
➖ Business Value Estimation and Calculation
➖ Data Quality
➖ Multivariate Drift Detection
➖ Univariate Drift Detection
➖ Custom Thresholds
Next steps
Get early access to NannyML Web App
Installing NannyML
Extras
Quickstart
What is NannyML?
Exemplary Workflow with NannyML
Loading data
Estimating Performance without Targets
Investigating Data Distribution Shifts
Comparing Estimated with Realized Performance when Targets Arrive
What’s next?
Tutorials
Data requirements
Data Periods
Reference Period
Analysis Period
Columns
Timestamp
Target
Features
Model Output columns
Predicted class probabilities
Prediction class labels
NannyML Functionality Requirements
What’s next
Estimating Performance
Why Estimate Performance
Estimating Performance for Binary Classification
Estimating Standard Performance Metrics for Binary Classification
Just The Code
Walkthrough
Insights
What’s next
Estimating Confusion Matrix Elements for Binary Classification
Just The Code
Walkthrough
Insights
What’s next
Estimating Business Value for Binary Classification
Just The Code
Walkthrough
Insights
What’s next
Creating and Estimating a Custom Binary Classification Metric
Just the Code
Walkthrough
Insights
What’s next
Estimating Performance for Multiclass Classification
Estimating Performance for Multiclass Classification
Just The Code
Walkthrough
Insights
What’s next
Estimating Confusion Matrix Elements for Multiclass Classification
Just The Code
Walkthrough
Insights
What’s next
Estimating Business Value for Multiclass Classification
Just The Code
Walkthrough
Insights
What’s next
Estimating Performance for Regression
Just The Code
Walkthrough
Insights
What’s next
Monitoring Realized Performance
Why Monitor Realized Performance
Monitoring Realized Performance for Binary Classification
Calculating Standard Performance Metrics for Binary Classification
Just The Code
Walkthrough
Insights
What’s Next
Calculating Confusion Matrix Elements for Binary Classification
Just The Code
Walkthrough
Insights
What’s Next
Calculating Business Value for Binary Classification
Just The Code
Walkthrough
Insights
What’s Next
Monitoring Realized Performance for Multiclass Classification
Calculating Standard Performance Metrics for Multiclass Classification
Just The Code
Walkthrough
Insights
What Next
Calculating Confusion Matrix Elements for Multiclass Classification
Just The Code
Walkthrough
Insights
What’s Next
Calculating Business Value for Multiclass Classification
Just The Code
Walkthrough
Insights
What’s Next
Monitoring Realized Performance for Regression
Just The Code
Walkthrough
Insights
What Next
Comparing Estimated and Realized Performance
Just the code
Walkthrough
Estimating performance without targets
Comparing to realized performance
Detecting Data Drift
Univariate Drift Detection
Just The Code
Walkthrough
Insights
What Next
Multivariate Drift Detection
Data Reconstruction with PCA
Just The Code
Walkthrough
Insights
What Next
Domain Classifier
Just The Code
Walkthrough
Insights
What Next
Ranking
Just The Code
Walkthrough
Alert Count Ranking
Correlation Ranking
Insights
What’s Next
Data Quality Checks
Missing Values Detection
Just The Code
Walkthrough
Insights
What Next
Unseen Values Detection
Just The Code
Walkthrough
Insights
What Next
Summary Statistics
Summation
Just The Code
Walkthrough
Insights
What Next
Average
Just The Code
Walkthrough
Insights
What Next
Standard Deviation
Just The Code
Walkthrough
Insights
What Next
Median
Just The Code
Walkthrough
Insights
What Next
Rows Count
Just The Code
Walkthrough
Insights
What Next
Storing and loading calculators
Just the code
Walkthrough
What’s Next
Working with results
What are NannyML Results?
Just the code
Walkthrough
The data structure
Filtering
Plotting
Comparing
Exporting
Adjusting Plots
Chunking
Why do we need chunks?
Walkthrough on creating chunks
Time-based chunking
Size-based chunking
Number-based chunking
Automatic chunking
Customize chunk behavior
Chunks on plots with results
Thresholds
Just the code
Walkthrough
Constant thresholds
Standard deviation thresholds
Setting custom thresholds for calculators and estimators
Default thresholds
What’s next?
How It Works
Estimation of Performance of the Monitored Model
Confidence-based Performance Estimation (CBPE)
The Intuition
Implementation details
Binary classification
Multiclass Classification
Assumptions and Limitations
Appendix: Probability calibration
Direct Loss Estimation (DLE)
The Intuition
Implementation details
Assumptions and limitations
Other Approaches to Estimate Performance of Regression Models
Bayesian approaches
Conformalized Quantile Regression
Conclusions from Bayesian and Conformalized Quantile Regression approaches
Business Value Estimation and Calculation
Introduction to Business Value
Business Value Formula
Calculation of Business Value For Classification
Estimation of Business Value For Classification
Normalization
Presenting Univariate Drift Detection Methods
Methods for Continuous Features
Kolmogorov-Smirnov Test
Jensen-Shannon Distance
Wasserstein Distance
Hellinger Distance
Methods for Categorical Variables
Chi-squared Test
Jensen-Shannon Distance
Hellinger Distance
L-Infinity Distance
Choosing Univariate Drift Detection Methods
Comparison of Methods for Continuous Variables
Shifting the Mean of the Analysis Data Set
Shifting the Standard Deviation of the Analysis Data Set
Tradeoffs of The Kolmogorov-Smirnov Statistic
Tradeoffs of Jensen-Shannon Distance and Hellinger Distance
Experiment 1
Experiment 2
Tradeoffs of Wasserstein Distance
Experiment 1
Experiment 2
Experiment 3
Comparison of Methods for Categorical Variables
Sensitivity to Sample Size of Different Drift Measures
Behavior When a Category Slowly Disappears
Behavior When Observations from a New Category Occur
Effect of Sample Size on Different Drift Measures
Effect of the Number of Categories on Different Drift Measures
Comparison of Drift Methods on Data Sets with Many Categories
Results Summary (TLDR)
Methods for Continuous Variables
Methods For Categorical Variables
Ranking
Alert Count Ranking
Correlation Ranking
Multivariate Drift Detection
Limitations of Univariate Drift Detection
“Butterfly” Dataset
Data Reconstruction with PCA
Understanding Reconstruction Error with PCA
Reconstruction Error with PCA on the butterfly dataset
Domain Classifier
Understanding Domain Classifier
Domain Classifier on the butterfly dataset
Chunking Considerations
Not Enough Chunks
Not Enough Observations in Chunk
Impact of Chunk Size on Reliability of Results
Calculating Sampling Error
Defining Sampling Error from Standard Error of the Mean
Sampling Error Estimation and Interpretation for NannyML features
Performance Estimation
Performance Monitoring
Multivariate Drift Detection with PCA
Univariate Drift Detection
Summary Statistics
Average
Summation
Standard Deviation
Median
Assumptions and Limitations
Thresholds
Threshold basics
Constant thresholds
Standard deviation thresholds
Examples
Binary Classification: California Housing Dataset
Load and prepare data
Performance Estimation
Comparison with the actual performance
Drift detection
Full Monitoring Workflow - Regression: NYC Green Taxi Dataset
Import libraries
Load the data
Preprocessing the data
Exploring the training data
Training a model
Evaluating the model
Deploying the model
Analysing ML model performance in production
Estimating the model’s performance
Detecting multivariate data drift
Detecting univariate data drift
Bonus: Comparing realized and estimated performance
Conclusion
Example Datasets
US Census Employment dataset
Data Source
Dataset Description
Preparing Data for NannyML
Fetching the Data
Defining Partitions and Preprocessing
Developing ML Model and Making Predictions
Splitting and Storing the Data
Appendix: Feature description
References
Synthetic Binary Classification Car Loan Dataset
Problem Description
Dataset Description
Data Quality Version
Synthetic Multiclass Classification Dataset
Problem Description
Dataset Description
Synthetic Regression Dataset
Problem Description
Dataset Description
California Housing Dataset
Modifying California Housing Dataset
Enriching the data
Training a Machine Learning Model
Meeting NannyML Data Requirements
Titanic Dataset
Problem Description
Dataset Description
Glossary
Command Line Interface (CLI)
Running the CLI
Installation
Configuration
Configuration file
Locations
Format
Input section
Output section
Writing to filesystem
Writing to a pickle file
Writing to a relational database
Column mapping section
Store section
Chunker section
Scheduling section
Standalone parameters section
Templating paths
Examples
Command overview
run
Syntax
Options
Example
Usage logging in NannyML
TLDR
What do we mean by usage statistics?
What about personal data
What about my dataset?
Why are we doing this?
Improving NannyML and prioritizing new features
Surviving as a company
How usage logging works
To opt in
or
not to opt in
, that’s the question
How to disable usage logging
Setting the environment variable
Providing a
.env
file
Turning off user analytics in code
API reference
nannyml package
Subpackages
nannyml.cli package
Submodules
Module contents
nannyml.data_quality package
Subpackages
Module contents
nannyml.datasets package
Subpackages
Submodules
Module contents
nannyml.distribution package
Subpackages
Module contents
nannyml.drift package
Subpackages
Submodules
Module contents
nannyml.io package
Subpackages
Submodules
Module contents
nannyml.performance_calculation package
Subpackages
Submodules
Module contents
nannyml.performance_estimation package
Subpackages
Module contents
nannyml.plots package
Subpackages
Submodules
Module contents
nannyml.sampling_error package
Submodules
Module contents
nannyml.stats package
Subpackages
Module contents
Submodules
nannyml.analytics module
nannyml.base module
nannyml.calibration module
nannyml.chunk module
nannyml.config module
nannyml.exceptions module
nannyml.runner module
nannyml.thresholds module
nannyml.usage_logging module
Module contents
Contributing
Spread the word
Be a part of the team
Contribute to the codebase
Get started coding
Pull Request Guidelines
Tips
NannyML
Index
Edit on GitHub
Index
_
|
A
|
B
|
C
|
D
|
E
|
F
|
G
|
H
|
I
|
J
|
K
|
L
|
M
|
N
|
O
|
P
|
R
|
S
|
T
|
U
|
V
|
W
|
Y
_
__eq__() (nannyml.drift.univariate.methods.Method method)
(nannyml.performance_calculation.metrics.base.Metric method)
(nannyml.performance_estimation.confidence_based.metrics.Metric method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.Metric method)
__len__() (nannyml.chunk.Chunk method)
__repr__() (nannyml.chunk.Chunk method)
__str__() (nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationAccuracy method)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationAP method)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationAUROC method)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationBusinessValue method)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationConfusionMatrix method)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationF1 method)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationPrecision method)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationRecall method)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationSpecificity method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationAccuracy method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationAP method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationAUROC method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationBusinessValue method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationConfusionMatrix method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationF1 method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationPrecision method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationRecall method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationSpecificity method)
(nannyml.performance_calculation.metrics.regression.MAE method)
(nannyml.performance_calculation.metrics.regression.MAPE method)
(nannyml.performance_calculation.metrics.regression.MSE method)
(nannyml.performance_calculation.metrics.regression.MSLE method)
(nannyml.performance_calculation.metrics.regression.RMSE method)
(nannyml.performance_calculation.metrics.regression.RMSLE method)
(nannyml.performance_estimation.direct_loss_estimation.dle.DLE method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.Metric method)
A
Abstract1DResult (class in nannyml.base)
Abstract2DResult (class in nannyml.base)
AbstractCalculator (class in nannyml.base)
AbstractEstimator (class in nannyml.base)
AbstractEstimatorResult (class in nannyml.base)
AbstractResult (class in nannyml.base)
accuracy_sampling_error() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
accuracy_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
add() (nannyml.plots.components.hover.Hover method)
add_alert() (nannyml.plots.components.figure.Figure method)
add_artificial_endpoint() (in module nannyml.plots.util)
add_confidence_band() (nannyml.plots.components.figure.Figure method)
add_metric() (nannyml.plots.components.figure.Figure method)
add_period_separator() (nannyml.plots.components.figure.Figure method)
add_threshold() (nannyml.plots.components.figure.Figure method)
Alert
alert() (in module nannyml.plots.components.joy_plot)
(in module nannyml.plots.components.stacked_bar_plot)
(in module nannyml.plots.components.step_plot)
(nannyml.drift.univariate.methods.Chi2Statistic method)
(nannyml.drift.univariate.methods.Method method)
(nannyml.performance_calculation.metrics.base.Metric method)
(nannyml.performance_estimation.confidence_based.metrics.Metric method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.Metric method)
AlertCountRanker (class in nannyml.drift.ranker)
alerts() (nannyml.base.AbstractResult method)
analysis_data (nannyml.config.InputConfig attribute)
(nannyml.runner.RunInput attribute)
ap_sampling_error() (in module nannyml.sampling_error.binary_classification)
ap_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
auroc_sampling_error() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
auroc_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
average_precision_sampling_error() (in module nannyml.sampling_error.multiclass_classification)
average_precision_sampling_error_components() (in module nannyml.sampling_error.multiclass_classification)
B
BinaryClassificationAccuracy (class in nannyml.performance_calculation.metrics.binary_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
BinaryClassificationAP (class in nannyml.performance_calculation.metrics.binary_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
BinaryClassificationAUROC (class in nannyml.performance_calculation.metrics.binary_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
BinaryClassificationBusinessValue (class in nannyml.performance_calculation.metrics.binary_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
BinaryClassificationConfusionMatrix (class in nannyml.performance_calculation.metrics.binary_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
BinaryClassificationF1 (class in nannyml.performance_calculation.metrics.binary_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
BinaryClassificationPrecision (class in nannyml.performance_calculation.metrics.binary_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
BinaryClassificationRecall (class in nannyml.performance_calculation.metrics.binary_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
BinaryClassificationSpecificity (class in nannyml.performance_calculation.metrics.binary_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
BLUE_SKY_CRAYOLA (nannyml.plots.colors.Colors attribute)
Business Value Matrix
business_value_sampling_error() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
business_value_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
Butterfly dataset
C
calculate() (nannyml.base.AbstractCalculator method)
(nannyml.drift.univariate.methods.Method method)
(nannyml.performance_calculation.metrics.base.Metric method)
calculate_chunk_distributions() (in module nannyml.distribution.continuous.calculator)
(in module nannyml.plots.components.joy_plot)
calculate_threshold_values() (in module nannyml.thresholds)
calculate_value_counts() (in module nannyml.distribution.categorical.calculator)
(in module nannyml.plots.components.stacked_bar_plot)
CalculatorConfig (class in nannyml.config)
CalculatorException
CalculatorFactory (class in nannyml.runner)
CalculatorNotFittedException
calculators (nannyml.config.Config attribute)
calibrate() (nannyml.calibration.Calibrator method)
(nannyml.calibration.IsotonicCalibrator method)
(nannyml.calibration.NoopCalibrator method)
Calibrator (class in nannyml.calibration)
CalibratorFactory (class in nannyml.calibration)
CATEGORICAL (nannyml.drift.univariate.methods.FeatureType attribute)
CategoricalDistributionCalculator (class in nannyml.distribution.categorical.calculator)
CategoricalHellingerDistance (class in nannyml.drift.univariate.methods)
CategoricalJensenShannonDistance (class in nannyml.drift.univariate.methods)
CBPE (class in nannyml.performance_estimation.confidence_based.cbpe)
(Confidence-Based Performance Estimation)
CBPE_ESTIMATOR_FIT (nannyml.analytics.UsageEvent attribute)
(nannyml.usage_logging.UsageEvent attribute)
CBPE_ESTIMATOR_RUN (nannyml.analytics.UsageEvent attribute)
(nannyml.usage_logging.UsageEvent attribute)
CBPE_PLOT (nannyml.usage_logging.UsageEvent attribute)
check_and_convert() (in module nannyml.plots.util)
check_is_compatible_with() (nannyml.distribution.categorical.result.Result method)
(nannyml.distribution.continuous.result.Result method)
Chi Squared test
Chi2Statistic (class in nannyml.drift.univariate.methods)
Child model
Chunk (class in nannyml.chunk)
chunk_end_dates (nannyml.base.Abstract1DResult property)
(nannyml.base.Abstract2DResult property)
(nannyml.distribution.categorical.result.Result property)
chunk_end_indices (nannyml.base.Abstract1DResult property)
(nannyml.base.Abstract2DResult property)
(nannyml.distribution.categorical.result.Result property)
chunk_indices (nannyml.base.Abstract1DResult property)
(nannyml.base.Abstract2DResult property)
(nannyml.distribution.categorical.result.Result property)
chunk_keys (nannyml.base.Abstract1DResult property)
(nannyml.base.Abstract2DResult property)
(nannyml.distribution.categorical.result.Result property)
chunk_periods (nannyml.base.Abstract1DResult property)
(nannyml.base.Abstract2DResult property)
(nannyml.distribution.categorical.result.Result property)
chunk_start_dates (nannyml.base.Abstract1DResult property)
(nannyml.base.Abstract2DResult property)
(nannyml.distribution.categorical.result.Result property)
chunk_start_index (nannyml.base.Abstract1DResult property)
(nannyml.base.Abstract2DResult property)
chunk_start_indices (nannyml.base.Abstract1DResult property)
(nannyml.base.Abstract2DResult property)
(nannyml.distribution.categorical.result.Result property)
Chunker (class in nannyml.chunk)
ChunkerException
ChunkerFactory (class in nannyml.chunk)
CLI_RUN (nannyml.usage_logging.UsageEvent attribute)
Colors (class in nannyml.plots.colors)
column_name (nannyml.drift.multivariate.data_reconstruction.result.Metric attribute)
(nannyml.drift.multivariate.domain_classifier.result.Metric attribute)
(nannyml.performance_calculation.metrics.base.Metric property)
(nannyml.performance_estimation.confidence_based.metrics.Metric property)
column_names (nannyml.performance_calculation.metrics.base.Metric property)
(nannyml.performance_estimation.confidence_based.metrics.Metric property)
common_nan_removal() (in module nannyml.base)
compare() (nannyml.plots.blueprints.comparisons.ResultCompareMixin method)
Concept Drift
Confidence Band
Confidence Score
Config (class in nannyml.config)
Confusion Matrix
ConstantThreshold (class in nannyml.thresholds)
CONTINUOUS (nannyml.drift.univariate.methods.FeatureType attribute)
ContinuousDistributionCalculator (class in nannyml.distribution.continuous.calculator)
ContinuousHellingerDistance (class in nannyml.drift.univariate.methods)
ContinuousJensenShannonDistance (class in nannyml.drift.univariate.methods)
CorrelationRanker (class in nannyml.drift.ranker)
CountBasedChunker (class in nannyml.chunk)
Covariate Shift
create() (nannyml.calibration.CalibratorFactory class method)
(nannyml.drift.univariate.methods.MethodFactory class method)
(nannyml.io.base.WriterFactory class method)
(nannyml.performance_calculation.metrics.base.MetricFactory class method)
(nannyml.performance_estimation.confidence_based.metrics.MetricFactory class method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.MetricFactory class method)
credentials (nannyml.config.InputDataConfig attribute)
(nannyml.config.StoreConfig attribute)
cron (nannyml.config.SchedulingConfig attribute)
CronSchedulingConfig (class in nannyml.config)
crontab (nannyml.config.CronSchedulingConfig attribute)
current_calculator (nannyml.runner.RunContext attribute)
current_calculator_config (nannyml.runner.RunContext attribute)
current_calculator_success (nannyml.runner.RunContext attribute)
current_step (nannyml.runner.RunContext attribute)
D
d() (in module nannyml.data_quality.unseen)
Data Chunk
Data Drift
Data Period
DataReconstructionDriftCalculator (class in nannyml.drift.multivariate.data_reconstruction.calculator)
days (nannyml.config.IntervalSchedulingConfig attribute)
DC_CALC_FIT (nannyml.usage_logging.UsageEvent attribute)
DC_CALC_RUN (nannyml.usage_logging.UsageEvent attribute)
DC_RESULTS_PLOT (nannyml.usage_logging.UsageEvent attribute)
DEFAULT_CHUNK_COUNT (nannyml.chunk.DefaultChunker attribute)
DEFAULT_COLUMNS (nannyml.base.AbstractEstimatorResult attribute)
(nannyml.base.AbstractResult attribute)
DefaultChunker (class in nannyml.chunk)
deserialize() (nannyml.io.store.serializers.Serializer method)
DeserializeException
disable_usage_logging() (in module nannyml.usage_logging)
display_name (nannyml.drift.multivariate.data_reconstruction.result.Metric attribute)
(nannyml.drift.multivariate.domain_classifier.result.Metric attribute)
(nannyml.performance_calculation.metrics.base.Metric property)
(nannyml.performance_estimation.confidence_based.metrics.Metric property)
display_names (nannyml.performance_calculation.metrics.base.Metric property)
(nannyml.performance_estimation.confidence_based.metrics.Metric property)
DLE (class in nannyml.performance_estimation.direct_loss_estimation.dle)
DLE_ESTIMATOR_FIT (nannyml.usage_logging.UsageEvent attribute)
DLE_ESTIMATOR_RUN (nannyml.usage_logging.UsageEvent attribute)
DLE_PLOT (nannyml.usage_logging.UsageEvent attribute)
Domain Classifier
DomainClassifierCalculator (class in nannyml.drift.multivariate.domain_classifier.calculator)
DQ_CALC_MISSING_VALUES_FIT (nannyml.usage_logging.UsageEvent attribute)
DQ_CALC_MISSING_VALUES_PLOT (nannyml.usage_logging.UsageEvent attribute)
DQ_CALC_MISSING_VALUES_RUN (nannyml.usage_logging.UsageEvent attribute)
DQ_CALC_UNSEEN_VALUES_FIT (nannyml.usage_logging.UsageEvent attribute)
DQ_CALC_UNSEEN_VALUES_PLOT (nannyml.usage_logging.UsageEvent attribute)
DQ_CALC_UNSEEN_VALUES_RUN (nannyml.usage_logging.UsageEvent attribute)
DQ_CALC_VALUES_OUT_OF_RANGE_FIT (nannyml.usage_logging.UsageEvent attribute)
DQ_CALC_VALUES_OUT_OF_RANGE_PLOT (nannyml.usage_logging.UsageEvent attribute)
DQ_CALC_VALUES_OUT_OF_RANGE_RUN (nannyml.usage_logging.UsageEvent attribute)
E
empty (nannyml.base.AbstractEstimatorResult property)
(nannyml.base.AbstractResult property)
enable_usage_logging() (in module nannyml.usage_logging)
enabled (nannyml.config.CalculatorConfig attribute)
ensure_numpy() (in module nannyml.plots.util)
Error
estimate() (nannyml.base.AbstractEstimator method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.Metric method)
estimate_accuracy() (in module nannyml.performance_estimation.confidence_based.metrics)
estimate_ap() (in module nannyml.performance_estimation.confidence_based.metrics)
estimate_business_value() (in module nannyml.performance_estimation.confidence_based.metrics)
estimate_f1() (in module nannyml.performance_estimation.confidence_based.metrics)
estimate_precision() (in module nannyml.performance_estimation.confidence_based.metrics)
estimate_recall() (in module nannyml.performance_estimation.confidence_based.metrics)
estimate_roc_auc() (in module nannyml.performance_estimation.confidence_based.metrics)
estimate_specificity() (in module nannyml.performance_estimation.confidence_based.metrics)
Estimated Performance
EstimatorException
F
f1_sampling_error() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
f1_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
false_negative_sampling_error() (in module nannyml.sampling_error.binary_classification)
false_negative_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
false_positive_sampling_error() (in module nannyml.sampling_error.binary_classification)
false_positive_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
Feature
FeatureType (class in nannyml.drift.univariate.methods)
Figure (class in nannyml.plots.components.figure)
filename (nannyml.config.StoreConfig attribute)
FileReader (class in nannyml.io.file_reader)
FilesystemStore (class in nannyml.io.store.file_store)
FileWriter (class in nannyml.io.file_writer)
filter() (nannyml.base.AbstractEstimatorResult method)
(nannyml.base.AbstractResult method)
fit() (nannyml.base.AbstractCalculator method)
(nannyml.base.AbstractEstimator method)
(nannyml.calibration.Calibrator method)
(nannyml.calibration.IsotonicCalibrator method)
(nannyml.calibration.NoopCalibrator method)
(nannyml.drift.ranker.CorrelationRanker method)
(nannyml.drift.univariate.methods.Chi2Statistic method)
(nannyml.drift.univariate.methods.Method method)
(nannyml.performance_calculation.metrics.base.Metric method)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationConfusionMatrix method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationConfusionMatrix method)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix method)
(nannyml.performance_estimation.confidence_based.metrics.Metric method)
(nannyml.performance_estimation.confidence_based.metrics.MulticlassClassificationConfusionMatrix method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.Metric method)
G
get_chunk_record() (nannyml.performance_calculation.metrics.base.Metric method)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationConfusionMatrix method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationConfusionMatrix method)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix method)
(nannyml.performance_estimation.confidence_based.metrics.Metric method)
(nannyml.performance_estimation.confidence_based.metrics.MulticlassClassificationConfusionMatrix method)
get_chunker() (nannyml.chunk.ChunkerFactory class method)
get_config_path() (in module nannyml.config)
get_custom_data() (nannyml.plots.components.hover.Hover method)
get_false_neg_info() (nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationConfusionMatrix method)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix method)
get_false_negative_estimate() (nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix method)
get_false_pos_info() (nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationConfusionMatrix method)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix method)
get_false_positive_estimate() (nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix method)
get_filepath_str() (in module nannyml.io.base)
get_logger() (in module nannyml.usage_logging)
get_output_writers() (in module nannyml.runner)
get_store() (in module nannyml.runner)
get_template() (nannyml.plots.components.hover.Hover method)
get_title() (nannyml.plots.blueprints.comparisons.ResultCompareMixin method)
get_true_neg_info() (nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationConfusionMatrix method)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix method)
get_true_negative_estimate() (nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix method)
get_true_pos_info() (nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationConfusionMatrix method)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix method)
get_true_positive_estimate() (nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix method)
GRAY (nannyml.plots.colors.Colors attribute)
GRAY_DARK (nannyml.plots.colors.Colors attribute)
GREEN_SEA (nannyml.plots.colors.Colors attribute)
Ground truth
H
has_non_null_data() (in module nannyml.plots.util)
hours (nannyml.config.IntervalSchedulingConfig attribute)
Hover (class in nannyml.plots.components.hover)
I
Identifier
ignore_errors (nannyml.config.Config attribute)
Imputation
increase_step() (nannyml.runner.RunContext method)
INDIGO_PERSIAN (nannyml.plots.colors.Colors attribute)
input (nannyml.config.Config attribute)
InputConfig (class in nannyml.config)
InputDataConfig (class in nannyml.config)
interval (nannyml.config.SchedulingConfig attribute)
IntervalSchedulingConfig (class in nannyml.config)
InvalidArgumentsException
invalidate (nannyml.config.StoreConfig attribute)
InvalidReferenceDataException
IOException
is_time_based_x_axis() (in module nannyml.plots.util)
IsotonicCalibrator (class in nannyml.calibration)
J
JoblibPickleSerializer (class in nannyml.io.store.serializers)
join_column (nannyml.config.TargetDataConfig attribute)
joy() (in module nannyml.plots.components.joy_plot)
K
keys() (nannyml.base.AbstractResult method)
(nannyml.data_quality.missing.result.Result method)
(nannyml.data_quality.range.result.Result method)
(nannyml.data_quality.unseen.result.Result method)
(nannyml.distribution.categorical.result.Result method)
(nannyml.distribution.continuous.result.Result method)
(nannyml.drift.multivariate.data_reconstruction.result.Result method)
(nannyml.drift.multivariate.domain_classifier.result.Result method)
(nannyml.drift.univariate.result.Result method)
(nannyml.performance_calculation.result.Result method)
(nannyml.performance_estimation.confidence_based.results.Result method)
(nannyml.performance_estimation.direct_loss_estimation.result.Result method)
(nannyml.stats.avg.result.Result method)
(nannyml.stats.count.result.Result method)
(nannyml.stats.median.result.Result method)
(nannyml.stats.std.result.Result method)
(nannyml.stats.sum.result.Result method)
Kolmogorov-Smirnov test
KolmogorovSmirnovStatistic (class in nannyml.drift.univariate.methods)
L
Label
Latent space
LIGHT_GRAY (nannyml.plots.colors.Colors attribute)
LInfinityDistance (class in nannyml.drift.univariate.methods)
load() (nannyml.config.Config class method)
(nannyml.io.store.base.Store method)
load_csv_file_to_df() (in module nannyml.datasets.datasets)
load_modified_california_housing_dataset() (in module nannyml.datasets.datasets)
load_pq_file_to_df() (in module nannyml.datasets.datasets)
load_synthetic_binary_classification_dataset() (in module nannyml.datasets.datasets)
load_synthetic_car_loan_data_quality_dataset() (in module nannyml.datasets.datasets)
load_synthetic_car_loan_dataset() (in module nannyml.datasets.datasets)
load_synthetic_car_price_dataset() (in module nannyml.datasets.datasets)
load_synthetic_multiclass_classification_dataset() (in module nannyml.datasets.datasets)
load_titanic_dataset() (in module nannyml.datasets.datasets)
load_us_census_ma_employment_data() (in module nannyml.datasets.datasets)
log() (nannyml.runner.RunnerLogger method)
(nannyml.usage_logging.UsageLogger method)
log_usage() (in module nannyml.usage_logging)
Loss
Loss Function
lower (nannyml.thresholds.ConstantThreshold attribute)
lower_confidence_bounds() (nannyml.base.AbstractResult method)
lower_thresholds() (nannyml.base.AbstractResult method)
M
MAE (class in nannyml.performance_calculation.metrics.regression)
(class in nannyml.performance_estimation.direct_loss_estimation.metrics)
mae_sampling_error() (in module nannyml.sampling_error.regression)
mae_sampling_error_components() (in module nannyml.sampling_error.regression)
MAPE (class in nannyml.performance_calculation.metrics.regression)
(class in nannyml.performance_estimation.direct_loss_estimation.metrics)
mape_sampling_error() (in module nannyml.sampling_error.regression)
mape_sampling_error_components() (in module nannyml.sampling_error.regression)
merge() (nannyml.chunk.Chunk method)
Method (class in nannyml.drift.univariate.methods)
MethodFactory (class in nannyml.drift.univariate.methods)
methods (nannyml.drift.univariate.result.Result property)
Metric (class in nannyml.drift.multivariate.data_reconstruction.result)
(class in nannyml.drift.multivariate.domain_classifier.result)
(class in nannyml.performance_calculation.metrics.base)
(class in nannyml.performance_estimation.confidence_based.metrics)
(class in nannyml.performance_estimation.direct_loss_estimation.metrics)
metric() (in module nannyml.plots.components.step_plot)
MetricFactory (class in nannyml.performance_calculation.metrics.base)
(class in nannyml.performance_estimation.confidence_based.metrics)
(class in nannyml.performance_estimation.direct_loss_estimation.metrics)
metrics (nannyml.performance_calculation.result.Result attribute)
minutes (nannyml.config.IntervalSchedulingConfig attribute)
MissingMetadataException
MissingValuesCalculator (class in nannyml.data_quality.missing.calculator)
Model inputs
Model outputs
Model predictions
model_computed_fields (nannyml.config.CalculatorConfig attribute)
(nannyml.config.Config attribute)
(nannyml.config.CronSchedulingConfig attribute)
(nannyml.config.InputConfig attribute)
(nannyml.config.InputDataConfig attribute)
(nannyml.config.IntervalSchedulingConfig attribute)
(nannyml.config.SchedulingConfig attribute)
(nannyml.config.StoreConfig attribute)
(nannyml.config.TargetDataConfig attribute)
(nannyml.config.WriterConfig attribute)
model_config (nannyml.config.CalculatorConfig attribute)
(nannyml.config.Config attribute)
(nannyml.config.CronSchedulingConfig attribute)
(nannyml.config.InputConfig attribute)
(nannyml.config.InputDataConfig attribute)
(nannyml.config.IntervalSchedulingConfig attribute)
(nannyml.config.SchedulingConfig attribute)
(nannyml.config.StoreConfig attribute)
(nannyml.config.TargetDataConfig attribute)
(nannyml.config.WriterConfig attribute)
model_fields (nannyml.config.CalculatorConfig attribute)
(nannyml.config.Config attribute)
(nannyml.config.CronSchedulingConfig attribute)
(nannyml.config.InputConfig attribute)
(nannyml.config.InputDataConfig attribute)
(nannyml.config.IntervalSchedulingConfig attribute)
(nannyml.config.SchedulingConfig attribute)
(nannyml.config.StoreConfig attribute)
(nannyml.config.TargetDataConfig attribute)
(nannyml.config.WriterConfig attribute)
module
nannyml
nannyml.analytics
nannyml.base
nannyml.calibration
nannyml.chunk
nannyml.cli
nannyml.cli.cli
nannyml.cli.run
nannyml.config
nannyml.data_quality
nannyml.data_quality.missing
nannyml.data_quality.missing.calculator
nannyml.data_quality.missing.result
nannyml.data_quality.range
nannyml.data_quality.range.calculator
nannyml.data_quality.range.result
nannyml.data_quality.unseen
nannyml.data_quality.unseen.calculator
nannyml.data_quality.unseen.result
nannyml.datasets
nannyml.datasets.data
nannyml.datasets.datasets
nannyml.distribution
nannyml.distribution.categorical
nannyml.distribution.categorical.calculator
nannyml.distribution.categorical.result
nannyml.distribution.continuous
nannyml.distribution.continuous.calculator
nannyml.distribution.continuous.result
nannyml.drift
nannyml.drift.multivariate
nannyml.drift.multivariate.data_reconstruction
nannyml.drift.multivariate.data_reconstruction.calculator
nannyml.drift.multivariate.data_reconstruction.result
nannyml.drift.multivariate.domain_classifier
nannyml.drift.multivariate.domain_classifier.calculator
nannyml.drift.multivariate.domain_classifier.result
nannyml.drift.ranker
nannyml.drift.univariate
nannyml.drift.univariate.calculator
nannyml.drift.univariate.methods
nannyml.drift.univariate.result
nannyml.exceptions
nannyml.io
nannyml.io.base
nannyml.io.file_reader
nannyml.io.file_writer
nannyml.io.pickle_file_writer
nannyml.io.raw_files_writer
nannyml.io.store
nannyml.io.store.base
nannyml.io.store.file_store
nannyml.io.store.serializers
nannyml.performance_calculation
nannyml.performance_calculation.calculator
nannyml.performance_calculation.metrics
nannyml.performance_calculation.metrics.base
nannyml.performance_calculation.metrics.binary_classification
nannyml.performance_calculation.metrics.multiclass_classification
nannyml.performance_calculation.metrics.regression
nannyml.performance_calculation.result
nannyml.performance_estimation
nannyml.performance_estimation.confidence_based
nannyml.performance_estimation.confidence_based.cbpe
nannyml.performance_estimation.confidence_based.metrics
nannyml.performance_estimation.confidence_based.results
nannyml.performance_estimation.direct_loss_estimation
nannyml.performance_estimation.direct_loss_estimation.dle
nannyml.performance_estimation.direct_loss_estimation.metrics
nannyml.performance_estimation.direct_loss_estimation.result
nannyml.plots
nannyml.plots.blueprints
nannyml.plots.blueprints.comparisons
nannyml.plots.blueprints.distributions
nannyml.plots.blueprints.metrics
nannyml.plots.colors
nannyml.plots.components
nannyml.plots.components.figure
nannyml.plots.components.hover
nannyml.plots.components.joy_plot
nannyml.plots.components.stacked_bar_plot
nannyml.plots.components.step_plot
nannyml.plots.util
nannyml.runner
nannyml.sampling_error
nannyml.sampling_error.binary_classification
nannyml.sampling_error.multiclass_classification
nannyml.sampling_error.regression
nannyml.sampling_error.summary_stats
nannyml.stats
nannyml.stats.avg
nannyml.stats.avg.calculator
nannyml.stats.avg.result
nannyml.stats.count
nannyml.stats.count.calculator
nannyml.stats.count.result
nannyml.stats.median
nannyml.stats.median.calculator
nannyml.stats.median.result
nannyml.stats.std
nannyml.stats.std.calculator
nannyml.stats.std.result
nannyml.stats.sum
nannyml.stats.sum.calculator
nannyml.stats.sum.result
nannyml.thresholds
nannyml.usage_logging
MSE (class in nannyml.performance_calculation.metrics.regression)
(class in nannyml.performance_estimation.direct_loss_estimation.metrics)
mse_sampling_error() (in module nannyml.sampling_error.regression)
mse_sampling_error_components() (in module nannyml.sampling_error.regression)
MSLE (class in nannyml.performance_calculation.metrics.regression)
(class in nannyml.performance_estimation.direct_loss_estimation.metrics)
msle_sampling_error() (in module nannyml.sampling_error.regression)
msle_sampling_error_components() (in module nannyml.sampling_error.regression)
multiclass_confusion_matrix_sampling_error() (in module nannyml.sampling_error.multiclass_classification)
multiclass_confusion_matrix_sampling_error_components() (in module nannyml.sampling_error.multiclass_classification)
MulticlassClassificationAccuracy (class in nannyml.performance_calculation.metrics.multiclass_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
MulticlassClassificationAP (class in nannyml.performance_calculation.metrics.multiclass_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
MulticlassClassificationAUROC (class in nannyml.performance_calculation.metrics.multiclass_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
MulticlassClassificationBusinessValue (class in nannyml.performance_calculation.metrics.multiclass_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
MulticlassClassificationConfusionMatrix (class in nannyml.performance_calculation.metrics.multiclass_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
MulticlassClassificationF1 (class in nannyml.performance_calculation.metrics.multiclass_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
MulticlassClassificationPrecision (class in nannyml.performance_calculation.metrics.multiclass_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
MulticlassClassificationRecall (class in nannyml.performance_calculation.metrics.multiclass_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
MulticlassClassificationSpecificity (class in nannyml.performance_calculation.metrics.multiclass_classification)
(class in nannyml.performance_estimation.confidence_based.metrics)
MULTIVAR_DRIFT_CALC_FIT (nannyml.usage_logging.UsageEvent attribute)
MULTIVAR_DRIFT_CALC_RUN (nannyml.usage_logging.UsageEvent attribute)
MULTIVAR_DRIFT_PLOT (nannyml.usage_logging.UsageEvent attribute)
MULTIVAR_RECONST_DRIFT_CALC_FIT (nannyml.analytics.UsageEvent attribute)
MULTIVAR_RECONST_DRIFT_CALC_RUN (nannyml.analytics.UsageEvent attribute)
Multivariate Drift Detection
N
name (nannyml.config.CalculatorConfig attribute)
Nanny model
nannyml
module
nannyml.analytics
module
nannyml.base
module
nannyml.calibration
module
nannyml.chunk
module
nannyml.cli
module
nannyml.cli.cli
module
nannyml.cli.run
module
nannyml.config
module
nannyml.data_quality
module
nannyml.data_quality.missing
module
nannyml.data_quality.missing.calculator
module
nannyml.data_quality.missing.result
module
nannyml.data_quality.range
module
nannyml.data_quality.range.calculator
module
nannyml.data_quality.range.result
module
nannyml.data_quality.unseen
module
nannyml.data_quality.unseen.calculator
module
nannyml.data_quality.unseen.result
module
nannyml.datasets
module
nannyml.datasets.data
module
nannyml.datasets.datasets
module
nannyml.distribution
module
nannyml.distribution.categorical
module
nannyml.distribution.categorical.calculator
module
nannyml.distribution.categorical.result
module
nannyml.distribution.continuous
module
nannyml.distribution.continuous.calculator
module
nannyml.distribution.continuous.result
module
nannyml.drift
module
nannyml.drift.multivariate
module
nannyml.drift.multivariate.data_reconstruction
module
nannyml.drift.multivariate.data_reconstruction.calculator
module
nannyml.drift.multivariate.data_reconstruction.result
module
nannyml.drift.multivariate.domain_classifier
module
nannyml.drift.multivariate.domain_classifier.calculator
module
nannyml.drift.multivariate.domain_classifier.result
module
nannyml.drift.ranker
module
nannyml.drift.univariate
module
nannyml.drift.univariate.calculator
module
nannyml.drift.univariate.methods
module
nannyml.drift.univariate.result
module
nannyml.exceptions
module
nannyml.io
module
nannyml.io.base
module
nannyml.io.file_reader
module
nannyml.io.file_writer
module
nannyml.io.pickle_file_writer
module
nannyml.io.raw_files_writer
module
nannyml.io.store
module
nannyml.io.store.base
module
nannyml.io.store.file_store
module
nannyml.io.store.serializers
module
nannyml.performance_calculation
module
nannyml.performance_calculation.calculator
module
nannyml.performance_calculation.metrics
module
nannyml.performance_calculation.metrics.base
module
nannyml.performance_calculation.metrics.binary_classification
module
nannyml.performance_calculation.metrics.multiclass_classification
module
nannyml.performance_calculation.metrics.regression
module
nannyml.performance_calculation.result
module
nannyml.performance_estimation
module
nannyml.performance_estimation.confidence_based
module
nannyml.performance_estimation.confidence_based.cbpe
module
nannyml.performance_estimation.confidence_based.metrics
module
nannyml.performance_estimation.confidence_based.results
module
nannyml.performance_estimation.direct_loss_estimation
module
nannyml.performance_estimation.direct_loss_estimation.dle
module
nannyml.performance_estimation.direct_loss_estimation.metrics
module
nannyml.performance_estimation.direct_loss_estimation.result
module
nannyml.plots
module
nannyml.plots.blueprints
module
nannyml.plots.blueprints.comparisons
module
nannyml.plots.blueprints.distributions
module
nannyml.plots.blueprints.metrics
module
nannyml.plots.colors
module
nannyml.plots.components
module
nannyml.plots.components.figure
module
nannyml.plots.components.hover
module
nannyml.plots.components.joy_plot
module
nannyml.plots.components.stacked_bar_plot
module
nannyml.plots.components.step_plot
module
nannyml.plots.util
module
nannyml.runner
module
nannyml.sampling_error
module
nannyml.sampling_error.binary_classification
module
nannyml.sampling_error.multiclass_classification
module
nannyml.sampling_error.regression
module
nannyml.sampling_error.summary_stats
module
nannyml.stats
module
nannyml.stats.avg
module
nannyml.stats.avg.calculator
module
nannyml.stats.avg.result
module
nannyml.stats.count
module
nannyml.stats.count.calculator
module
nannyml.stats.count.result
module
nannyml.stats.median
module
nannyml.stats.median.calculator
module
nannyml.stats.median.result
module
nannyml.stats.std
module
nannyml.stats.std.calculator
module
nannyml.stats.std.result
module
nannyml.stats.sum
module
nannyml.stats.sum.calculator
module
nannyml.stats.sum.result
module
nannyml.thresholds
module
nannyml.usage_logging
module
NannyMLException
needs_calibration() (in module nannyml.calibration)
NoopCalibrator (class in nannyml.calibration)
NotFittedException
NumericalRangeCalculator (class in nannyml.data_quality.range.calculator)
O
OUTPUT_DRIFT_CALC_FIT (nannyml.analytics.UsageEvent attribute)
OUTPUT_DRIFT_CALC_RUN (nannyml.analytics.UsageEvent attribute)
outputs (nannyml.config.CalculatorConfig attribute)
P
pairwise() (in module nannyml.plots.util)
params (nannyml.config.CalculatorConfig attribute)
(nannyml.config.WriterConfig attribute)
parse() (nannyml.config.Config class method)
parse_object() (nannyml.thresholds.Threshold class method)
Partition Column
path (nannyml.config.InputDataConfig attribute)
(nannyml.config.StoreConfig attribute)
PCA
PerColumnResult (class in nannyml.base)
Performance Estimation
PERFORMANCE_CALC_FIT (nannyml.analytics.UsageEvent attribute)
(nannyml.usage_logging.UsageEvent attribute)
PERFORMANCE_CALC_RUN (nannyml.analytics.UsageEvent attribute)
(nannyml.usage_logging.UsageEvent attribute)
PERFORMANCE_PLOT (nannyml.usage_logging.UsageEvent attribute)
PerformanceCalculator (class in nannyml.performance_calculation.calculator)
PeriodBasedChunker (class in nannyml.chunk)
PerMetricPerColumnResult (class in nannyml.base)
PerMetricResult (class in nannyml.base)
PickleFileWriter (class in nannyml.io.pickle_file_writer)
PickleSerializer (class in nannyml.io.store.serializers)
plot() (nannyml.base.AbstractEstimatorResult method)
(nannyml.base.AbstractResult method)
(nannyml.data_quality.missing.result.Result method)
(nannyml.data_quality.range.result.Result method)
(nannyml.data_quality.unseen.result.Result method)
(nannyml.distribution.categorical.result.Result method)
(nannyml.distribution.continuous.result.Result method)
(nannyml.drift.multivariate.data_reconstruction.result.Result method)
(nannyml.drift.multivariate.domain_classifier.result.Result method)
(nannyml.drift.univariate.result.Result method)
(nannyml.performance_calculation.result.Result method)
(nannyml.performance_estimation.confidence_based.results.Result method)
(nannyml.performance_estimation.direct_loss_estimation.result.Result method)
(nannyml.plots.blueprints.comparisons.ResultComparison method)
(nannyml.stats.avg.result.Result method)
(nannyml.stats.count.result.Result method)
(nannyml.stats.median.result.Result method)
(nannyml.stats.std.result.Result method)
(nannyml.stats.sum.result.Result method)
plot_2d_compare_step_to_step() (in module nannyml.plots.blueprints.comparisons)
plot_distributions() (in module nannyml.plots.blueprints.distributions)
plot_metric() (in module nannyml.plots.blueprints.metrics)
plot_metrics() (in module nannyml.plots.blueprints.metrics)
precision_sampling_error() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
precision_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
Predicted labels
Predicted probabilities
Predicted scores
Predictions
preprocess_categorical_features() (in module nannyml.drift.multivariate.domain_classifier.calculator)
R
raise_if_metrics_require_y_pred() (in module nannyml.performance_calculation.calculator)
(in module nannyml.performance_estimation.confidence_based.cbpe)
rank() (nannyml.drift.ranker.AlertCountRanker method)
(nannyml.drift.ranker.CorrelationRanker method)
RANKER_ALERT_COUNT_RUN (nannyml.usage_logging.UsageEvent attribute)
RANKER_CORRELATION_FIT (nannyml.usage_logging.UsageEvent attribute)
RANKER_CORRELATION_RUN (nannyml.usage_logging.UsageEvent attribute)
Ranking
RawFilesWriter (class in nannyml.io.raw_files_writer)
read() (nannyml.io.base.Reader method)
read_args (nannyml.config.InputDataConfig attribute)
read_data() (in module nannyml.runner)
Reader (class in nannyml.io.base)
ReaderException
Realized Performance
realized_performance() (nannyml.performance_estimation.direct_loss_estimation.metrics.MAE method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.MAPE method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.Metric method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.MSE method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.MSLE method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.RMSE method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.RMSLE method)
recall_sampling_error() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
recall_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
Reconstruction Error
RED_IMPERIAL (nannyml.plots.colors.Colors attribute)
reference_data (nannyml.config.InputConfig attribute)
(nannyml.runner.RunInput attribute)
register() (nannyml.calibration.CalibratorFactory class method)
(nannyml.drift.univariate.methods.MethodFactory class method)
(nannyml.io.base.WriterFactory class method)
(nannyml.performance_calculation.metrics.base.MetricFactory class method)
(nannyml.performance_estimation.confidence_based.metrics.MetricFactory class method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.MetricFactory class method)
(nannyml.runner.CalculatorFactory class method)
register_calibrator() (nannyml.calibration.CalibratorFactory class method)
registry (nannyml.drift.univariate.methods.MethodFactory attribute)
(nannyml.io.base.WriterFactory attribute)
(nannyml.performance_calculation.metrics.base.MetricFactory attribute)
(nannyml.performance_estimation.confidence_based.metrics.MetricFactory attribute)
(nannyml.performance_estimation.direct_loss_estimation.metrics.MetricFactory attribute)
(nannyml.runner.CalculatorFactory attribute)
render_alert_string() (in module nannyml.plots.components.hover)
render_display_name() (in module nannyml.plots.blueprints.comparisons)
render_metric_display_name() (in module nannyml.plots.blueprints.comparisons)
render_partial_target_string() (in module nannyml.plots.components.hover)
render_period_string() (in module nannyml.plots.components.hover)
render_x_coordinate() (in module nannyml.plots.components.hover)
Residual
Result (class in nannyml.data_quality.missing.result)
(class in nannyml.data_quality.range.result)
(class in nannyml.data_quality.unseen.result)
(class in nannyml.distribution.categorical.result)
(class in nannyml.distribution.continuous.result)
(class in nannyml.drift.multivariate.data_reconstruction.result)
(class in nannyml.drift.multivariate.domain_classifier.result)
(class in nannyml.drift.univariate.result)
(class in nannyml.performance_calculation.result)
(class in nannyml.performance_estimation.confidence_based.results)
(class in nannyml.performance_estimation.direct_loss_estimation.result)
(class in nannyml.stats.avg.result)
(class in nannyml.stats.count.result)
(class in nannyml.stats.median.result)
(class in nannyml.stats.std.result)
(class in nannyml.stats.sum.result)
result (nannyml.runner.RunContext attribute)
ResultCompareMixin (class in nannyml.plots.blueprints.comparisons)
ResultComparison (class in nannyml.plots.blueprints.comparisons)
RMSE (class in nannyml.performance_calculation.metrics.regression)
(class in nannyml.performance_estimation.direct_loss_estimation.metrics)
rmse_sampling_error() (in module nannyml.sampling_error.regression)
rmse_sampling_error_components() (in module nannyml.sampling_error.regression)
RMSLE (class in nannyml.performance_calculation.metrics.regression)
(class in nannyml.performance_estimation.direct_loss_estimation.metrics)
rmsle_sampling_error() (in module nannyml.sampling_error.regression)
rmsle_sampling_error_components() (in module nannyml.sampling_error.regression)
run() (in module nannyml.runner)
run_context() (in module nannyml.runner)
run_success (nannyml.runner.RunContext attribute)
RunContext (class in nannyml.runner)
RunInput (class in nannyml.runner)
RunnerLogger (class in nannyml.runner)
S
SAFFRON (nannyml.plots.colors.Colors attribute)
Sampling Error
sampling_error() (nannyml.base.AbstractResult method)
(nannyml.performance_calculation.metrics.base.Metric method)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationConfusionMatrix method)
(nannyml.performance_estimation.direct_loss_estimation.metrics.Metric method)
scheduling (nannyml.config.Config attribute)
SchedulingConfig (class in nannyml.config)
SEGMENT_WRITE_KEY (nannyml.analytics.SegmentUsageTracker attribute)
(nannyml.usage_logging.SegmentUsageTracker attribute)
SegmentUsageTracker (class in nannyml.analytics)
(class in nannyml.usage_logging)
serialize() (nannyml.io.store.serializers.Serializer method)
SerializeException
Serializer (class in nannyml.io.store.serializers)
SizeBasedChunker (class in nannyml.chunk)
specificity_sampling_error() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
specificity_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
(in module nannyml.sampling_error.multiclass_classification)
split() (nannyml.chunk.Chunker method)
stacked_bar() (in module nannyml.plots.components.stacked_bar_plot)
Standard Error
StandardDeviationThreshold (class in nannyml.thresholds)
STATS_AVG_FIT (nannyml.usage_logging.UsageEvent attribute)
STATS_AVG_PLOT (nannyml.usage_logging.UsageEvent attribute)
STATS_AVG_RUN (nannyml.usage_logging.UsageEvent attribute)
STATS_COUNT_FIT (nannyml.usage_logging.UsageEvent attribute)
STATS_COUNT_PLOT (nannyml.usage_logging.UsageEvent attribute)
STATS_COUNT_RUN (nannyml.usage_logging.UsageEvent attribute)
STATS_MEDIAN_FIT (nannyml.usage_logging.UsageEvent attribute)
STATS_MEDIAN_PLOT (nannyml.usage_logging.UsageEvent attribute)
STATS_MEDIAN_RUN (nannyml.usage_logging.UsageEvent attribute)
STATS_STD_FIT (nannyml.usage_logging.UsageEvent attribute)
STATS_STD_PLOT (nannyml.usage_logging.UsageEvent attribute)
STATS_STD_RUN (nannyml.usage_logging.UsageEvent attribute)
STATS_SUM_FIT (nannyml.usage_logging.UsageEvent attribute)
STATS_SUM_PLOT (nannyml.usage_logging.UsageEvent attribute)
STATS_SUM_RUN (nannyml.usage_logging.UsageEvent attribute)
std_lower_multiplier (nannyml.thresholds.StandardDeviationThreshold attribute)
std_upper_multiplier (nannyml.thresholds.StandardDeviationThreshold attribute)
STEPS_PER_CALCULATOR (nannyml.runner.RunContext attribute)
Store (class in nannyml.io.store.base)
store (nannyml.config.CalculatorConfig attribute)
store() (nannyml.io.store.base.Store method)
StoreConfig (class in nannyml.config)
StoreException
summary_stats_median_sampling_error() (in module nannyml.sampling_error.summary_stats)
summary_stats_median_sampling_error_components() (in module nannyml.sampling_error.summary_stats)
summary_stats_std_sampling_error() (in module nannyml.sampling_error.summary_stats)
summary_stats_std_sampling_error_components() (in module nannyml.sampling_error.summary_stats)
SummaryStatsAvgCalculator (class in nannyml.stats.avg.calculator)
SummaryStatsMedianCalculator (class in nannyml.stats.median.calculator)
SummaryStatsRowCountCalculator (class in nannyml.stats.count.calculator)
SummaryStatsStdCalculator (class in nannyml.stats.std.calculator)
SummaryStatsSumCalculator (class in nannyml.stats.sum.calculator)
SUPPORTED_METRIC_STYLES (nannyml.plots.components.figure.Figure attribute)
T
Target
target_data (nannyml.config.InputConfig attribute)
(nannyml.runner.RunInput attribute)
TARGET_DISTRIBUTION_DRIFT_CALC_FIT (nannyml.analytics.UsageEvent attribute)
TARGET_DISTRIBUTION_DRIFT_CALC_RUN (nannyml.analytics.UsageEvent attribute)
target_join_column (nannyml.runner.RunInput attribute)
TargetDataConfig (class in nannyml.config)
TEST_UUID (nannyml.analytics.SegmentUsageTracker attribute)
Threshold
(class in nannyml.thresholds)
ThresholdException
thresholds() (nannyml.thresholds.ConstantThreshold method)
(nannyml.thresholds.StandardDeviationThreshold method)
(nannyml.thresholds.Threshold method)
Timestamp
titles (nannyml.plots.blueprints.comparisons.ResultCompareMixin property)
to_df() (nannyml.base.AbstractEstimatorResult method)
(nannyml.base.AbstractResult method)
(nannyml.distribution.categorical.result.Result method)
total_steps (nannyml.runner.RunContext attribute)
track() (in module nannyml.analytics)
(nannyml.analytics.UsageTracker method)
transparent() (nannyml.plots.colors.Colors method)
true_negative_sampling_error() (in module nannyml.sampling_error.binary_classification)
true_negative_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
true_positive_sampling_error() (in module nannyml.sampling_error.binary_classification)
true_positive_sampling_error_components() (in module nannyml.sampling_error.binary_classification)
tune_hyperparams() (nannyml.drift.multivariate.domain_classifier.calculator.DomainClassifierCalculator method)
type (nannyml.config.CalculatorConfig attribute)
(nannyml.config.WriterConfig attribute)
U
UNIVAR_DRIFT_CALC_FIT (nannyml.usage_logging.UsageEvent attribute)
UNIVAR_DRIFT_CALC_RUN (nannyml.usage_logging.UsageEvent attribute)
UNIVAR_DRIFT_PLOT (nannyml.usage_logging.UsageEvent attribute)
UNIVAR_STAT_DRIFT_CALC_FIT (nannyml.analytics.UsageEvent attribute)
UNIVAR_STAT_DRIFT_CALC_RUN (nannyml.analytics.UsageEvent attribute)
Univariate Drift Detection
UnivariateDriftCalculator (class in nannyml.drift.univariate.calculator)
Unseen Values
UnseenValuesCalculator (class in nannyml.data_quality.unseen.calculator)
upper (nannyml.thresholds.ConstantThreshold attribute)
upper_confidence_bounds() (nannyml.base.AbstractResult method)
upper_thresholds() (nannyml.base.AbstractResult method)
UsageEvent (class in nannyml.analytics)
(class in nannyml.usage_logging)
UsageLogger (class in nannyml.usage_logging)
UsageTracker (class in nannyml.analytics)
user_metadata (nannyml.analytics.UsageTracker attribute)
V
value_counts() (nannyml.distribution.categorical.result.Result method)
values() (nannyml.base.AbstractResult method)
W
WassersteinDistance (class in nannyml.drift.univariate.methods)
weeks (nannyml.config.IntervalSchedulingConfig attribute)
write() (nannyml.io.base.Writer method)
write_args (nannyml.config.WriterConfig attribute)
WRITE_DB (nannyml.usage_logging.UsageEvent attribute)
write_key (nannyml.analytics.SegmentUsageTracker attribute)
(nannyml.usage_logging.SegmentUsageTracker attribute)
WRITE_PICKLE (nannyml.usage_logging.UsageEvent attribute)
WRITE_RAW (nannyml.usage_logging.UsageEvent attribute)
Writer (class in nannyml.io.base)
WriterConfig (class in nannyml.config)
WriterException
WriterFactory (class in nannyml.io.base)
Y
y_pred (nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationAccuracy attribute)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationBusinessValue attribute)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationConfusionMatrix attribute)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationF1 attribute)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationPrecision attribute)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationRecall attribute)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationSpecificity attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationAccuracy attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationBusinessValue attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationConfusionMatrix attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationF1 attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationPrecision attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationRecall attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationSpecificity attribute)
(nannyml.performance_calculation.metrics.regression.MAE attribute)
(nannyml.performance_calculation.metrics.regression.MAPE attribute)
(nannyml.performance_calculation.metrics.regression.MSE attribute)
(nannyml.performance_calculation.metrics.regression.MSLE attribute)
(nannyml.performance_calculation.metrics.regression.RMSE attribute)
(nannyml.performance_calculation.metrics.regression.RMSLE attribute)
y_pred_proba (nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationAP attribute)
(nannyml.performance_calculation.metrics.binary_classification.BinaryClassificationAUROC attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationAccuracy attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationAP attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationAUROC attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationBusinessValue attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationConfusionMatrix attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationF1 attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationPrecision attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationRecall attribute)
(nannyml.performance_calculation.metrics.multiclass_classification.MulticlassClassificationSpecificity attribute)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationAccuracy attribute)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationAP attribute)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationAUROC attribute)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationBusinessValue attribute)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationConfusionMatrix attribute)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationF1 attribute)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationPrecision attribute)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationRecall attribute)
(nannyml.performance_estimation.confidence_based.metrics.BinaryClassificationSpecificity attribute)
(nannyml.performance_estimation.confidence_based.metrics.MulticlassClassificationBusinessValue attribute)
(nannyml.performance_estimation.confidence_based.metrics.MulticlassClassificationConfusionMatrix attribute)
Read the Docs
v: stable
Versions
latest
stable
v0.12.1
v0.12.0
v0.11.0
v0.10.7
v0.10.6
v0.10.5
v0.10.4
v0.10.3
v0.10.2
v0.10.1
v0.10.0
v0.9.1
v0.9.0
v0.8.6
v0.8.5
v0.8.4
v0.8.3
v0.8.2
v0.8.1
v0.8.0
v0.7.0
v0.6.3
v0.6.2
v0.6.1
v0.6.0
v0.5.3
v0.5.2
v0.5.1
v0.5.0
v0.4.1
v0.4.0
v0.3.2
v0.3.1
v0.3.0
v0.2.1
v0.2.0
main
Downloads
On Read the Docs
Project Home
Builds