nannyml.performance_estimation.direct_loss_estimation.metrics module
- class nannyml.performance_estimation.direct_loss_estimation.metrics.MAE(estimator)[source]
Bases:
Metric
Creates a new Metric instance.
- Parameters:
display_name (str) – The name of the metric. Used to display in plots. If not given this name will be derived from the
calculation_function
.column_name (str) – The name used to indicate the metric in columns of a DataFrame.
- class nannyml.performance_estimation.direct_loss_estimation.metrics.MAPE(estimator)[source]
Bases:
Metric
Creates a new Metric instance.
- Parameters:
display_name (str) – The name of the metric. Used to display in plots. If not given this name will be derived from the
calculation_function
.column_name (str) – The name used to indicate the metric in columns of a DataFrame.
- class nannyml.performance_estimation.direct_loss_estimation.metrics.MSE(estimator)[source]
Bases:
Metric
Creates a new Metric instance.
- Parameters:
display_name (str) – The name of the metric. Used to display in plots. If not given this name will be derived from the
calculation_function
.column_name (str) – The name used to indicate the metric in columns of a DataFrame.
- class nannyml.performance_estimation.direct_loss_estimation.metrics.MSLE(estimator)[source]
Bases:
Metric
Creates a new Metric instance.
- Parameters:
display_name (str) – The name of the metric. Used to display in plots. If not given this name will be derived from the
calculation_function
.column_name (str) – The name used to indicate the metric in columns of a DataFrame.
- class nannyml.performance_estimation.direct_loss_estimation.metrics.Metric(display_name: str, column_name: str, estimator: AbstractEstimator, upper_threshold_limit: Optional[float] = None, lower_threshold_limit: Optional[float] = None)[source]
Bases:
ABC
A performance metric used to estimate regression performance.
Creates a new Metric instance.
- Parameters:
display_name (str) – The name of the metric. Used to display in plots. If not given this name will be derived from the
calculation_function
.column_name (str) – The name used to indicate the metric in columns of a DataFrame.
- estimate(data: DataFrame)[source]
Calculates performance metrics on data.
- Parameters:
data (pd.DataFrame) – The data to estimate performance metrics for. Requires presence of either the predicted labels or prediction scores/probabilities (depending on the metric to be calculated).
- fit(reference_data: DataFrame)[source]
Fits a Metric on reference data.
- Parameters:
reference_data (pd.DataFrame) – The reference data used for fitting. Must have target data available.
- sampling_error(data: DataFrame)[source]
Calculates the sampling error with respect to the reference data for a given chunk of data.
- Parameters:
data (pd.DataFrame) – The data to calculate the sampling error on, with respect to the reference data.
- Returns:
sampling_error – The expected sampling error.
- Return type:
float
- class nannyml.performance_estimation.direct_loss_estimation.metrics.MetricFactory[source]
Bases:
object
A factory class that produces Metric instances based on a given magic string or a metric specification.
- classmethod create(key: str, problem_type: ProblemType, kwargs: Optional[Dict[str, Any]] = None) Metric [source]
- registry: Dict[str, Dict[ProblemType, Metric]] = {'mae': {ProblemType.REGRESSION: <class 'nannyml.performance_estimation.direct_loss_estimation.metrics.MAE'>}, 'mape': {ProblemType.REGRESSION: <class 'nannyml.performance_estimation.direct_loss_estimation.metrics.MAPE'>}, 'mse': {ProblemType.REGRESSION: <class 'nannyml.performance_estimation.direct_loss_estimation.metrics.MSE'>}, 'msle': {ProblemType.REGRESSION: <class 'nannyml.performance_estimation.direct_loss_estimation.metrics.MSLE'>}, 'rmse': {ProblemType.REGRESSION: <class 'nannyml.performance_estimation.direct_loss_estimation.metrics.RMSE'>}, 'rmsle': {ProblemType.REGRESSION: <class 'nannyml.performance_estimation.direct_loss_estimation.metrics.RMSLE'>}}
- class nannyml.performance_estimation.direct_loss_estimation.metrics.RMSE(estimator)[source]
Bases:
Metric
Creates a new Metric instance.
- Parameters:
display_name (str) – The name of the metric. Used to display in plots. If not given this name will be derived from the
calculation_function
.column_name (str) – The name used to indicate the metric in columns of a DataFrame.
- class nannyml.performance_estimation.direct_loss_estimation.metrics.RMSLE(estimator)[source]
Bases:
Metric
Creates a new Metric instance.
- Parameters:
display_name (str) – The name of the metric. Used to display in plots. If not given this name will be derived from the
calculation_function
.column_name (str) – The name used to indicate the metric in columns of a DataFrame.