Prediction sets
This module provides prediction sets for conformal prediction. To be used when building a calibrator.
- prediction_sets.constant_bbox(Y_pred, scores_quantile)
Generate the upper and lower bounds of the bounding box coordinates for a given prediction.
- Parameters:
Y_pred (np.ndarray) – the predicted bounding box coordinates.
scores_quantile (np.ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\).
- Returns:
the lower bound and upper bound coordinates of the bounding box.
- Return type:
Tuple[np.ndarray, np.ndarray]
- Raises:
TypeError – unsupported data types.
- prediction_sets.constant_interval(y_pred, scores_quantile)
Constant prediction interval centered on y_pred. The size of the margin is scores_quantile (noted \(\gamma_{\alpha}\)).
\[I = [y_{\text{pred}} - \gamma_{\alpha}, y_{\text{pred}} + \gamma_{\alpha}]\]- Parameters:
y_pred (Iterable) – predictions.
scores_quantile (ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\).
- Returns:
prediction intervals \(I\).
- Return type:
Tuple[ndarray]
- Raises:
TypeError – unsupported data types.
- prediction_sets.cqr_interval(Y_pred, scores_quantile)
CQR prediction interval. Considering \(Y_{\text{pred}} = (q_{\text{lo}}, q_{\text{hi}})\), the prediction interval is built from the upper and lower quantiles predictions and scores_quantile \(\gamma_{\alpha}\).
\[I = [q_{\text{lo}} - \gamma_{\alpha}, q_{\text{lo}} + \gamma_{\alpha}]\]- Parameters:
y_pred (Iterable) – predictions.
scores_quantile (ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\).
- Returns:
scaled prediction intervals \(I\).
- Return type:
Tuple[ndarray]
- Raises:
TypeError – unsupported data types.
- prediction_sets.raps_set(Y_pred, scores_quantile, lambd=0, k_reg=1, rand=True)
RAPS prediction set.
Warning
This signature is incompatible with the interface of calibrators. Use
raps_set_builder()
to properly initializedeel.puncc.api.calibration.BaseCalibrator
.- Parameters:
Y_pred (Iterable) – \(Y_{\text{pred}} = (P_{\text{C}_1}, ..., P_{\text{C}_n})\) where \(P_{\text{C}_i}\) is logit associated to class i.
scores_quantile (ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\)
lambd (float) – positive weight associated to the regularization term that encourages small set sizes. If \(\lambda = 0\), there is no regularization and the implementation identifies with APS.
k_reg (float) – class rank (ordered by descending probability) starting from which the regularization is applied. For example, if \(k_{reg} = 3\), then the fourth most likely estimated class has an extra penalty of size \(\lambda\).
- : param bool rand: turn on or off the randomization term that smoothes the
discrete probability mass jump when including a new class.
- Returns:
RAPS prediction sets.
- Return type:
Iterable
- prediction_sets.raps_set_builder(lambd=0, k_reg=1, rand=True)
RAPS prediction set builder. When called, returns a RAPS prediction set function
raps_set()
with given initialitation of regularization hyperparameters.- Parameters:
lambd (float) – positive weight associated to the regularization term that encourages small set sizes. If \(\lambda = 0\), there is no regularization and the implementation identifies with APS.
k_reg (float) – class rank (ordered by descending probability) starting from which the regularization is applied. For example, if \(k_{reg} = 3\), then the fourth most likely estimated class has an extra penalty of size \(\lambda\).
- : param bool rand: turn on or off the randomization term that smoothes the
discrete probability mass jump when including a new class.
- Returns:
RAPS prediction set function that takes two parameters: Y_pred and scores_quantile.
- Return type:
Callable
- Raises:
ValueError – incorrect value of lambd or k_reg.
TypeError – unsupported data types.
- prediction_sets.scaled_bbox(Y_pred, scores_quantile)
Scaled upper and lower bounds of the bounding box coordinates for a given prediction coordinates.
- Parameters:
Y_pred (np.ndarray) – the predicted bounding box coordinates.
scores_quantile (np.ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\).
- Returns:
The coordinates of the inner and outer bounding boxes.
- Return type:
Tuple[np.ndarray, np.ndarray]
- Raises:
TypeError – unsupported data types.
- prediction_sets.scaled_interval(Y_pred, scores_quantile, eps=1e-12)
Scaled prediction interval centered on y_pred. Considering \(Y_{\text{pred}} = (\mu_{\text{pred}}, \sigma_{\text{pred}})\), the size of the margin is proportional to scores_quantile \(\gamma_{\alpha}\).
\[I = [\mu_{\text{pred}} - \gamma_{\alpha} \cdot \sigma_{\text{pred}}, y_{\text{pred}} + \gamma_{\alpha} \cdot \sigma_{\text{pred}}]\]- Parameters:
y_pred (Iterable) – predictions.
scores_quantile (ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\).
eps (float) – small positive value to avoid singleton sets.
- Returns:
scaled prediction intervals \(I\).
- Return type:
Tuple[ndarray]
- Raises:
TypeError – unsupported data types.