Prediction sets

This module provides prediction sets for conformal prediction. To be used when building a calibrator.

prediction_sets.constant_bbox(Y_pred, scores_quantile)

Generate the upper and lower bounds of the bounding box coordinates for a given prediction.

Parameters:

  • Y_pred (np.ndarray) – the predicted bounding box coordinates.

  • scores_quantile (np.ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\).

Returns:

the lower bound and upper bound coordinates of the bounding box.

Return type:

Tuple[np.ndarray, np.ndarray]

Raises:

TypeError – unsupported data types.

prediction_sets.constant_interval(y_pred, scores_quantile)

Constant prediction interval centered on y_pred. The size of the margin is scores_quantile (noted \(\gamma_{\alpha}\)).

\[I = [y_{\text{pred}} - \gamma_{\alpha}, y_{\text{pred}} + \gamma_{\alpha}]\]
Parameters:

  • y_pred (Iterable) – predictions.

  • scores_quantile (ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\).

Returns:

prediction intervals \(I\).

Return type:

Tuple[ndarray]

Raises:

TypeError – unsupported data types.

prediction_sets.cqr_interval(Y_pred, scores_quantile)

CQR prediction interval. Considering \(Y_{\text{pred}} = (q_{\text{lo}}, q_{\text{hi}})\), the prediction interval is built from the upper and lower quantiles predictions and scores_quantile \(\gamma_{\alpha}\).

\[I = [q_{\text{lo}} - \gamma_{\alpha}, q_{\text{lo}} + \gamma_{\alpha}]\]
Parameters:

  • y_pred (Iterable) – predictions.

  • scores_quantile (ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\).

Returns:

scaled prediction intervals \(I\).

Return type:

Tuple[ndarray]

Raises:

TypeError – unsupported data types.

prediction_sets.lac_set(Y_pred, scores_quantile)

LAC prediction set.

Parameters:

  • Y_pred (Iterable) – \(Y_{\text{pred}} = (P_{\text{C}_1}, ..., P_{\text{C}_n})\) where \(P_{\text{C}_i}\) is logit associated to class i.

  • scores_quantile (ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\)

Returns:

LAC prediction sets.

Return type:

Iterable

prediction_sets.raps_set(Y_pred, scores_quantile, lambd=0, k_reg=1, rand=True)

RAPS prediction set.

Warning

This signature is incompatible with the interface of calibrators. Use raps_set_builder() to properly initialize deel.puncc.api.calibration.BaseCalibrator.

Parameters:

  • Y_pred (Iterable) – \(Y_{\text{pred}} = (P_{\text{C}_1}, ..., P_{\text{C}_n})\) where \(P_{\text{C}_i}\) is logit associated to class i.

  • scores_quantile (ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\)

  • lambd (float) – positive weight associated to the regularization term that encourages small set sizes. If \(\lambda = 0\), there is no regularization and the implementation identifies with APS.

  • k_reg (float) – class rank (ordered by descending probability) starting from which the regularization is applied. For example, if \(k_{reg} = 3\), then the fourth most likely estimated class has an extra penalty of size \(\lambda\).

: param bool rand: turn on or off the randomization term that smoothes the

discrete probability mass jump when including a new class.

Returns:

RAPS prediction sets.

Return type:

Iterable

prediction_sets.raps_set_builder(lambd=0, k_reg=1, rand=True)

RAPS prediction set builder. When called, returns a RAPS prediction set function raps_set() with given initialitation of regularization hyperparameters.

Parameters:

  • lambd (float) – positive weight associated to the regularization term that encourages small set sizes. If \(\lambda = 0\), there is no regularization and the implementation identifies with APS.

  • k_reg (float) – class rank (ordered by descending probability) starting from which the regularization is applied. For example, if \(k_{reg} = 3\), then the fourth most likely estimated class has an extra penalty of size \(\lambda\).

: param bool rand: turn on or off the randomization term that smoothes the

discrete probability mass jump when including a new class.

Returns:

RAPS prediction set function that takes two parameters: Y_pred and scores_quantile.

Return type:

Callable

Raises:

  • ValueError – incorrect value of lambd or k_reg.

  • TypeError – unsupported data types.

prediction_sets.scaled_bbox(Y_pred, scores_quantile)

Scaled upper and lower bounds of the bounding box coordinates for a given prediction coordinates.

Parameters:

  • Y_pred (np.ndarray) – the predicted bounding box coordinates.

  • scores_quantile (np.ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\).

Returns:

The coordinates of the inner and outer bounding boxes.

Return type:

Tuple[np.ndarray, np.ndarray]

Raises:

TypeError – unsupported data types.

prediction_sets.scaled_interval(Y_pred, scores_quantile, eps=1e-12)

Scaled prediction interval centered on y_pred. Considering \(Y_{\text{pred}} = (\mu_{\text{pred}}, \sigma_{\text{pred}})\), the size of the margin is proportional to scores_quantile \(\gamma_{\alpha}\).

\[I = [\mu_{\text{pred}} - \gamma_{\alpha} \cdot \sigma_{\text{pred}}, y_{\text{pred}} + \gamma_{\alpha} \cdot \sigma_{\text{pred}}]\]
Parameters:

  • y_pred (Iterable) – predictions.

  • scores_quantile (ndarray) – quantile of nonconformity scores computed on a calibration set for a given \(\alpha\).

  • eps (float) – small positive value to avoid singleton sets.

Returns:

scaled prediction intervals \(I\).

Return type:

Tuple[ndarray]

Raises:

TypeError – unsupported data types.