🖼️ Object Detection

Currently implemented conformal object detection methods are listed in this page.

Each of these wrappers conformalize object localization models that are passed as argument in the object constructor.

class deel.puncc.object_detection.SplitBoxWise(predictor, *, train=False, weight_func=None, method='additive', random_state=0)

Implementation of box-wise conformal object detection. For more info, we refer the user to the theory overview page

Parameters:

  • predictor (BasePredictor | Any) – a predictive model.

  • train (bool) – if False, prediction model(s) will not be (re)trained. Defaults to False.

  • weight_func (callable) – function that takes as argument an array of features X and returns associated “conformality” weights, defaults to None.

  • method (str) – chose between “additive” or “multiplicative” box-wise conformalization.

  • random_state (int) – random seed used when the user does not provide a custom fit/calibration split in fit method.

Raises:

ValueError – if method is not ‘additive’ or ‘multiplicative’.

Example:

from deel.puncc.object_detection import SplitBoxWise
import numpy as np

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestRegressor

from deel.puncc.metrics import object_detection_mean_coverage
from deel.puncc.metrics import object_detection_mean_area

# Generate a random regression problem
X, y = make_regression(
    n_samples=1000,
    n_features=4,
    n_informative=2,
    n_targets=4,
    random_state=0,
    shuffle=False,
)

# Create dummy object localization data formated as (x1, y1, x2, y2)
y = np.abs(y)
x1 = np.min(y[:, :2], axis=1)
y1 = np.min(y[:, 2:], axis=1)
x2 = np.max(y[:, :2], axis=1)
y2 = np.max(y[:, 2:], axis=1)
y = np.array([x1, y1, x2, y2]).T


# Split data into train and test
X, X_test, y, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

# Split train data into fit and calibration
X_fit, X_calib, y_fit, y_calib = train_test_split(
    X, y, test_size=0.2, random_state=0
)

# Create a random forest model
rf_model = RandomForestRegressor(n_estimators=100, random_state=0)

# CP method initialization
od_cp = SplitBoxWise(rf_model, method="multiplicative", train=True)

# The call to `fit` trains the model and computes the nonconformity
# scores on the calibration set
od_cp.fit(X_fit=X_fit, y_fit=y_fit, X_calib=X_calib, y_calib=y_calib)

# The predict method infers prediction intervals with respect to
# the significance level alpha = 20%
y_pred, box_inner, box_outer = od_cp.predict(X_test, alpha=0.2)

# Compute marginal coverage and average width of the prediction intervals
coverage = object_detection_mean_coverage(box_outer, y_test)
average_area = object_detection_mean_area(box_outer)
print(f"Marginal coverage: {np.round(coverage, 2)}")
fit(*, X=None, y=None, fit_ratio=0.8, X_fit=None, y_fit=None, X_calib=None, y_calib=None, use_cached=False, **kwargs)

This method fits the models on the fit data and computes nonconformity scores on calibration data. If (X, y) are provided, randomly split data into fit and calib subsets w.r.t to the fit_ratio. In case (X_fit, y_fit) and (X_calib, y_calib) are provided, the conformalization is performed on the given user defined fit and calibration sets.

Note

If X and y are provided, fit ignores any user-defined fit/calib split.

Parameters:

  • X (Iterable) – features from the training dataset.

  • y (Iterable) – labels from the training dataset.

  • fit_ratio (float) – the proportion of samples assigned to the fit subset.

  • X_fit (Iterable) – features from the fit dataset.

  • y_fit (Iterable) – labels from the fit dataset.

  • X_calib (Iterable) – features from the calibration dataset.

  • y_calib (Iterable) – labels from the calibration dataset.

  • use_cached (bool) – if set, enables to add the previously computed nonconformity scores (if any) to the pool estimated in the current call to fit. The aggregation follows the CV+ procedure.

  • kwargs (dict) – predict configuration to be passed to the model’s fit method.

Raises:

RuntimeError – no dataset provided.

get_nonconformity_scores()

Get computed nonconformity scores.

Returns:

computed nonconfomity scores.

Return type:

ndarray

predict(X_test, alpha, correction_func=<function SplitBoxWise.<lambda>>)

Conformal object detection (w.r.t target miscoverage alpha) for new samples.

Parameters:

  • X_test (Iterable) – features of new samples.

  • alpha (float) – target maximum miscoverage.

  • correction_func (Callable) – correction for multiple hypothesis testing in the case of multivariate regression. Defaults to Bonferroni correction.

Returns:

y_pred, y_lower, y_higher

Return type:

Tuple[ndarray]