Average Increase in Confidence

• Metrics: Average Drop/Increase/Gain Fidelity

Average Increase in Confidence (AIC) measures the fraction of samples for which the masked input yields a higher score than the original input. This binary indicator provides a simple measure of explanation quality.

Quote

We use the Average Increase metric to evaluate whether the saliency map highlights features that, when isolated, increase the model's confidence.

-- Grad-CAM++: Generalized Gradient-based Visual Explanations for Deep Convolutional Networks (2018)¹

Formally, for each sample \(i\):

\[ AIC_i = \mathbb{1}[after_i > base_i] \]

Where: - \(base_i = g(f, x_i, y_i)\) is the model score on the original input - \(after_i = g(f, x_i \odot M_i, y_i)\) is the model score after masking with explanation-derived mask \(M_i\)

The dataset-level AIC is the mean of all \(AIC_i\) values.

Info

The better the explanation, the higher the Average Increase score. A high score indicates that explanations capture truly discriminative features.

Score Interpretation

• Higher scores are better: A high Average Increase means that for more samples, the explanation identifies features which, when isolated, are sufficient or even more predictive than the full input.
• Returns binary indicators (0 or 1); the dataset-level metric is the proportion of samples showing an increase.
• This metric reveals whether explanations capture truly discriminative features.

Example

from xplique.metrics import AverageIncreaseMetric
from xplique.attributions import Saliency

# load images, labels and model
# ...
explainer = Saliency(model)
explanations = explainer(inputs, labels)

metric = AverageIncreaseMetric(model, inputs, labels, activation="softmax")
score = metric.evaluate(explanations)

Tip

This metric works best with probabilistic outputs. Set activation="softmax" or "sigmoid" if your model returns logits.

AverageIncreaseMetric

Average Increase in Confidence (AIC) — fraction of samples for which the masked input yields a higher score (higher is better).

__init__(self,
         model: keras.src.engine.training.Model,
         inputs: Union[tf.Dataset, tensorflow.python.framework.tensor.Tensor, numpy.ndarray],
         targets: Union[tensorflow.python.framework.tensor.Tensor, numpy.ndarray, None] = None,
         batch_size: Optional[int] = 64,
         operator: Union[str, Callable, None] = None,
         activation: Optional[str] = None)

Parameters

• model : keras.src.engine.training.Model

  • Model used for computing metric.

• inputs : Union[tf.Dataset, tensorflow.python.framework.tensor.Tensor, numpy.ndarray]

  • Input samples under study.

• targets : Union[tensorflow.python.framework.tensor.Tensor, numpy.ndarray, None] = None

  • One-hot encoded labels or regression target (e.g {+1, -1}), one for each sample.

• batch_size : Optional[int] = 64

  • Number of samples to process at once, if None compute all at once.

• operator : Union[str, Callable, None] = None

  • Function g to explain. It should take 3 parameters (f, x, y) and return a scalar, with f the model, x the inputs and y the targets. If None, use the standard operator g(f, x, y) = f(x)[y].

• activation : Optional[str] = None

  • A string that belongs to [None, 'sigmoid', 'softmax']. Specify if we should add an activation layer once the model has been called.

detailed_evaluate(self,
                  inputs: tensorflow.python.framework.tensor.Tensor,
                  targets: tensorflow.python.framework.tensor.Tensor,
                  explanations: tensorflow.python.framework.tensor.Tensor) -> numpy.ndarray

Compute Average Increase indicators for a batch of samples.

Parameters

• inputs : tf.Tensor

  • Batch of input samples. Shape: (B, H, W, C) for images, (B, T, F) for time series, or (B, ...) for other data types.

• targets : tf.Tensor

  • Batch of target labels. Shape: (B, num_classes) for one-hot encoded, or (B,) for class indices/regression targets.

• explanations : tf.Tensor

  • Batch of attribution maps. Shape must be compatible with inputs (same spatial/temporal dimensions, optionally without channel dimension).

Return

• scores : np.ndarray

  • Per-sample binary indicators, shape (B,).

    Values are 0 (no increase) or 1 (confidence increased).

    Higher mean values indicate better explanations.

evaluate(self,
         explanations: Union[tensorflow.python.framework.tensor.Tensor, numpy.ndarray]) -> float

Evaluate the metric over the entire dataset by iterating over batches.

---

1. Grad-CAM++: Generalized Gradient-based Visual Explanations for Deep Convolutional Networks (2018) ↩