Sobol Attribution Method

View colab tutorial | View source | 📰 Paper

The Sobol attribution method from Fel, Cadène & al.¹ is an attribution method grounded in Sensitivity Analysis. Beyond modeling the individual contributions of image regions, Sobol indices provide efficient way to capture higher-order interactions between image regions and their contributions to a neural network’s prediction through the lens of variance.

Quote

The total Sobol index $S{T}_i$ which measures the contribution of the variable $X_i$ as well as its interactions of any order with any other input variables to the model output variance.

-- Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis (2021)¹

More precisely, the attribution score ${\varphi }_i$ for an input variable $x_i$, is defined as

$${\varphi }_i=\frac{{\mathbb{E}}_{X\sim i}(Va{r}_{X_i}(f(x)|X_{\sim i}))}{Var(f(X))}$$

Where ${\mathbb{E}}_{X\sim i}(Va{r}_{X_i}(f(x)|X_{\sim i}))$ is the expected variance that would be left if all variables but $X_{\sim i}$ were to be fixed.

In order to generate stochasticity($X_i$), a perturbation function is used and uses perturbation masks to modulate the generated perturbation. The perturbation functions available are inpainting that modulates pixel regions to a baseline state, amplitude and blurring.

The calculation of the indices also requires an estimator -- in practice this parameter does not change the results much -- JansenEstimator being recommended.

Finally the exploration of the manifold exploration is made using a sampling method, several samplers are proposed: Quasi-Monte Carlo (ScipySobolSequence, recommended) using Scipy's sobol sequence, Latin hypercubes -- LHSAmpler -- or Halton's sequences HaltonSequence.

Tip

For a quick a faithful explanations, we recommend to use grid_size in $[7,12)$, nb_design in $\{16,32,64\}$ (more is useless), and a QMC sampler.

Example

from xplique.attributions import SobolAttributionMethod
from xplique.attributions.global_sensitivity_analysis import (
    JansenEstimator, GlenEstimator,
    LHSampler, ScipySobolSequence,
    HaltonSequence)

# load images, labels and model
# ...

# default explainer (recommended)
explainer = SobolAttributionMethod(model, grid_size=8, nb_design=32)
explanations = method(images, labels) # one-hot encoded labels

If you want to change the estimator or the sampling:

from xplique.attributions import SobolAttributionMethod
from xplique.attributions.global_sensitivity_analysis import (
    JansenEstimator, GlenEstimator,
    LHSampler, ScipySobolSequence,
    HaltonSequence)

# load images, labels and model
# ...

explainer_lhs = SobolAttributionMethod(model, grid_size=8, nb_design=32, 
                                       sampler=LHSampler(), 
                                       estimator=GlenEstimator())
explanations_lhs = explainer_lhs(images, labels)

Notebooks

• Attribution Methods: Getting started

SobolAttributionMethod

Sobol' Attribution Method. Compute the total order Sobol' indices using a perturbation function on a grid and an adapted sampling as described in the original paper.

__init__(self,
         model,
         grid_size: int = 8,
         nb_design: int = 32,
         sampler: Optional[xplique.attributions.global_sensitivity_analysis.replicated_designs.ReplicatedSampler] = None,
         estimator: Optional[xplique.attributions.global_sensitivity_analysis.sobol_estimators.SobolEstimator] = None,
         perturbation_function: Union[Callable, str, None] = 'inpainting',
         batch_size=256,
         operator: Optional[Callable[[keras.engine.training.Model, tf.Tensor, tf.Tensor], float]] = None)

Parameters

• model : model

  • Model used for computing explanations.

• grid_size : int = 8

  • Cut the image in a grid of (grid_size, grid_size) to estimate an indice per cell.

• nb_design : int = 32

  • Must be a power of two. Number of design, the number of forward will be: nb_design * (grid_size**2 + 2). Generally not above 32.

• sampler : Optional[xplique.attributions.global_sensitivity_analysis.replicated_designs.ReplicatedSampler] = None

  • Sampler used to generate the (quasi-)monte carlo samples, QMC (sobol sequence recommended). For more option, see the sampler module.

• estimator : Optional[xplique.attributions.global_sensitivity_analysis.sobol_estimators.SobolEstimator] = None

  • Estimator used to compute the total order sobol' indices, Jansen recommended. For more option, see the estimator module.

• perturbation_function : Union[Callable, str, None] = 'inpainting'

  • Function to call to apply the perturbation on the input. Can also be string in 'inpainting', 'blur'.

• batch_size : batch_size=256

  • Batch size to use for the forwards.

• operator : Optional[Callable[[keras.engine.training.Model, tf.Tensor, tf.Tensor], float]] = None

  • Function g to explain, g take 3 parameters (f, x, y) and should return a scalar, with f the model, x the inputs and y the targets. If None, use the standard operator g(f, x, y) = f(x)[y].

explain(self,
        inputs: Union[tf.Dataset, tf.Tensor, numpy.ndarray],
        targets: Union[tf.Tensor, numpy.ndarray, None] = None) -> tf.Tensor

Compute the total Sobol' indices according to the explainer parameter (perturbation function, grid size...). Accept Tensor, numpy array or tf.data.Dataset (in that case targets is None).

Parameters

• inputs : Union[tf.Dataset, tf.Tensor, numpy.ndarray]

  • Images to be explained, either tf.dataset, Tensor or numpy array.

    If Dataset, targets should not be provided (included in Dataset).

    Expected shape (N, W, H, C) or (N, W, H).

• targets : Union[tf.Tensor, numpy.ndarray, None] = None

  • One-hot encoding for classification or direction {-1, +1} for regression.

    Tensor or numpy array.

    Expected shape (N, C) or (N).

Return

• attributions_maps : tf.Tensor

  • GSA Attribution Method explanations, same shape as the inputs except for the channels.

---

1. Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis (2021) ↩↩