Square Grad

View colab tutorial | View source | 📰 Paper

Similar to SmoothGrad, Square-Grad is a gradient-based explanation method, which, as the name suggests, averages the square of the gradient at several points corresponding to small perturbations around the point of interest. The smoothing effect induced by the average help reducing the visual noise, and hence improve the explanations.

More precisely, the explanation \(\phi\) for an input \(x\) and a classifier \(f\) is defined as

\[ \phi = \mathbb{E}_{\delta ~\sim~ \mathcal{N}(0, \sigma^2) }( (\nabla_x f(x + \delta))^2 ) \approx \frac{1}{N} \sum_{i=0}^N (\nabla_x f(x + \delta_i))^2 \]

The \(\sigma\) in the formula is controlled using the noise parameter, and the expectation is estimated using \(N\) samples controlled by the nb_samples parameter.

Tip

It is recommended to have a noise level \(\sigma\) at about 20% of the range of your inputs, i.e. \(\sigma=0.2\) if your inputs are between \([0, 1]\) or \(\sigma=0.4\) if your inputs are between \([-1, 1]\).

Example

from xplique.attributions import SquareGrad

# load images, labels and model
# ...

method = SquareGrad(model, nb_samples=50, noise=0.15)
explanations = method.explain(images, labels)

Notebooks

• Attribution Methods: Getting started
• SquareGrad: Going Further

SquareGrad

SquareGrad (or SmoothGrad^2) is an unpublished variant of classic SmoothGrad which squares each gradients of the noisy inputs before averaging.

__init__(self,
         model: keras.src.engine.training.Model,
         output_layer: Union[str, int, None] = None,
         batch_size: Optional[int] = 32,
         operator: Union[xplique.commons.operators.Tasks, str,
         Callable[[keras.src.engine.training.Model, tf.Tensor, tf.Tensor], float], None] = None,
         nb_samples: int = 50,
         noise: float = 0.2)

Parameters

• model : keras.src.engine.training.Model

  • The model from which we want to obtain explanations

• output_layer : Union[str, int, None] = None

  • Layer to target for the outputs (e.g logits or after softmax).

    If an int is provided it will be interpreted as a layer index.

    If a string is provided it will look for the layer name.

    Default to the last layer.

    It is recommended to use the layer before Softmax.

• batch_size : Optional[int] = 32

  • Number of inputs to explain at once, if None compute all at once.

• operator : Union[xplique.commons.operators.Tasks, str, Callable[[keras.src.engine.training.Model, tf.Tensor, tf.Tensor], float], None] = None

  • Function g to explain, g take 3 parameters (f, x, y) and should return a scalar, with f the model, x the inputs and y the targets. If None, use the standard operator g(f, x, y) = f(x)[y].

• nb_samples : int = 50

  • Number of noisy samples generated for the smoothing procedure.

• noise : float = 0.2

  • Scalar, noise used as standard deviation of a normal law centered on zero.

explain(self,
        inputs: Union[tf.Dataset, tf.Tensor, numpy.ndarray],
        targets: Union[tf.Tensor, numpy.ndarray, None] = None) -> tf.Tensor

Compute SmoothGrad for a batch of samples.

Parameters

• inputs : Union[tf.Dataset, tf.Tensor, numpy.ndarray]

  • Dataset, Tensor or Array. Input samples to be explained.

    If Dataset, targets should not be provided (included in Dataset).

    Expected shape among (N, W), (N, T, W), (N, H, W, C).

    More information in the documentation.

• targets : Union[tf.Tensor, numpy.ndarray, None] = None

  • Tensor or Array. One-hot encoding of the model's output from which an explanation is desired. One encoding per input and only one output at a time. Therefore, the expected shape is (N, output_size).

    More information in the documentation.

Return

• explanations : tf.Tensor

  • Smoothed gradients, same shape as the inputs.