FEM (Feature Explanation Method)

View source | 📰 Paper

FEM is a fast, class-agnostic explanation that relies only on activations of a convolutional layer. For each channel, FEM applies a k-sigma threshold to detect unusually strong responses, builds a binary mask, and aggregates those masks weighted by the channel mean.

More formally, let \(A^k \in \mathbb{R}^{H \times W}\) be the \(k\)-th activation map and define its mean \(\mu_k\) and standard deviation \(\sigma_k\) over the spatial dimensions. The threshold is \(T_k = \mu_k + k\sigma_k\) (with \(k\) typically 2). The binary mask is \(M_k = \mathbb{1}(A^k \geq T_k)\) and the channel weight is \(w_k = \mu_k\). The final FEM attribution is

\[ \phi = \sum_k w_k \cdot M_k. \]

Because no gradients are required, FEM is inexpensive and suitable for inspecting general regions of interest, though it is class-agnostic by design.

Limitations / Compatibility

FEM is class-agnostic and does not use labels or targets, so it cannot isolate a specific class. In Xplique, FEM is implemented for TensorFlow/Keras and expects convolutional feature maps from image inputs (rank 4 tensors). Models without a convolutional layer are not supported, and FEM will raise an error if no suitable conv_layer can be found.

Example

from xplique.attributions import FEM

# load images and model
# ...

explainer = FEM(model, k=2.0)
explanations = explainer.explain(images)

Notebooks

• Attribution Methods: Getting started
• Attributions: FEM

References

• Fuad et al., "Features Understanding in 3D CNNs for Actions Recognition in Video" (IPTA 2020, IEEE). DOI
• Bourroux et al., "Multi Layered Feature Explanation Method for Convolutional Neural Networks" (ICPRAI 2022, LNCS). DOI
• Zhukov et al., "Evaluation of Explanation Methods of AI - CNNs in Image Classification Tasks with Reference-based and No-reference Metrics" (Advances in Artificial Intelligence and Machine Learning, 2023). DOI
• Zhukov & Benois-Pineau et al., "FEM and Multi-Layered FEM: Feature Explanation Methods with Statistical Filtering of Important Features" in Emerging Topics in Pattern Recognition and Artificial Intelligence (World Scientific, 2024). Book
• Ayyar et al., "Review of white box methods for explanations of convolutional neural networks in image classification tasks" (Journal of Electronic Imaging, 2021). DOI
• Ayyar et al., "A Feature Understanding Method for Explanation of Image Classification by Convolutional Neural Networks" in Explainable Deep Learning AI (Academic Press, 2023). Book

FEM

Gradient-free, class-agnostic explanation built from activations.

__init__(self,
         model: keras.src.engine.training.Model,
         batch_size: Optional[int] = 32,
         conv_layer: Union[str, int, None] = None,
         k: float = 2.0)

Parameters

• model : keras.src.engine.training.Model

  • The model from which we want to obtain explanations.

• batch_size : Optional[int] = 32

  • Number of inputs to explain at once, if None compute all at once.

• conv_layer : Union[str, int, None] = None

  • Layer to target for FEM. If an int is provided it will be interpreted as a layer index.

    If a string is provided it will look for the layer name. Defaults to last conv layer.

• k : float = 2.0

  • Number of standard deviations used in the threshold (k-sigma rule).

explain(self,
        inputs: Union[tf.Dataset, tensorflow.python.framework.tensor.Tensor, numpy.ndarray],
        targets: Union[tensorflow.python.framework.tensor.Tensor, numpy.ndarray, None] = None) -> tensorflow.python.framework.tensor.Tensor

Compute FEM maps for a batch of samples.

Parameters

• inputs : Union[tf.Dataset, tensorflow.python.framework.tensor.Tensor, numpy.ndarray]

  • Dataset, Tensor or Array. Input samples to be explained.

    If Dataset, targets should not be provided (included in Dataset).

    Expected shape (N, H, W, C).

    More information in the documentation.

• targets : Union[tensorflow.python.framework.tensor.Tensor, numpy.ndarray, None] = None

  • Not used. Present for API symmetry.

Return

• explanations : tensorflow.python.framework.tensor.Tensor

  • FEM maps resized to the spatial resolution of the inputs.