deel.lip.model

This module contains equivalents for Model and Sequential. These classes add support for condensation and vanilla exportation.

Model

Bases: Model

Equivalent of keras.Model but support condensation and vanilla exportation.

Warning

As lipschitz constant are multiplicative along layer, the Model class cannot set a global Lipschitz constant (problem with branching inside a model).

condense

condense()

The condense operation allows to overwrite the kernel with constrained kernel and ensure that other variables are still consistent.

Source code in deel/lip/model.py

def condense(self):
    """
    The condense operation allows to overwrite the kernel with constrained kernel
    and ensure that other variables are still consistent.
    """
    for layer in self.layers:
        if isinstance(layer, Condensable):
            layer.condense()

vanilla_export

vanilla_export()

Export this model to a "Vanilla" model, i.e. a model without Condensable layers.

RETURNS	DESCRIPTION
Model	A Keras model, identical to this model, but where condensable layers have been replaced with their vanilla equivalent (e.g. SpectralConv2D with Conv2D).

Source code in deel/lip/model.py

def vanilla_export(self) -> KerasModel:
    """
    Export this model to a "Vanilla" model, i.e. a model without Condensable
    layers.

    Returns:
        A Keras model, identical to this model, but where condensable layers have
            been replaced with their vanilla equivalent (e.g. SpectralConv2D with
            Conv2D).
    """
    return vanillaModel(self)

Sequential

Sequential(layers=None, name=None, k_coef_lip=1.0)

Bases: Sequential, LipschitzLayer, Condensable

Equivalent of keras.Sequential but allow to set k-lip factor globally. Also support condensation and vanilla exportation. For now constant repartition is implemented (each layer get n_sqrt(k_lip_factor), where n is the number of layers) But in the future other repartition function may be implemented.

PARAMETER	DESCRIPTION
layers	list of layers to add to the model. TYPE: list DEFAULT: None
name	name of the model, can be None TYPE: str DEFAULT: None
k_coef_lip	the Lipschitz coefficient to ensure globally on the model. TYPE: float DEFAULT: 1.0

Source code in deel/lip/model.py

def __init__(
    self,
    layers=None,
    name=None,
    k_coef_lip=1.0,
):
    """
    Equivalent of keras.Sequential but allow to set k-lip factor globally. Also
    support condensation and vanilla exportation.
    For now constant repartition is implemented (each layer
    get n_sqrt(k_lip_factor), where n is the number of layers)
    But in the future other repartition function may be implemented.

    Args:
        layers (list): list of layers to add to the model.
        name (str): name of the model, can be None
        k_coef_lip (float): the Lipschitz coefficient to ensure globally on the
            model.
    """
    super(Sequential, self).__init__(layers, name)
    self.set_klip_factor(k_coef_lip)

coef_lip class-attribute instance-attribute

coef_lip = None

define correction coefficient (ie. Lipschitz bound ) of the layer ( multiply the output of the layer by this constant )

k_coef_lip class-attribute instance-attribute

k_coef_lip = 1.0

variable used to store the lipschitz factor

vanillaModel

vanillaModel(model)

Transform a model to its equivalent "vanilla" model, i.e. a model where Condensable layers are replaced with their vanilla equivalent. For example, SpectralConv2D layers are converted to tf.keras Conv2D layers.

The input model can be a tf.keras Sequential/Model or a deel.lip Sequential/Model.

PARAMETER	DESCRIPTION
model	a tf.keras or deel.lip model with Condensable layers.

RETURNS	DESCRIPTION
	A Keras model, identical to the input model where Condensable layers are replaced with their vanilla counterparts.

Source code in deel/lip/model.py

def vanillaModel(model):
    """
    Transform a model to its equivalent "vanilla" model, i.e. a model where
    `Condensable` layers are replaced with their vanilla equivalent. For example,
    `SpectralConv2D` layers are converted to tf.keras `Conv2D` layers.

    The input model can be a tf.keras Sequential/Model or a deel.lip Sequential/Model.

    Args:
        model: a tf.keras or deel.lip model with Condensable layers.

    Returns:
        A Keras model, identical to the input model where `Condensable` layers are
            replaced with their vanilla counterparts.
    """

    def _replace_condensable_layer(layer):
        # Return a vanilla layer if Condensable, else return a copy of the layer
        if isinstance(layer, Condensable):
            return layer.vanilla_export()
        new_layer = layer.__class__.from_config(layer.get_config())
        new_layer.build(layer.input_shape)
        new_layer.set_weights(layer.get_weights())
        return new_layer

    return clone_model(model, clone_function=_replace_condensable_layer)