deel.torchlip.init

The deel.torchlip.init contains functions that can be used to initialize weights of neural networks layers. Similar to the functions from torch.nn.init, these functions are in-place functions as indicated by their trailing _.

Warning

These initializers are provided for completeness but we recommend using torch.nn.init.orthogonal_() to initialize your weights when training Lipschitz neural networks.

Initializers

deel.torchlip.init.bjorck_(tensor: torch.Tensor, n_iterations: int = 15, beta: float = 0.5)

Apply Bjorck normalization on the given tensor in-place.

See also bjorck_normalization().

Warning

This function is provided for completeness but we recommend using torch.nn.init.orthogonal_() instead to obtain a proper (semi) orthogonal matrix.

Parameters

• tensor – A 2-dimensional torch.Tensor.

• n_iterations – Number of iterations to perform.

• beta – Value to use for the $\beta$ parameter.

deel.torchlip.init.spectral_(tensor: torch.Tensor, n_power_iterations: int = 10)

Apply spectral normalization on the given tensor in-place.

See also spectral_normalization().

Warning

This function is provided for completeness but we recommend using torch.nn.init.orthogonal_() instead to obtain a proper (semi) orthogonal matrix.

Parameters

• tensor – A 2-dimensional torch.Tensor.

• n_power_iterations – Number of iterations to perform.