Normalization#

normalize() and denormalize() treat dimension 0 as batch and dimension 1 as channel; normalize_min_max() takes \((*, C, H, W)\). ZCA uses its dim argument as the sample axis and flattens all other axes into features.

Functions#

kornia.enhance.normalize(data, mean, std)[source]#

Normalize an image/video torch.Tensor with mean and standard deviation.

Try in browser

Convention:: This function treats dimension 0 as batch and dimension 1 as channel. mean and std may be (C,), (1, C), or (B, C); the output has the input shape.

\[\text{input[channel] = (input[channel] - mean[channel]) / std[channel]}\]

Where mean is \((M_1, ..., M_n)\) and std \((S_1, ..., S_n)\) for n channels,

Parameters:

data (Tensor) – Image torch.Tensor of size \((B, C, *)\).
mean (Tensor) – Mean for each channel.
std (Tensor) – Standard deviations for each channel.

Return type:

Tensor

Returns:

Normalised torch.Tensor with same size as input \((B, C, *)\).

Note

An integer or bool data is converted to torch’s default floating dtype (float32 unless changed) before mean and std are applied, so a fractional statistic is not truncated and the result is floating. Floating and complex inputs keep their dtype. Pixel values are not rescaled: give mean and std in the 0-255 scale for a uint8 image.

Examples

>>> x = torch.rand(1, 4, 3, 3)
>>> out = normalize(x, torch.tensor([0.0]), torch.tensor([255.]))
>>> out.shape
torch.Size([1, 4, 3, 3])

>>> x = torch.tensor([[[[0, 255]]]], dtype=torch.uint8)
>>> normalize(x, 127.5, 127.5)
tensor([[[[-1.,  1.]]]])

>>> x = torch.rand(1, 4, 3, 3)
>>> mean = torch.zeros(4)
>>> std = 255. * torch.ones(4)
>>> out = normalize(x, mean, std)
>>> out.shape
torch.Size([1, 4, 3, 3])

kornia.enhance.normalize_min_max(input, min_val=0.0, max_val=1.0, eps=1e-6)[source]#

Normalise an image/video torch.Tensor by MinMax and re-scales the value between a range.

Try in browser

Convention:: Input is (*, C, H, W): minima and maxima are taken over H and W separately for every leading index and channel. Constant planes map to min_val. Empty leading dimensions are preserved; channel and spatial dimensions must be nonzero.

The data is normalised using the following formulation:

\[y_i = (b - a) * \frac{x_i - \text{min}(x)}{\text{max}(x) - \text{min}(x)} + a\]

where \(a\) is \(\text{min_val}\) and \(b\) is \(\text{max_val}\).

Parameters:

input (Tensor) – The image torch.Tensor to be normalised with shape \((*, C, H, W)\).
min_val (float, optional) – The minimum value for the new range. Default: 0.0
max_val (float, optional) – The maximum value for the new range. Default: 1.0
eps (float, optional) – Float number to avoid zero division. Default: 1e-6

Return type:

Tensor

Returns:

The normalised image torch.Tensor with same shape as input \((*, C, H, W)\).

Example

>>> x = torch.rand(1, 5, 3, 3)
>>> x_norm = normalize_min_max(x, min_val=-1., max_val=1.)
>>> x_norm.min()
tensor(-1.)
>>> x_norm.max()
tensor(1.0000)

kornia.enhance.denormalize(data, mean, std)[source]#

Denormalize an image/video torch.Tensor with mean and standard deviation.

Convention:: This is the elementwise inverse of normalize() for matching mean and std on (B, C, *) input.

\[\text{input[channel] = (input[channel] * std[channel]) + mean[channel]}\]

Where mean is \((M_1, ..., M_n)\) and std \((S_1, ..., S_n)\) for n channels,

Parameters:

data (Tensor) – Image torch.Tensor of size \((B, C, *)\).
mean (Union[Tensor, float]) – Mean for each channel.
std (Union[Tensor, float]) – Standard deviations for each channel.

Return type:

Tensor

Returns:

Denormalised torch.Tensor with same size as input \((B, C, *)\).

Note

An integer or bool data is converted to torch’s default floating dtype (float32 unless changed) before mean and std are applied, so the result is floating and cannot wrap. Floating and complex inputs keep their dtype.

Examples

>>> x = torch.rand(1, 4, 3, 3)
>>> out = denormalize(x, 0.0, 255.)
>>> out.shape
torch.Size([1, 4, 3, 3])

>>> x = torch.tensor([[[[0, 255]]]], dtype=torch.uint8)
>>> denormalize(x, 10., 2.)
tensor([[[[ 10., 520.]]]])

>>> x = torch.rand(1, 4, 3, 3, 3)
>>> mean = torch.zeros(1, 4)
>>> std = 255. * torch.ones(1, 4)
>>> out = denormalize(x, mean, std)
>>> out.shape
torch.Size([1, 4, 3, 3, 3])

kornia.enhance.zca_mean(inp, dim=0, unbiased=True, eps=1e-6, return_inverse=False)[source]#

Compute the ZCA whitening matrix and mean vector.

Convention:: See ZCAWhitening: dim selects samples and all other axes form features. Fitting runs with autocast disabled (not under TorchScript). Float16 and bfloat16 fitting uses float32 arithmetic; the mean and transforms keep the input dtype.

The output can be used with linear_transform(). See ZCAWhitening for details.

Parameters:

inp (Tensor) – input data torch.Tensor.
dim (int, optional) – Specifies the dimension that serves as the samples dimension. Default: 0
unbiased (bool, optional) – Whether to use the unbiased estimate of the covariance matrix. Default: True
eps (float, optional) – a small number used for numerical stability. Default: 1e-6
return_inverse (bool, optional) – Whether to return the inverse ZCA transform. Default: False

Shapes:

inp: \((D_0,...,D_{\text{dim}},...,D_N)\) is a batch of N-D tensors.
transform_matrix: \((\Pi_{d=0,d\neq \text{dim}}^N D_d, \Pi_{d=0,d\neq \text{dim}}^N D_d)\)
mean_vector: \((1, \Pi_{d=0,d\neq \text{dim}}^N D_d)\)
inv_transform: same shape as the transform matrix

Return type:: Tuple[Tensor, Tensor, Optional[Tensor]]
Returns:: A tuple containing the ZCA matrix and the mean vector. If return_inverse is set to True, then it returns the inverse ZCA matrix, otherwise it returns None.
Raises:: ValueError – If the sample dimension has fewer than two entries with unbiased=True, or none with unbiased=False.

Note

See a working example here.

Examples

>>> x = torch.tensor([[0,1],[1,0],[-1,0],[0,-1]], dtype = torch.float32)
>>> transform_matrix, mean_vector,_ = zca_mean(x) # Returns transformation matrix and data mean
>>> x = torch.rand(3,20,2,2)
>>> transform_matrix, mean_vector, inv_transform = zca_mean(x, dim = 1, return_inverse = True)
>>> # transform_matrix.size() equals (12,12) and the mean vector.size equal (1,12)

kornia.enhance.zca_whiten(inp, dim=0, unbiased=True, eps=1e-6)[source]#

Apply ZCA whitening transform.

Convention:: Fits and applies ZCA in one call; see ZCAWhitening for sample-axis and covariance-normalization conventions.

See ZCAWhitening for details.

Parameters:

inp (Tensor) – input data torch.Tensor.
dim (int, optional) – Specifies the dimension that serves as the samples dimension. Default: 0
unbiased (bool, optional) – Whether to use the unbiased estimate of the covariance matrix. Default: True
eps (float, optional) – a small number used for numerical stability. Default: 1e-6

Return type:

Tensor

Returns:

Whiten Input data.

Raises:

ValueError – If the sample dimension has fewer than two entries with unbiased=True, or none with unbiased=False.

Note

See a working example here.

Examples

>>> x = torch.tensor([[0,1],[1,0],[-1,0]], dtype = torch.float32)
>>> zca_whiten(x).round(decimals=3)
tensor([[ 0.0000,  1.1550],
        [ 1.0000, -0.5770],
        [-1.0000, -0.5770]])

kornia.enhance.linear_transform(inp, transform_matrix, mean_vector, dim=0)[source]#

Apply a transformation matrix after centering and flattening the input.

This function flattens the input torch.Tensor along the given dimension and subtracts the mean vector. The dot product with the transformation matrix is then reshaped to the original input shape.

Convention:: dim selects rows (samples); all other dimensions form a feature row. The result is (X - mean) @ transform_matrix with the original layout restored.

\[\mathbf{X}_{T} = (\mathbf{X - \mu})(T)\]

Parameters:

inp (Tensor) – Input data \(X\).
transform_matrix (Tensor) – Transform matrix \(T\).
mean_vector (Tensor) – mean vector \(\mu\).
dim (int, optional) – Batch dimension. Default: 0

Shapes:

inp: \((D_0,...,D_{\text{dim}},...,D_N)\) is a batch of N-D tensors.
transform_matrix: \((\Pi_{d=0,d\neq \text{dim}}^N D_d, \Pi_{d=0,d\neq \text{dim}}^N D_d)\)
mean_vector: \((1, \Pi_{d=0,d\neq \text{dim}}^N D_d)\)

Return type:: Tensor
Returns:: Transformed data.

Example

>>> # Example where dim = 3
>>> inp = torch.ones((10,3,4,5))
>>> transform_mat = torch.ones((10*3*4,10*3*4))
>>> mean = 2*torch.ones((1,10*3*4))
>>> out = linear_transform(inp, transform_mat, mean, 3)
>>> print(out.shape, out.unique())  # Should a be (10,3,4,5) torch.tensor of -120s
torch.Size([10, 3, 4, 5]) tensor([-120.])

>>> # Example where dim = 0
>>> inp = torch.ones((10,2))
>>> transform_mat = torch.ones((2,2))
>>> mean = torch.zeros((1,2))
>>> out = linear_transform(inp, transform_mat, mean)
>>> print(out.shape, out.unique()) # Should a be (10,2) torch.tensor of 2s
torch.Size([10, 2]) tensor([2.])

Modules#

class kornia.enhance.Normalize(mean, std)[source]#

Normalize a torch.Tensor image with mean and standard deviation.

Try in browser

Convention:: The channel axis is dimension 1: inputs are (B, C, *) and one statistic per channel has shape (C,) (or (B, C) for separate batch statistics). See normalize() for the functional contract.

\[\text{input[channel] = (input[channel] - mean[channel]) / std[channel]}\]

Where mean is \((M_1, ..., M_n)\) and std \((S_1, ..., S_n)\) for n channels,

Parameters:

mean (Union[Tensor, Tuple[float], List[float], float]) – Mean for each channel.
std (Union[Tensor, Tuple[float], List[float], float]) – Standard deviations for each channel.

Shape:

Input: Image torch.Tensor of size \((B, C, *)\).
Output: Normalised torch.Tensor with same size as input \((B, C, *)\).

Note

An integer or bool input is converted to torch’s default floating dtype first; see normalize().

Examples

>>> x = torch.rand(1, 4, 3, 3)
>>> out = Normalize(0.0, 255.)(x)
>>> out.shape
torch.Size([1, 4, 3, 3])

>>> x = torch.rand(1, 4, 3, 3)
>>> mean = torch.zeros(4)
>>> std = 255. * torch.ones(4)
>>> out = Normalize(mean, std)(x)
>>> out.shape
torch.Size([1, 4, 3, 3])

class kornia.enhance.Denormalize(mean, std)[source]#

Denormalize a torch.Tensor image with mean and standard deviation.

Convention:: See denormalize(); the inverse uses dimension 1 as channel on (B, C, *) input, matching Normalize.

\[\text{input[channel] = (input[channel] * std[channel]) + mean[channel]}\]

Where mean is \((M_1, ..., M_n)\) and std \((S_1, ..., S_n)\) for n channels,

Parameters:

mean (Union[Tensor, float]) – Mean for each channel.
std (Union[Tensor, float]) – Standard deviations for each channel.

Shape:

Input: Image torch.Tensor of size \((B, C, *)\).
Output: Denormalised torch.Tensor with same size as input \((B, C, *)\).

Note

An integer or bool input is converted to torch’s default floating dtype first; see denormalize().

Examples

>>> x = torch.rand(1, 4, 3, 3)
>>> out = Denormalize(0.0, 255.)(x)
>>> out.shape
torch.Size([1, 4, 3, 3])

>>> x = torch.rand(1, 4, 3, 3, 3)
>>> mean = torch.zeros(1, 4)
>>> std = 255. * torch.ones(1, 4)
>>> out = Denormalize(mean, std)(x)
>>> out.shape
torch.Size([1, 4, 3, 3, 3])

class kornia.enhance.ZCAWhitening(dim=0, eps=1e-6, unbiased=True, detach_transforms=True, compute_inv=False)[source]#

Compute the ZCA whitening matrix transform and the mean vector and applies the transform to the data.

Convention:: dim is the sample axis; every other axis is flattened into the feature vector. unbiased=True divides covariance by N - 1 and False by N. Call fit or pass include_fit=True before transforming. zca_whiten() computes the same transform per call.

The data torch.Tensor is flattened, and the mean \(\mathbf{\mu}\) and covariance matrix \(\mathbf{\Sigma}\) are computed from the flattened data \(\mathbf{X} \in \mathbb{R}^{N \times D}\), where \(N\) is the sample size and \(D\) is flattened dimensionality (e.g. for a torch.Tensor with size 5x3x2x2 \(N = 5\) and \(D = 12\)). The ZCA whitening transform is given by:

\[\mathbf{X}_{\text{zca}} = (\mathbf{X - \mu})(US^{-\frac{1}{2}}U^T)^T\]

where \(U\) are the eigenvectors of \(\Sigma\) and \(S\) contain the corresponding eigenvalues of \(\Sigma\). After the transform is applied, the output is reshaped to same shape.

The fitted mean and transforms are buffers: state_dict saves them, .to() moves and casts them, and loading a fitted state into an unfitted module fits it. An unfitted module holds no tensors, so the loaded ones keep the device and dtype they were saved with; move the module after loading. A checkpoint saved before the fitted state was persisted holds none of it, and loading it keeps the current fit.

Parameters:

dim (int, optional) – Determines the dimension that represents the samples axis. Default: 0
eps (float, optional) – a small number used for numerical stability. Default: 1e-6
unbiased (bool, optional) – Whether to divide covariance by N - 1 rather than N. Default: True
compute_inv (bool, optional) – Compute the inverse transform matrix. Default: False
detach_transforms (bool, optional) – Detaches gradient from the ZCA fitting. Default: True

shape:

x: \((D_0,...,D_{\text{dim}},...,D_N)\) is a batch of N-D tensors.
x_whiten: \((D_0,...,D_{\text{dim}},...,D_N)\) same shape as input.

Note

See a working example here.

Examples

>>> x = torch.tensor([[0,1],[1,0],[-1,0],[0,-1]], dtype = torch.float32)
>>> zca = ZCAWhitening().fit(x)
>>> x_whiten = zca(x)
>>> zca = ZCAWhitening()
>>> x_whiten = zca(x, include_fit = True) # Includes the fitting step
>>> x_whiten = zca(x) # Can run now without the fitting set
>>> # Enable backprop through ZCA fitting process
>>> zca = ZCAWhitening(detach_transforms = False)
>>> x_whiten = zca(x, include_fit = True) # Includes the fitting step

Note

This implementation uses svd() which yields NaNs in the backwards step if the singular values are not unique. See here for more information.

References

[1] Stanford PCA & ZCA whitening tutorial

fit(x)[source]#

Fit ZCA whitening matrices to the data.

Parameters:: x (Tensor) – Input data.
Return type:: ZCAWhitening
Returns:: Returns a fitted ZCAWhiten object instance.
Raises:: ValueError – If the sample dimension has fewer than two entries with unbiased=True, or none with unbiased=False.

forward(x, include_fit=False)[source]#

Apply the whitening transform to the data.

Parameters:

x (Tensor) – Input data.
include_fit (bool, optional) – Indicates whether to fit the data as part of the forward pass. Default: False

Return type:

Tensor

Returns:

The transformed data.

inverse_transform(x)[source]#

Apply the inverse transform to the whitened data.

Uses the same sample dimension specified by dim when fitting.

Parameters:: x (Tensor) – Whitened data.
Return type:: Tensor
Returns:: Original data.