Normalization#

Use normalize() with a known mean and standard deviation, normalize_min_max() to scale each channel plane by its own range, and zca_whiten() or ZCAWhitening to decorrelate features.

Functions#

kornia.enhance.normalize(data, mean, std)[source]#

Normalize an image or video with a per-channel mean and standard deviation.

Try in browser

out = normalize(data, mean, std) returns (data - mean) / std for each channel of data (B, C, *), with the channel on dimension 1.

Convention:

mean and std are converted to the dtype and device of data; a floating data keeps its dtype.
An integer or bool data is converted to torch’s default floating dtype first, so a fractional statistic is not truncated. Pixel values are not rescaled: give mean and std in the 0-255 scale for a uint8 image.

Parameters:

data (Tensor) – image or video with shape (B, C, *).
mean (Tensor) – mean of each channel: a number, a 0-d tensor, or a tensor with shape (C,), (1, C) for all samples or (B, C) for one per sample. A list raises.
std (Tensor) – standard deviation of each channel, in the same forms.

Return type:

Tensor

Returns:

Normalized tensor with the shape of data.

See also

denormalize(): the inverse.
normalize_min_max(): scales each channel plane to a value range instead of using fixed statistics.
Normalize: the module form.

Examples

>>> x = torch.rand(1, 4, 3, 3)
>>> out = normalize(x, torch.tensor([0.0]), torch.tensor([255.]))
>>> out.shape
torch.Size([1, 4, 3, 3])

>>> x = torch.tensor([[[[0, 255]]]], dtype=torch.uint8)
>>> normalize(x, 127.5, 127.5)
tensor([[[[-1.,  1.]]]])

>>> x = torch.rand(1, 4, 3, 3)
>>> mean = torch.zeros(4)
>>> std = 255. * torch.ones(4)
>>> out = normalize(x, mean, std)
>>> out.shape
torch.Size([1, 4, 3, 3])

kornia.enhance.normalize_min_max(input, min_val=0.0, max_val=1.0, eps=1e-6)[source]#

Rescale each channel plane of an image linearly to a value range.

Try in browser

out = normalize_min_max(input, min_val, max_val) maps the minimum of each H x W plane of input to min_val and its maximum to just below max_val, using that plane’s own range: (max_val - min_val) * (x - x.min()) / (x.max() - x.min() + eps) + min_val. A constant plane maps to min_val.

Convention:

Input is (*, C, H, W). Empty leading dimensions are preserved; channel and spatial dimensions must be nonzero.
Integer and boolean inputs are promoted to torch’s default floating dtype first, as in normalize(); floating inputs keep their dtype.

Parameters:

input (Tensor) – image with shape (*, C, H, W).
min_val (float, optional) – value of each plane’s minimum, a float. Default: 0.0
max_val (float, optional) – target of each plane’s maximum, a float. Default: 1.0
eps (float, optional) – value added to each plane’s range r = x.max() - x.min(), so the maximum lands at max_val - (max_val - min_val) * eps / (r + eps). Default: 1e-6

Return type:

Tensor

Returns:

Rescaled image with the shape of input.

See also

normalize(): subtracts a given mean and divides by a given standard deviation instead.

Example

>>> x = torch.rand(1, 5, 3, 3)
>>> x_norm = normalize_min_max(x, min_val=-1., max_val=1.)
>>> x_norm.min()
tensor(-1.)
>>> x_norm.max()
tensor(1.0000)

kornia.enhance.denormalize(data, mean, std)[source]#

Undo a per-channel mean and standard-deviation normalization of an image or video.

out = denormalize(data, mean, std) returns data * std + mean for each channel of data (B, C, *), which inverts normalize() for the same mean and std.

Convention:

mean and std take the forms of normalize(), are converted to the dtype and device of data, and a floating data keeps its dtype.
An integer or bool data is converted to torch’s default floating dtype first, so the result is floating and cannot wrap.

Parameters:

data (Tensor) – image or video with shape (B, C, *).
mean (Union[Tensor, float]) – mean of each channel: a number, a 0-d tensor, or a tensor with shape (C,), (1, C) or (B, C).
std (Union[Tensor, float]) – standard deviation of each channel, in the same forms.

Return type:

Tensor

Returns:

Denormalized tensor with the shape of data.

See also

normalize(): the inverse.
Denormalize: the module form.

Examples

>>> x = torch.rand(1, 4, 3, 3)
>>> out = denormalize(x, 0.0, 255.)
>>> out.shape
torch.Size([1, 4, 3, 3])

>>> x = torch.tensor([[[[0, 255]]]], dtype=torch.uint8)
>>> denormalize(x, 10., 2.)
tensor([[[[ 10., 520.]]]])

>>> x = torch.rand(1, 4, 3, 3, 3)
>>> mean = torch.zeros(1, 4)
>>> std = 255. * torch.ones(1, 4)
>>> out = denormalize(x, mean, std)
>>> out.shape
torch.Size([1, 4, 3, 3, 3])

kornia.enhance.zca_mean(inp, dim=0, unbiased=True, eps=1e-6, return_inverse=False)[source]#

Fit the ZCA whitening matrix and mean vector of a data set.

T, mean, T_inv = zca_mean(inp, dim) returns the ZCA matrix T with shape (D, D), the mean vector with shape (1, D) and, if return_inverse=True, the inverse matrix T_inv (else None). D is the product of the sizes of all axes except dim, the sample axis. Apply the result with linear_transform().

Convention:

See ZCAWhitening for the covariance and the transform.
The fit runs with autocast disabled (not under TorchScript). Float16 and bfloat16 inputs are fitted in float32 and return float32 tensors, to avoid cancellation when applying a rank-deficient fit.

Parameters:

inp (Tensor) – data tensor with shape (D_0, ..., D_dim, ..., D_N).
dim (int, optional) – sample axis. Default: 0
unbiased (bool, optional) – divide the covariance by \(N - 1\) if True, by \(N\) otherwise. Default: True
eps (float, optional) – value added to the eigenvalues before the inverse square root. Default: 1e-6
return_inverse (bool, optional) – also return the inverse transform. Default: False

Return type:

Tuple[Tensor, Tensor, Optional[Tensor]]

Returns:

Tuple (T, mean, T_inv).

Raises:

ValueError – If the sample dimension has fewer than two entries with unbiased=True, or none with unbiased=False.

See also

ZCAWhitening: stores the fit and applies it.
zca_whiten(): fits and whitens in one call.
linear_transform(): applies the result.

Examples

>>> x = torch.tensor([[0,1],[1,0],[-1,0],[0,-1]], dtype = torch.float32)
>>> transform_matrix, mean_vector,_ = zca_mean(x) # Returns transformation matrix and data mean
>>> x = torch.rand(3,20,2,2)
>>> transform_matrix, mean_vector, inv_transform = zca_mean(x, dim = 1, return_inverse = True)
>>> # transform_matrix.size() equals (12,12) and the mean vector.size equal (1,12)

kornia.enhance.zca_whiten(inp, dim=0, unbiased=True, eps=1e-6)[source]#

Whiten data with a ZCA transform fitted on the same data.

out = zca_whiten(inp, dim) fits ZCA on inp and returns inp centered and whitened, with its shape. To reuse the fit on other data, use ZCAWhitening.

Convention:

See ZCAWhitening for the sample axis and the covariance normalization.

Parameters:

inp (Tensor) – data tensor with shape (D_0, ..., D_dim, ..., D_N).
dim (int, optional) – sample axis. Default: 0
unbiased (bool, optional) – divide the covariance by \(N - 1\) if True, by \(N\) otherwise. Default: True
eps (float, optional) – value added to the eigenvalues before the inverse square root. Default: 1e-6

Return type:

Tensor

Returns:

Whitened tensor with the shape of inp.

Raises:

ValueError – If the sample dimension has fewer than two entries with unbiased=True, or none with unbiased=False.

See also

ZCAWhitening: fits once and whitens other data, and inverts the transform.
zca_mean(): returns the transform and mean without applying them.

Examples

>>> x = torch.tensor([[0,1],[1,0],[-1,0]], dtype = torch.float32)
>>> zca_whiten(x).round(decimals=3)
tensor([[ 0.0000,  1.1550],
        [ 1.0000, -0.5770],
        [-1.0000, -0.5770]])

kornia.enhance.linear_transform(inp, transform_matrix, mean_vector, dim=0)[source]#

Center data and multiply it by a transform matrix.

out = linear_transform(inp, transform_matrix, mean_vector, dim) flattens all axes of inp except the sample axis dim into one feature vector per sample, computes (X - mean_vector) @ transform_matrix and restores the shape of inp.

Convention:: Half inputs are centered and multiplied in float32 with autocast disabled (outside TorchScript), then rounded to the input dtype. Keep the fitted transform and mean in float32 to preserve their accuracy.

Parameters:

inp (Tensor) – data tensor with shape (D_0, ..., D_dim, ..., D_N).
transform_matrix (Tensor) – matrix with shape (D, D), D being the product of the sizes of all axes except dim.
mean_vector (Tensor) – mean with shape (1, D).
dim (int, optional) – sample axis. Default: 0

Return type:

Tensor

Returns:

Transformed tensor with the shape of inp.

See also

zca_mean(): computes the whitening matrix and mean to pass here.
ZCAWhitening: stores them and applies this transform.

Example

>>> # Example where dim = 3
>>> inp = torch.ones((10,3,4,5))
>>> transform_mat = torch.ones((10*3*4,10*3*4))
>>> mean = 2*torch.ones((1,10*3*4))
>>> out = linear_transform(inp, transform_mat, mean, 3)
>>> print(out.shape, out.unique())  # a (10,3,4,5) tensor of -120s
torch.Size([10, 3, 4, 5]) tensor([-120.])

>>> # Example where dim = 0
>>> inp = torch.ones((10,2))
>>> transform_mat = torch.ones((2,2))
>>> mean = torch.zeros((1,2))
>>> out = linear_transform(inp, transform_mat, mean)
>>> print(out.shape, out.unique()) # a (10,2) tensor of 2s
torch.Size([10, 2]) tensor([2.])

Modules#

class kornia.enhance.Normalize(mean, std)[source]#

Normalize an image with a stored per-channel mean and standard deviation (module form).

Try in browser

op = Normalize(mean, std) stores the statistics and out = op(input) is normalize() with them. A tensor statistic is stored as a non-persistent buffer, and an nn.Parameter stays a parameter that optimizers and checkpoints include.

Convention:

See the convention block of normalize().

Parameters:

mean (Union[Tensor, Tuple[float], List[float], float]) – mean of each channel, a number, a list, a tuple or a tensor.
std (Union[Tensor, Tuple[float], List[float], float]) – standard deviation of each channel, in the same forms.

Shape:

Input: image (B, C, *).
Output: same shape as the input.

See also

normalize(): the one-call form.
Denormalize: the inverse.
Normalize: the same operation inside an augmentation pipeline.

Examples

>>> x = torch.rand(1, 4, 3, 3)
>>> out = Normalize(0.0, 255.)(x)
>>> out.shape
torch.Size([1, 4, 3, 3])

>>> x = torch.rand(1, 4, 3, 3)
>>> mean = torch.zeros(4)
>>> std = 255. * torch.ones(4)
>>> out = Normalize(mean, std)(x)
>>> out.shape
torch.Size([1, 4, 3, 3])

class kornia.enhance.Denormalize(mean, std)[source]#

Undo a per-channel mean and standard-deviation normalization with stored statistics (module form).

op = Denormalize(mean, std) stores the statistics and out = op(input) is denormalize() with them. A tensor statistic is stored as a non-persistent buffer, and an nn.Parameter stays a parameter that optimizers and checkpoints include.

Convention:

See the convention block of denormalize().

Parameters:

mean (Union[Tensor, float]) – mean of each channel, a number, a list, a tuple or a tensor.
std (Union[Tensor, float]) – standard deviation of each channel, in the same forms.

Shape:

Input: image (B, C, *).
Output: same shape as the input.

See also

denormalize(): the one-call form.
Normalize: the inverse.
Denormalize: the same operation inside an augmentation pipeline.

Examples

>>> x = torch.rand(1, 4, 3, 3)
>>> out = Denormalize(0.0, 255.)(x)
>>> out.shape
torch.Size([1, 4, 3, 3])

>>> x = torch.rand(1, 4, 3, 3, 3)
>>> mean = torch.zeros(1, 4)
>>> std = 255. * torch.ones(1, 4)
>>> out = Denormalize(mean, std)(x)
>>> out.shape
torch.Size([1, 4, 3, 3, 3])

class kornia.enhance.ZCAWhitening(dim=0, eps=1e-6, unbiased=True, detach_transforms=True, compute_inv=False)[source]#

Whiten data with a ZCA transform fitted on a data set.

zca = ZCAWhitening(dim) is fitted with zca.fit(x), after which zca(x) returns x centered and whitened. dim is the sample axis and all other axes are flattened into one feature vector, so a (5, 3, 2, 2) tensor has \(N = 5\) samples of \(D = 12\) features.

Convention:

The mean \(\mathbf{\mu}\) and covariance \(\mathbf{\Sigma}\) come from the flattened data \(\mathbf{X} \in \mathbb{R}^{N \times D}\). With \(U\) the eigenvectors and \(S\) the eigenvalues of \(\Sigma\), the transform is

\[\mathbf{X}_{\text{zca}} = (\mathbf{X} - \mathbf{\mu})(U(S + \epsilon I)^{-1/2}U^T)^T\]
The output has the shape of the input. zca(x, include_fit=True) fits on x before transforming it.
Float16 and bfloat16 inputs are fitted and transformed in float32, and only the transformed output is rounded to the input dtype. The fitted buffers stay float32; casting the module to half precision discards that accuracy.
The fitted mean_vector, transform_matrix and transform_inv are buffers: state_dict saves them and .to() moves and casts them. Loading a fitted state into an unfitted module fits it, and the loaded tensors keep the device and dtype they were saved with, so move the module after loading. A checkpoint saved before the fitted state was persisted holds none of it, and loading it keeps the current fit.

Parameters:

dim (int, optional) – sample axis. Default: 0
eps (float, optional) – value added to the eigenvalues before the inverse square root. Default: 1e-6
unbiased (bool, optional) – divide the covariance by \(N - 1\) if True, by \(N\) otherwise. Default: True
detach_transforms (bool, optional) – detach the fitted tensors from the graph, so gradients do not flow through the fit. Default: True
compute_inv (bool, optional) – also compute the inverse transform that inverse_transform needs. Default: False

Shape:

x: tensor (D_0, ..., D_dim, ..., D_N) whose axis dim indexes the samples.
x_whiten: same shape as x.

Note

The fit takes an SVD of the covariance, whose backward step yields NaNs if the singular values are not unique, see torch.svd(). A working example is in the ZCA whitening tutorial.

See also

zca_whiten(): fits and whitens in one call.
zca_mean(): computes the transform and mean this module stores.
linear_transform(): applies a given transform and mean.

Examples

>>> x = torch.tensor([[0,1],[1,0],[-1,0],[0,-1]], dtype = torch.float32)
>>> zca = ZCAWhitening().fit(x)
>>> x_whiten = zca(x)
>>> zca = ZCAWhitening()
>>> x_whiten = zca(x, include_fit = True) # Includes the fitting step
>>> x_whiten = zca(x) # Can run now without the fitting set
>>> # Enable backprop through ZCA fitting process
>>> zca = ZCAWhitening(detach_transforms = False)
>>> x_whiten = zca(x, include_fit = True) # Includes the fitting step

References

[1] Stanford PCA & ZCA whitening tutorial

fit(x)[source]#

Fit ZCA whitening matrices to the data.

Parameters:: x (Tensor) – Input data.
Return type:: ZCAWhitening
Returns:: Returns a fitted ZCAWhitening object instance.
Raises:: ValueError – If the sample dimension has fewer than two entries with unbiased=True, or none with unbiased=False.

forward(x, include_fit=False)[source]#

Apply the whitening transform to the data.

Parameters:

x (Tensor) – Input data.
include_fit (bool, optional) – Indicates whether to fit the data as part of the forward pass. Default: False

Return type:

Tensor

Returns:

The transformed data.

inverse_transform(x)[source]#

Apply the inverse transform to the whitened data.

Uses the same sample dimension specified by dim when fitting.

Parameters:: x (Tensor) – Whitened data.
Return type:: Tensor
Returns:: Original data.