Layers and other models#

Layers#

class kornia.feature.FilterResponseNorm2d(num_features, eps=1e-6, is_bias=True, is_scale=True, is_eps_leanable=False)[source]#

Filter Response Normalization layer from ‘Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks’, see [SK20] for more details.

\[y = \gamma \frac{x}{\sqrt{\nu^2 + |\epsilon|}} + \beta\]

Convention:

\(\nu^2\) is the mean of \(x^2\) over the H x W map of each sample and channel, so the output does not depend on the rest of the batch; no mean is subtracted. \(\gamma\) (weight) and \(\beta\) (bias) are per-channel parameters initialised to 1 and 0, applied only when is_scale and is_bias are set.

Parameters:

num_features (int) – number of channels
eps (float, optional) – normalization constant Default: 1e-6
is_bias (bool, optional) – use bias Default: True
is_scale (bool, optional) – use scale Default: True
is_eps_leanable (bool, optional) – if eps is learnable Default: False

Returns:

Normalized features

Return type:

torch.Tensor

Shape:

Input: \((B, \text{num_features}, H, W)\)
Output: \((B, \text{num_features}, H, W)\)

class kornia.feature.TLU(num_features)[source]#

TLU layer from ‘Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks’, see [SK20] for more details. \({\tau}\) is learnable per channel.

\[y = \max(x, {\tau})\]

Convention:

\(\tau\) (tau) is a (1, num_features, 1, 1) parameter, one threshold per channel, initialised to -1.

Parameters:: num_features (int) – number of channels
Returns:: torch.Tensor

Shape:

Input: \((B, \text{num_features}, H, W)\)
Output: \((B, \text{num_features}, H, W)\)

Fast moving objects#

class kornia.feature.DeFMO(pretrained=False)[source]#

nn.Module that disentangle a fast-moving object from the background and performs deblurring.

This is based on the original code from paper “DeFMO: Deblurring and Shape Recovery

of Fast Moving Objects”. See [ROF+21] for more details.

Convention:

The input is \((B, 6, H, W)\): the RGB channels of the blurred image, then those of the background estimate, in \([0, 1]\) for the pretrained weights. H and W that are multiples of 16 give an output of the same size; other sizes give an output rounded up to the next multiples of 16.
The output is \((B, 24, 4, H, W)\): 24 sub-frames in time order, sub-frame k rendered at the time k / 23, each with red, green, blue and alpha channels in \([0, 1]\) after a sigmoid.

Parameters:: pretrained (bool, optional) – Download and set pretrained weights to the model. Default: false.
Returns:: Temporal super-resolution without background.

Shape:

Input: (B, 6, H, W)
Output: (B, S, 4, H, W), with H and W rounded up to multiples of 16

Examples

>>> import kornia
>>> input = torch.rand(2, 6, 240, 320)
>>> defmo = kornia.feature.DeFMO()
>>> tsr_nobgr = defmo(input) # 2x24x4x240x320

forward(input_data)[source]#

Deblur a fast-moving object into a sequence of RGBA sub-frames.

Parameters:: input_data (Tensor) – Tensor with shape \((B, 6, H, W)\) containing the blurred RGB image concatenated with an RGB background estimate.
Return type:: Tensor
Returns:: Tensor with shape \((B, T, 4, H, W)\), where T is the number of temporal sub-frames and 4 stores red, green, blue, and alpha channels.