kornia.geometry.subpix#

Module with useful functionalities to extract coordinates sub-pixel accuracy.

Convolutional#

kornia.geometry.subpix.conv_soft_argmax2d(input, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), temperature=tensor(1.0), normalized_coordinates=True, eps=1e-8, output_value=False)[source]#

Compute the convolutional spatial Soft-Argmax 2D over the windows of a given heatmap.

\[ij(X) = \frac{\sum{(i,j)} * exp(x / T) \in X} {\sum{exp(x / T) \in X}}\]
\[val(X) = \frac{\sum{x * exp(x / T) \in X}} {\sum{exp(x / T) \in X}}\]

where \(T\) is temperature.

Parameters:

  • input – the given heatmap with shape \((N, C, H_{in}, W_{in})\).

  • kernel_size (optional) – the size of the window. Default: (3, 3)

  • stride (optional) – the stride of the window. Default: (1, 1)

  • padding (optional) – input zero padding. Default: (1, 1)

  • temperature (optional) – factor to apply to input. Default: tensor(1.0)

  • normalized_coordinates (optional) – whether to return the coordinates normalized in the range of \([-1, 1]\). Otherwise, it will return the coordinates in the range of the input shape. Default: True

  • eps (optional) – small value to avoid zero division. Default: 1e-8

  • output_value (optional) – if True, val is output, if False, only ij. Default: False

Returns:

Function has two outputs - argmax coordinates and the softmaxpooled heatmap values themselves. On each window, the function computed returns with shapes \((N, C, 2, H_{out}, W_{out})\), \((N, C, H_{out}, W_{out})\),

where

\[H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\]
\[W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\]

Examples

>>> input = torch.randn(20, 16, 50, 32)
>>> nms_coords, nms_val = conv_soft_argmax2d(input, (3,3), (2,2), (1,1), output_value=True)
kornia.geometry.subpix.conv_soft_argmax3d(input, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), temperature=tensor(1.0), normalized_coordinates=False, eps=1e-8, output_value=True, strict_maxima_bonus=0.0)[source]#

Compute the convolutional spatial Soft-Argmax 3D over the windows of a given heatmap.

\[ijk(X) = \frac{\sum{(i,j,k)} * exp(x / T) \in X} {\sum{exp(x / T) \in X}}\]
\[val(X) = \frac{\sum{x * exp(x / T) \in X}} {\sum{exp(x / T) \in X}}\]

where T is temperature.

Parameters:

  • input – the given heatmap with shape \((N, C, D_{in}, H_{in}, W_{in})\).

  • kernel_size (optional) – size of the window. Default: (3, 3, 3)

  • stride (optional) – stride of the window. Default: (1, 1, 1)

  • padding (optional) – input zero padding. Default: (1, 1, 1)

  • temperature (optional) – factor to apply to input. Default: tensor(1.0)

  • normalized_coordinates (optional) – whether to return the coordinates normalized in the range of :math:[-1, 1]`. Otherwise, it will return the coordinates in the range of the input shape. Default: False

  • eps (optional) – small value to avoid zero division. Default: 1e-8

  • output_value (optional) – if True, val is output, if False, only ij. Default: True

  • strict_maxima_bonus (optional) – pixels, which are strict maxima will score (1 + strict_maxima_bonus) * value. This is needed for mimic behavior of strict NMS in classic local features Default: 0.0

Returns:

Function has two outputs - argmax coordinates and the softmaxpooled heatmap values themselves. On each window, the function computed returns with shapes \((N, C, 3, D_{out}, H_{out}, W_{out})\), \((N, C, D_{out}, H_{out}, W_{out})\),

where

\[D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\]
\[H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\]
\[W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor\]

Examples

>>> input = torch.randn(20, 16, 3, 50, 32)
>>> nms_coords, nms_val = conv_soft_argmax3d(input, (3, 3, 3), (1, 2, 2), (0, 1, 1))
kornia.geometry.subpix.conv_quad_interp3d(input, strict_maxima_bonus=10.0, eps=1e-7)[source]#

Compute the single iteration of quadratic interpolation of the extremum (max or min).

Parameters:

  • input – the given heatmap with shape \((N, C, D_{in}, H_{in}, W_{in})\).

  • strict_maxima_bonus (optional) – pixels, which are strict maxima will score (1 + strict_maxima_bonus) * value. This is needed for mimic behavior of strict NMS in classic local features Default: 10.0

  • eps (optional) – parameter to control the hessian matrix ill-condition number. Default: 1e-7

Returns:

the location and value per each 3x3x3 window which contains strict extremum, similar to one done is SIFT. \((N, C, 3, D_{out}, H_{out}, W_{out})\), \((N, C, D_{out}, H_{out}, W_{out})\),

where

\[D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\]
\[H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\]
\[W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor\]

Examples

>>> input = torch.randn(20, 16, 3, 50, 32)
>>> nms_coords, nms_val = conv_quad_interp3d(input, 1.0)

Spatial#

kornia.geometry.subpix.spatial_softmax2d(input, temperature=torch.tensor(1.0))[source]#

Apply the Softmax function over features in each image channel.

Note that this function behaves differently to torch.nn.Softmax2d, which instead applies Softmax over features at each spatial location.

Parameters:

  • input (Tensor) – the input tensor with shape \((B, N, H, W)\).

  • temperature (Tensor, optional) – factor to apply to input, adjusting the “smoothness” of the output distribution. Default: torch.tensor(1.0)

Return type:

Tensor

Returns:

a 2D probability distribution per image channel with shape \((B, N, H, W)\).

Examples

>>> heatmaps = torch.tensor([[[
... [0., 0., 0.],
... [0., 0., 0.],
... [0., 1., 2.]]]])
>>> spatial_softmax2d(heatmaps)
tensor([[[[0.0585, 0.0585, 0.0585],
          [0.0585, 0.0585, 0.0585],
          [0.0585, 0.1589, 0.4319]]]])
kornia.geometry.subpix.spatial_expectation2d(input, normalized_coordinates=True)[source]#

Compute the expectation of coordinate values using spatial probabilities.

The input heatmap is assumed to represent a valid spatial probability distribution, which can be achieved using spatial_softmax2d().

Parameters:

  • input (Tensor) – the input tensor representing dense spatial probabilities with shape \((B, N, H, W)\).

  • normalized_coordinates (bool, optional) – whether to return the coordinates normalized in the range of \([-1, 1]\). Otherwise, it will return the coordinates in the range of the input shape. Default: True

Return type:

Tensor

Returns:

expected value of the 2D coordinates with shape \((B, N, 2)\). Output order of the coordinates is (x, y).

Examples

>>> heatmaps = torch.tensor([[[
... [0., 0., 0.],
... [0., 0., 0.],
... [0., 1., 0.]]]])
>>> spatial_expectation2d(heatmaps, False)
tensor([[[1., 2.]]])
kornia.geometry.subpix.spatial_soft_argmax2d(input, temperature=tensor(1.0), normalized_coordinates=True)[source]#

Compute the Spatial Soft-Argmax 2D of a given input heatmap.

Parameters:

  • input (Tensor) – the given heatmap with shape \((B, N, H, W)\).

  • temperature (Tensor, optional) – factor to apply to input. Default: tensor(1.0)

  • normalized_coordinates (bool, optional) – whether to return the coordinates normalized in the range of \([-1, 1]\). Otherwise, it will return the coordinates in the range of the input shape. Default: True

Return type:

Tensor

Returns:

the index of the maximum 2d coordinates of the give map \((B, N, 2)\). The output order is x-coord and y-coord.

Examples

>>> input = torch.tensor([[[
... [0., 0., 0.],
... [0., 10., 0.],
... [0., 0., 0.]]]])
>>> spatial_soft_argmax2d(input, normalized_coordinates=False)
tensor([[[1.0000, 1.0000]]])
kornia.geometry.subpix.render_gaussian2d(mean, std, size, normalized_coordinates=True)[source]#

Render the PDF of a 2D Gaussian distribution.

Parameters:

  • mean – the mean location of the Gaussian to render, \((\mu_x, \mu_y)\). Shape: \((*, 2)\).

  • std – the standard deviation of the Gaussian to render, \((\sigma_x, \sigma_y)\). Shape \((*, 2)\). Should be able to be broadcast with mean.

  • size – the (height, width) of the output image.

  • normalized_coordinates (optional) – whether mean and std are assumed to use coordinates normalized in the range of \([-1, 1]\). Otherwise, coordinates are assumed to be in the range of the output shape. Default: True

Returns:

tensor including rendered points with shape \((*, H, W)\).

Non Maxima Suppression#

kornia.geometry.subpix.nms2d(input, kernel_size, mask_only=False)[source]#

Apply non maxima suppression to filter.

See NonMaximaSuppression2d for details.

kornia.geometry.subpix.nms3d(input, kernel_size, mask_only=False)[source]#

Apply non maxima suppression to filter.

See :class: ~kornia.feature.NonMaximaSuppression3d for details.

Module#

class kornia.geometry.subpix.SpatialSoftArgmax2d(temperature=tensor(1.0), normalized_coordinates=True)[source]#

Compute the Spatial Soft-Argmax 2D of a given heatmap.

See spatial_soft_argmax2d() for details.

class kornia.geometry.subpix.ConvSoftArgmax2d(kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), temperature=tensor(1.0), normalized_coordinates=True, eps=1e-8, output_value=False)[source]#

Module that calculates soft argmax 2d per window.

See :func: ~kornia.geometry.subpix.conv_soft_argmax2d for details.

class kornia.geometry.subpix.ConvSoftArgmax3d(kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), temperature=tensor(1.0), normalized_coordinates=False, eps=1e-8, output_value=True, strict_maxima_bonus=0.0)[source]#

Module that calculates soft argmax 3d per window.

See :func: ~kornia.geometry.subpix.conv_soft_argmax3d for details.

class kornia.geometry.subpix.ConvQuadInterp3d(strict_maxima_bonus=10.0, eps=1e-7)[source]#

Calculate soft argmax 3d per window.

See :func: ~kornia.geometry.subpix.conv_quad_interp3d for details.

class kornia.geometry.subpix.NonMaximaSuppression2d(kernel_size)[source]#

Apply non maxima suppression to filter.

Flag minima_are_also_good is useful, when you want to detect both maxima and minima, e.g. for DoG

class kornia.geometry.subpix.NonMaximaSuppression3d(kernel_size)[source]#

Apply non maxima suppression to filter.