kornia.geometry.subpix#

Module with useful functionalities to extract coordinates sub-pixel accuracy.

Convolutional#

kornia.geometry.subpix.conv_soft_argmax2d(input, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), temperature=torch.tensor(1.0), normalized_coordinates=True, eps=1e-08, output_value=False)[source]#

Compute the convolutional spatial Soft-Argmax 2D over the windows of a given heatmap.

\[ij(X) = \frac{\sum{(i,j)} * exp(x / T) \in X} {\sum{exp(x / T) \in X}}\]

\[val(X) = \frac{\sum{x * exp(x / T) \in X}} {\sum{exp(x / T) \in X}}\]

where \(T\) is temperature.

Parameters

input (Tensor) – the given heatmap with shape \((N, C, H_{in}, W_{in})\).
kernel_size (Tuple[int, int], optional) – the size of the window. Default: (3, 3)
stride (Tuple[int, int], optional) – the stride of the window. Default: (1, 1)
padding (Tuple[int, int], optional) – input zero padding. Default: (1, 1)
temperature (Union[Tensor, float], optional) – factor to apply to input. Default: torch.tensor(1.0)
normalized_coordinates (bool, optional) – whether to return the coordinates normalized in the range of \([-1, 1]\). Otherwise, it will return the coordinates in the range of the input shape. Default: True
eps (float, optional) – small value to avoid zero division. Default: 1e-08
output_value (bool, optional) – if True, val is output, if False, only ij. Default: False

Return type

Union[Tensor, Tuple[Tensor, Tensor]]

Returns

Function has two outputs - argmax coordinates and the softmaxpooled heatmap values themselves. On each window, the function computed returns with shapes \((N, C, 2, H_{out}, W_{out})\), \((N, C, H_{out}, W_{out})\),

where

\[H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\]

\[W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\]

Examples

>>> input = torch.randn(20, 16, 50, 32)
>>> nms_coords, nms_val = conv_soft_argmax2d(input, (3,3), (2,2), (1,1), output_value=True)

kornia.geometry.subpix.conv_soft_argmax3d(input, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), temperature=torch.tensor(1.0), normalized_coordinates=False, eps=1e-08, output_value=True, strict_maxima_bonus=0.0)[source]#

Compute the convolutional spatial Soft-Argmax 3D over the windows of a given heatmap.

\[ijk(X) = \frac{\sum{(i,j,k)} * exp(x / T) \in X} {\sum{exp(x / T) \in X}}\]

\[val(X) = \frac{\sum{x * exp(x / T) \in X}} {\sum{exp(x / T) \in X}}\]

where T is temperature.

Parameters

input (Tensor) – the given heatmap with shape \((N, C, D_{in}, H_{in}, W_{in})\).
kernel_size (Tuple[int, int, int], optional) – size of the window. Default: (3, 3, 3)
stride (Tuple[int, int, int], optional) – stride of the window. Default: (1, 1, 1)
padding (Tuple[int, int, int], optional) – input zero padding. Default: (1, 1, 1)
temperature (Union[Tensor, float], optional) – factor to apply to input. Default: torch.tensor(1.0)
normalized_coordinates (bool, optional) – whether to return the coordinates normalized in the range of :math:[-1, 1]`. Otherwise, it will return the coordinates in the range of the input shape. Default: False
eps (float, optional) – small value to avoid zero division. Default: 1e-08
output_value (bool, optional) – if True, val is output, if False, only ij. Default: True
strict_maxima_bonus (float, optional) – pixels, which are strict maxima will score (1 + strict_maxima_bonus) * value. This is needed for mimic behavior of strict NMS in classic local features Default: 0.0

Return type

Union[Tensor, Tuple[Tensor, Tensor]]

Returns

Function has two outputs - argmax coordinates and the softmaxpooled heatmap values themselves. On each window, the function computed returns with shapes \((N, C, 3, D_{out}, H_{out}, W_{out})\), \((N, C, D_{out}, H_{out}, W_{out})\),

where

\[D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\]

\[H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\]

\[W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor\]

Examples

>>> input = torch.randn(20, 16, 3, 50, 32)
>>> nms_coords, nms_val = conv_soft_argmax3d(input, (3, 3, 3), (1, 2, 2), (0, 1, 1))

kornia.geometry.subpix.conv_quad_interp3d(input, strict_maxima_bonus=10.0, eps=1e-07)[source]#

Compute the single iteration of quadratic interpolation of the extremum (max or min).

Parameters

input (Tensor) – the given heatmap with shape \((N, C, D_{in}, H_{in}, W_{in})\).
strict_maxima_bonus (float, optional) – pixels, which are strict maxima will score (1 + strict_maxima_bonus) * value. This is needed for mimic behavior of strict NMS in classic local features Default: 10.0
eps (float, optional) – parameter to control the hessian matrix ill-condition number. Default: 1e-07

Return type

Tuple[Tensor, Tensor]

Returns

the location and value per each 3x3x3 window which contains strict extremum, similar to one done is SIFT. \((N, C, 3, D_{out}, H_{out}, W_{out})\), \((N, C, D_{out}, H_{out}, W_{out})\),

where

\[D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\]

\[H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\]

\[W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor\]

Examples

>>> input = torch.randn(20, 16, 3, 50, 32)
>>> nms_coords, nms_val = conv_quad_interp3d(input, 1.0)

Spatial#

kornia.geometry.subpix.spatial_softmax2d(input, temperature=torch.tensor(1.0))[source]#

Apply the Softmax function over features in each image channel.

Note that this function behaves differently to torch.nn.Softmax2d, which instead applies Softmax over features at each spatial location.

Parameters

input (Tensor) – the input tensor with shape \((B, N, H, W)\).
temperature (Tensor, optional) – factor to apply to input, adjusting the “smoothness” of the output distribution. Default: torch.tensor(1.0)

Return type

Tensor

Returns

a 2D probability distribution per image channel with shape \((B, N, H, W)\).

Examples

>>> heatmaps = torch.tensor([[[
... [0., 0., 0.],
... [0., 0., 0.],
... [0., 1., 2.]]]])
>>> spatial_softmax2d(heatmaps)
tensor([[[[0.0585, 0.0585, 0.0585],
          [0.0585, 0.0585, 0.0585],
          [0.0585, 0.1589, 0.4319]]]])

kornia.geometry.subpix.spatial_expectation2d(input, normalized_coordinates=True)[source]#

Compute the expectation of coordinate values using spatial probabilities.

The input heatmap is assumed to represent a valid spatial probability distribution, which can be achieved using spatial_softmax2d().

Parameters

input (Tensor) – the input tensor representing dense spatial probabilities with shape \((B, N, H, W)\).
normalized_coordinates (bool, optional) – whether to return the coordinates normalized in the range of \([-1, 1]\). Otherwise, it will return the coordinates in the range of the input shape. Default: True

Return type

Tensor

Returns

expected value of the 2D coordinates with shape \((B, N, 2)\). Output order of the coordinates is (x, y).

Examples

>>> heatmaps = torch.tensor([[[
... [0., 0., 0.],
... [0., 0., 0.],
... [0., 1., 0.]]]])
>>> spatial_expectation2d(heatmaps, False)
tensor([[[1., 2.]]])

kornia.geometry.subpix.spatial_soft_argmax2d(input, temperature=torch.tensor(1.0), normalized_coordinates=True)[source]#

Compute the Spatial Soft-Argmax 2D of a given input heatmap.

Parameters

input (Tensor) – the given heatmap with shape \((B, N, H, W)\).
temperature (Tensor, optional) – factor to apply to input. Default: torch.tensor(1.0)
normalized_coordinates (bool, optional) – whether to return the coordinates normalized in the range of \([-1, 1]\). Otherwise, it will return the coordinates in the range of the input shape. Default: True

Return type

Tensor

Returns

the index of the maximum 2d coordinates of the give map \((B, N, 2)\). The output order is x-coord and y-coord.

Examples

>>> input = torch.tensor([[[
... [0., 0., 0.],
... [0., 10., 0.],
... [0., 0., 0.]]]])
>>> spatial_soft_argmax2d(input, normalized_coordinates=False)
tensor([[[1.0000, 1.0000]]])

kornia.geometry.subpix.render_gaussian2d(mean, std, size, normalized_coordinates=True)[source]#

Render the PDF of a 2D Gaussian distribution.

Parameters

mean (Tensor) – the mean location of the Gaussian to render, \((\mu_x, \mu_y)\). Shape: \((*, 2)\).
std (Tensor) – the standard deviation of the Gaussian to render, \((\sigma_x, \sigma_y)\). Shape \((*, 2)\). Should be able to be broadcast with mean.
size (Tuple[int, int]) – the (height, width) of the output image.
normalized_coordinates (bool, optional) – whether mean and std are assumed to use coordinates normalized in the range of \([-1, 1]\). Otherwise, coordinates are assumed to be in the range of the output shape. Default: True

Returns

tensor including rendered points with shape \((*, H, W)\).

Non Maxima Suppression#

kornia.geometry.subpix.nms2d(input, kernel_size, mask_only=False)[source]#

Apply non maxima suppression to filter.

See NonMaximaSuppression2d for details.

Return type: Tensor

kornia.geometry.subpix.nms3d(input, kernel_size, mask_only=False)[source]#

Apply non maxima suppression to filter.

See NonMaximaSuppression3d for details.

Return type: Tensor

Module#

class kornia.geometry.subpix.SpatialSoftArgmax2d(temperature=torch.tensor(1.0), normalized_coordinates=True)[source]#

Compute the Spatial Soft-Argmax 2D of a given heatmap.

See spatial_soft_argmax2d() for details.

class kornia.geometry.subpix.ConvSoftArgmax2d(kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), temperature=torch.tensor(1.0), normalized_coordinates=True, eps=1e-08, output_value=False)[source]#

Module that calculates soft argmax 2d per window.

See conv_soft_argmax2d() for details.

class kornia.geometry.subpix.ConvSoftArgmax3d(kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), temperature=torch.tensor(1.0), normalized_coordinates=False, eps=1e-08, output_value=True, strict_maxima_bonus=0.0)[source]#

Module that calculates soft argmax 3d per window.

See conv_soft_argmax3d() for details.

class kornia.geometry.subpix.ConvQuadInterp3d(strict_maxima_bonus=10.0, eps=1e-07)[source]#

Calculate soft argmax 3d per window

See conv_quad_interp3d() for details.

class kornia.geometry.subpix.NonMaximaSuppression2d(kernel_size)[source]#: Apply non maxima suppression to filter.

class kornia.geometry.subpix.NonMaximaSuppression3d(kernel_size)[source]#: Apply non maxima suppression to filter.