# kornia.geometry.subpix¶

Module with useful functionalities to extract coordinates sub-pixel accuracy.

## Convolutional¶

conv_soft_argmax2d(input, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), temperature=tensor(1.0), normalized_coordinates=True, eps=1e-08, output_value=False)[source]

Computes the convolutional spatial Soft-Argmax 2D over the windows of a given heatmap.

$ij(X) = \frac{\sum{(i,j)} * exp(x / T) \in X} {\sum{exp(x / T) \in X}}$
$val(X) = \frac{\sum{x * exp(x / T) \in X}} {\sum{exp(x / T) \in X}}$

where $$T$$ is temperature.

Parameters
Return type
Returns

Function has two outputs - argmax coordinates and the softmaxpooled heatmap values themselves. On each window, the function computed returns with shapes $$(N, C, 2, H_{out}, W_{out})$$, $$(N, C, H_{out}, W_{out})$$,

where

$H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor$
$W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor$

Examples

>>> input = torch.randn(20, 16, 50, 32)
>>> nms_coords, nms_val = conv_soft_argmax2d(input, (3,3), (2,2), (1,1), output_value=True)

conv_soft_argmax3d(input, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), temperature=tensor(1.0), normalized_coordinates=False, eps=1e-08, output_value=True, strict_maxima_bonus=0.0)[source]

Computes the convolutional spatial Soft-Argmax 3D over the windows of a given heatmap.

$ijk(X) = \frac{\sum{(i,j,k)} * exp(x / T) \in X} {\sum{exp(x / T) \in X}}$
$val(X) = \frac{\sum{x * exp(x / T) \in X}} {\sum{exp(x / T) \in X}}$

where T is temperature.

Parameters
Return type
Returns

Function has two outputs - argmax coordinates and the softmaxpooled heatmap values themselves. On each window, the function computed returns with shapes $$(N, C, 3, D_{out}, H_{out}, W_{out})$$, $$(N, C, D_{out}, H_{out}, W_{out})$$,

where

$D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor$
$H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor$
$W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor$

Examples

>>> input = torch.randn(20, 16, 3, 50, 32)
>>> nms_coords, nms_val = conv_soft_argmax3d(input, (3, 3, 3), (1, 2, 2), (0, 1, 1))


Function that computes the single iteration of quadratic interpolation of the extremum (max or min).

Parameters
• input (Tensor) – the given heatmap with shape $$(N, C, D_{in}, H_{in}, W_{in})$$.

• strict_maxima_bonus (float, optional) – pixels, which are strict maxima will score (1 + strict_maxima_bonus) * value. This is needed for mimic behavior of strict NMS in classic local features Default: 10.0

• eps (float, optional) – parameter to control the hessian matrix ill-condition number. Default: 1e-07

Return type
Returns

the location and value per each 3x3x3 window which contains strict extremum, similar to one done is SIFT. $$(N, C, 3, D_{out}, H_{out}, W_{out})$$, $$(N, C, D_{out}, H_{out}, W_{out})$$,

where

$D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor$
$H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor$
$W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor$

Examples

>>> input = torch.randn(20, 16, 3, 50, 32)
>>> nms_coords, nms_val = conv_quad_interp3d(input, 1.0)


## Spatial¶

spatial_softmax2d(input, temperature=tensor(1.0))[source]

Applies the Softmax function over features in each image channel.

Note that this function behaves differently to torch.nn.Softmax2d, which instead applies Softmax over features at each spatial location.

Parameters
• input (Tensor) – the input tensor with shape $$(B, N, H, W)$$.

• temperature (Tensor, optional) – factor to apply to input, adjusting the “smoothness” of the output distribution. Default: tensor(1.)

Return type

Tensor

Returns

a 2D probability distribution per image channel with shape $$(B, N, H, W)$$.

Examples

>>> heatmaps = torch.tensor([[[
... [0., 0., 0.],
... [0., 0., 0.],
... [0., 1., 2.]]]])
>>> spatial_softmax2d(heatmaps)
tensor([[[[0.0585, 0.0585, 0.0585],
[0.0585, 0.0585, 0.0585],
[0.0585, 0.1589, 0.4319]]]])

spatial_expectation2d(input, normalized_coordinates=True)[source]

Computes the expectation of coordinate values using spatial probabilities.

The input heatmap is assumed to represent a valid spatial probability distribution, which can be achieved using spatial_softmax2d().

Parameters
• input (Tensor) – the input tensor representing dense spatial probabilities with shape $$(B, N, H, W)$$.

• normalized_coordinates (bool, optional) – whether to return the coordinates normalized in the range of $$[-1, 1]$$. Otherwise, it will return the coordinates in the range of the input shape. Default: True

Return type

Tensor

Returns

expected value of the 2D coordinates with shape $$(B, N, 2)$$. Output order of the coordinates is (x, y).

Examples

>>> heatmaps = torch.tensor([[[
... [0., 0., 0.],
... [0., 0., 0.],
... [0., 1., 0.]]]])
>>> spatial_expectation2d(heatmaps, False)
tensor([[[1., 2.]]])

spatial_soft_argmax2d(input, temperature=tensor(1.0), normalized_coordinates=True, eps=1e-08)[source]

Function that computes the Spatial Soft-Argmax 2D of a given input heatmap.

Parameters
• input (Tensor) – the given heatmap with shape $$(B, N, H, W)$$.

• temperature (Tensor, optional) – factor to apply to input. Default: tensor(1.)

• normalized_coordinates (bool, optional) – whether to return the coordinates normalized in the range of $$[-1, 1]$$. Otherwise, it will return the coordinates in the range of the input shape. Default: True

• eps (float, optional) – small value to avoid zero division. Default: 1e-08

Return type

Tensor

Returns

the index of the maximum 2d coordinates of the give map $$(B, N, 2)$$. The output order is x-coord and y-coord.

Examples

>>> input = torch.tensor([[[
... [0., 0., 0.],
... [0., 10., 0.],
... [0., 0., 0.]]]])
>>> spatial_soft_argmax2d(input, normalized_coordinates=False)
tensor([[[1.0000, 1.0000]]])

render_gaussian2d(mean, std, size, normalized_coordinates=True)[source]

Renders the PDF of a 2D Gaussian distribution.

Parameters
• mean (Tensor) – the mean location of the Gaussian to render, $$(\mu_x, \mu_y)$$. Shape: $$(*, 2)$$.

• std (Tensor) – the standard deviation of the Gaussian to render, $$(\sigma_x, \sigma_y)$$. Shape $$(*, 2)$$. Should be able to be broadcast with mean.

• size (Tuple[int, int]) – the (height, width) of the output image.

• normalized_coordinates (bool, optional) – whether mean and std are assumed to use coordinates normalized in the range of $$[-1, 1]$$. Otherwise, coordinates are assumed to be in the range of the output shape. Default: True

Returns

tensor including rendered points with shape $$(*, H, W)$$.

## Module¶

class SpatialSoftArgmax2d(temperature=tensor(1.0), normalized_coordinates=True, eps=1e-08)[source]

Module that computes the Spatial Soft-Argmax 2D of a given heatmap.

See spatial_soft_argmax2d() for details.

class ConvSoftArgmax2d(kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), temperature=tensor(1.0), normalized_coordinates=True, eps=1e-08, output_value=False)[source]

Module that calculates soft argmax 2d per window.

See conv_soft_argmax2d() for details.

class ConvSoftArgmax3d(kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), temperature=tensor(1.0), normalized_coordinates=False, eps=1e-08, output_value=True, strict_maxima_bonus=0.0)[source]

Module that calculates soft argmax 3d per window.

See conv_soft_argmax3d() for details.

See conv_quad_interp3d() for details.