kornia.geometry.subpix#
Module with useful functionalities to extract coordinates sub-pixel accuracy.
Convolutional#
- kornia.geometry.subpix.conv_soft_argmax2d(input, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), temperature=tensor(1.0), normalized_coordinates=True, eps=1e-08, output_value=False)[source]#
Compute the convolutional spatial Soft-Argmax 2D over the windows of a given heatmap.
\[ij(X) = \frac{\sum{(i,j)} * exp(x / T) \in X} {\sum{exp(x / T) \in X}}\]\[val(X) = \frac{\sum{x * exp(x / T) \in X}} {\sum{exp(x / T) \in X}}\]where \(T\) is temperature.
- Parameters:
input – the given heatmap with shape \((N, C, H_{in}, W_{in})\).
kernel_size (optional) – the size of the window. Default:
(3, 3)
stride (optional) – the stride of the window. Default:
(1, 1)
padding (optional) – input zero padding. Default:
(1, 1)
temperature (optional) – factor to apply to input. Default:
tensor(1.0)
normalized_coordinates (optional) – whether to return the coordinates normalized in the range of \([-1, 1]\). Otherwise, it will return the coordinates in the range of the input shape. Default:
True
eps (optional) – small value to avoid zero division. Default:
1e-08
output_value (optional) – if True, val is output, if False, only ij. Default:
False
- Returns:
Function has two outputs - argmax coordinates and the softmaxpooled heatmap values themselves. On each window, the function computed returns with shapes \((N, C, 2, H_{out}, W_{out})\), \((N, C, H_{out}, W_{out})\),
where
\[H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\]\[W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\]
Examples
>>> input = torch.randn(20, 16, 50, 32) >>> nms_coords, nms_val = conv_soft_argmax2d(input, (3,3), (2,2), (1,1), output_value=True)
- kornia.geometry.subpix.conv_soft_argmax3d(input, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), temperature=tensor(1.0), normalized_coordinates=False, eps=1e-08, output_value=True, strict_maxima_bonus=0.0)[source]#
Compute the convolutional spatial Soft-Argmax 3D over the windows of a given heatmap.
\[ijk(X) = \frac{\sum{(i,j,k)} * exp(x / T) \in X} {\sum{exp(x / T) \in X}}\]\[val(X) = \frac{\sum{x * exp(x / T) \in X}} {\sum{exp(x / T) \in X}}\]where
T
is temperature.- Parameters:
input – the given heatmap with shape \((N, C, D_{in}, H_{in}, W_{in})\).
kernel_size (optional) – size of the window. Default:
(3, 3, 3)
stride (optional) – stride of the window. Default:
(1, 1, 1)
padding (optional) – input zero padding. Default:
(1, 1, 1)
temperature (optional) – factor to apply to input. Default:
tensor(1.0)
normalized_coordinates (optional) – whether to return the coordinates normalized in the range of :math:[-1, 1]`. Otherwise, it will return the coordinates in the range of the input shape. Default:
False
eps (optional) – small value to avoid zero division. Default:
1e-08
output_value (optional) – if True, val is output, if False, only ij. Default:
True
strict_maxima_bonus (optional) – pixels, which are strict maxima will score (1 + strict_maxima_bonus) * value. This is needed for mimic behavior of strict NMS in classic local features Default:
0.0
- Returns:
Function has two outputs - argmax coordinates and the softmaxpooled heatmap values themselves. On each window, the function computed returns with shapes \((N, C, 3, D_{out}, H_{out}, W_{out})\), \((N, C, D_{out}, H_{out}, W_{out})\),
where
\[D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\]\[H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\]\[W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor\]
Examples
>>> input = torch.randn(20, 16, 3, 50, 32) >>> nms_coords, nms_val = conv_soft_argmax3d(input, (3, 3, 3), (1, 2, 2), (0, 1, 1))
- kornia.geometry.subpix.conv_quad_interp3d(input, strict_maxima_bonus=10.0, eps=1e-07)[source]#
Compute the single iteration of quadratic interpolation of the extremum (max or min).
- Parameters:
input – the given heatmap with shape \((N, C, D_{in}, H_{in}, W_{in})\).
strict_maxima_bonus (optional) – pixels, which are strict maxima will score (1 + strict_maxima_bonus) * value. This is needed for mimic behavior of strict NMS in classic local features Default:
10.0
eps (optional) – parameter to control the hessian matrix ill-condition number. Default:
1e-07
- Returns:
the location and value per each 3x3x3 window which contains strict extremum, similar to one done is SIFT. \((N, C, 3, D_{out}, H_{out}, W_{out})\), \((N, C, D_{out}, H_{out}, W_{out})\),
where
\[D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\]\[H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\]\[W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor\]
Examples
>>> input = torch.randn(20, 16, 3, 50, 32) >>> nms_coords, nms_val = conv_quad_interp3d(input, 1.0)
Spatial#
- kornia.geometry.subpix.spatial_softmax2d(input, temperature=torch.tensor(1.0))[source]#
Apply the Softmax function over features in each image channel.
Note that this function behaves differently to
torch.nn.Softmax2d
, which instead applies Softmax over features at each spatial location.- Parameters:
- Return type:
- Returns:
a 2D probability distribution per image channel with shape \((B, N, H, W)\).
Examples
>>> heatmaps = torch.tensor([[[ ... [0., 0., 0.], ... [0., 0., 0.], ... [0., 1., 2.]]]]) >>> spatial_softmax2d(heatmaps) tensor([[[[0.0585, 0.0585, 0.0585], [0.0585, 0.0585, 0.0585], [0.0585, 0.1589, 0.4319]]]])
- kornia.geometry.subpix.spatial_expectation2d(input, normalized_coordinates=True)[source]#
Compute the expectation of coordinate values using spatial probabilities.
The input heatmap is assumed to represent a valid spatial probability distribution, which can be achieved using
spatial_softmax2d()
.- Parameters:
input (
Tensor
) – the input tensor representing dense spatial probabilities with shape \((B, N, H, W)\).normalized_coordinates (
bool
, optional) – whether to return the coordinates normalized in the range of \([-1, 1]\). Otherwise, it will return the coordinates in the range of the input shape. Default:True
- Return type:
- Returns:
expected value of the 2D coordinates with shape \((B, N, 2)\). Output order of the coordinates is (x, y).
Examples
>>> heatmaps = torch.tensor([[[ ... [0., 0., 0.], ... [0., 0., 0.], ... [0., 1., 0.]]]]) >>> spatial_expectation2d(heatmaps, False) tensor([[[1., 2.]]])
- kornia.geometry.subpix.spatial_soft_argmax2d(input, temperature=tensor(1.0), normalized_coordinates=True)[source]#
Compute the Spatial Soft-Argmax 2D of a given input heatmap.
- Parameters:
input (
Tensor
) – the given heatmap with shape \((B, N, H, W)\).temperature (
Tensor
, optional) – factor to apply to input. Default:tensor(1.0)
normalized_coordinates (
bool
, optional) – whether to return the coordinates normalized in the range of \([-1, 1]\). Otherwise, it will return the coordinates in the range of the input shape. Default:True
- Return type:
- Returns:
the index of the maximum 2d coordinates of the give map \((B, N, 2)\). The output order is x-coord and y-coord.
Examples
>>> input = torch.tensor([[[ ... [0., 0., 0.], ... [0., 10., 0.], ... [0., 0., 0.]]]]) >>> spatial_soft_argmax2d(input, normalized_coordinates=False) tensor([[[1.0000, 1.0000]]])
- kornia.geometry.subpix.render_gaussian2d(mean, std, size, normalized_coordinates=True)[source]#
Render the PDF of a 2D Gaussian distribution.
- Parameters:
mean – the mean location of the Gaussian to render, \((\mu_x, \mu_y)\). Shape: \((*, 2)\).
std – the standard deviation of the Gaussian to render, \((\sigma_x, \sigma_y)\). Shape \((*, 2)\). Should be able to be broadcast with mean.
size – the (height, width) of the output image.
normalized_coordinates (optional) – whether
mean
andstd
are assumed to use coordinates normalized in the range of \([-1, 1]\). Otherwise, coordinates are assumed to be in the range of the output shape. Default:True
- Returns:
tensor including rendered points with shape \((*, H, W)\).
Non Maxima Suppression#
- kornia.geometry.subpix.nms2d(input, kernel_size, mask_only=False)[source]#
Apply non maxima suppression to filter.
See
NonMaximaSuppression2d
for details.
Module#
- class kornia.geometry.subpix.SpatialSoftArgmax2d(temperature=tensor(1.0), normalized_coordinates=True)[source]#
Compute the Spatial Soft-Argmax 2D of a given heatmap.
See
spatial_soft_argmax2d()
for details.
- class kornia.geometry.subpix.ConvSoftArgmax2d(kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), temperature=tensor(1.0), normalized_coordinates=True, eps=1e-08, output_value=False)[source]#
Module that calculates soft argmax 2d per window.
See :func: ~kornia.geometry.subpix.conv_soft_argmax2d for details.
- class kornia.geometry.subpix.ConvSoftArgmax3d(kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), temperature=tensor(1.0), normalized_coordinates=False, eps=1e-08, output_value=True, strict_maxima_bonus=0.0)[source]#
Module that calculates soft argmax 3d per window.
See :func: ~kornia.geometry.subpix.conv_soft_argmax3d for details.
- class kornia.geometry.subpix.ConvQuadInterp3d(strict_maxima_bonus=10.0, eps=1e-07)[source]#
Calculate soft argmax 3d per window.
See :func: ~kornia.geometry.subpix.conv_quad_interp3d for details.