Descriptors#

class kornia.feature.DenseSIFTDescriptor(num_ang_bins=8, num_spatial_bins=4, spatial_bin_size=4, rootsift=True, clipval=0.2, stride=1, padding=1)[source]#

nn.Module, which computes SIFT descriptor densely over the image.

Parameters:

num_ang_bins (int, optional) – Number of angular bins. (8 is default) Default: 8
num_spatial_bins (int, optional) – Number of spatial bins per descriptor (4 is default). You might want to set an odd number and relevant padding to keep the feature map size. Default: 4
spatial_bin_size (int, optional) – Size of a spatial bin in pixels (4 is default) Default: 4
clipval (float, optional) – clipping value to reduce single-bin dominance Default: 0.2
rootsift (bool, optional) – (bool) if True, RootSIFT (Arandjelović et. al, 2012) is computed Default: True
stride (int, optional) – default 1 Default: 1
padding (int, optional) – default 0 Default: 1

Returns:

DenseSIFT descriptor of the image

Return type:

torch.Tensor

Shape:

Input: (B, 1, H, W)
Output: (B, num_ang_bins * num_spatial_bins ** 2, (H+padding)/stride, (W+padding)/stride)

Examples::

>>> input =  torch.rand(2, 1, 200, 300)
>>> SIFT = DenseSIFTDescriptor()
>>> descs = SIFT(input) # 2x128x194x294

class kornia.feature.SIFTDescriptor(patch_size=41, num_ang_bins=8, num_spatial_bins=4, rootsift=True, clipval=0.2)[source]#

nn.Module which computes SIFT descriptors of given patches.

Parameters:

patch_size (int, optional) – Input patch size in pixels. Default: 41
num_ang_bins (int, optional) – Number of angular bins. Default: 8
num_spatial_bins (int, optional) – Number of spatial bins. Default: 4
clipval (float, optional) – clipping value to reduce single-bin dominance Default: 0.2
rootsift (bool, optional) – if True, RootSIFT (Arandjelović et. al, 2012) is computed. Default: True

Returns:

SIFT descriptor of the patches.

Shape:

Input: \((B, 1, \text{patch_size}, \text{patch_size})\)
Output: \((B, \text{num_ang_bins * num_spatial_bins ** 2})\)

Example

>>> input = torch.rand(23, 1, 32, 32)
>>> SIFT = SIFTDescriptor(32, 8, 4)
>>> descs = SIFT(input) # 23x128

class kornia.feature.MKDDescriptor(patch_size=32, kernel_type='concat', whitening='pcawt', training_set='liberty', output_dims=128)[source]#

nn.Module that computes Multiple Kernel local descriptors.

This is based on the paper “Understanding and Improving Kernel Local Descriptors”. See [MTB+19] for more details.

Parameters:

patch_size (int, optional) – Input patch size in pixels. Default: 32
kernel_type (str, optional) – Parametrization of kernel 'concat', 'cart', 'polar'. Default: "concat"
whitening (str, optional) – Whitening transform to apply None, 'lw', 'pca', 'pcawt', 'pcaws'. Default: "pcawt"
training_set (str, optional) – Set that model was trained on 'liberty', 'notredame', 'yosemite'. Default: "liberty"
output_dims (int, optional) – Dimensionality reduction. Default: 128

Returns:

Explicit cartesian or polar embedding.

Shape:

Input: \((B, in_{dims}, fmap_{size}, fmap_{size})\).
Output: \((B, out_{dims}, fmap_{size}, fmap_{size})\),

Examples

>>> patches = torch.rand(23, 1, 32, 32)
>>> mkd = MKDDescriptor(patch_size=32,
...                     kernel_type='concat',
...                     whitening='pcawt',
...                     training_set='liberty',
...                     output_dims=128)
>>> desc = mkd(patches) # 23x128

class kornia.feature.HardNet(pretrained=False)[source]#

Module, which computes HardNet descriptors of given grayscale patches of 32x32.

This is based on the original code from paper “Working hard to know your neighbor’s margins: Local descriptor learning loss”. See [MMRM17] for more details.

Parameters:: pretrained (bool, optional) – Download and set pretrained weights to the model. Default: False
Returns:: HardNet descriptor of the patches.
Return type:: torch.Tensor

Shape:

Input: \((B, 1, 32, 32)\)
Output: \((B, 128)\)

Examples

>>> input = torch.rand(16, 1, 32, 32)
>>> hardnet = HardNet()
>>> descs = hardnet(input) # 16x128

class kornia.feature.HardNet8(pretrained=False)[source]#

Module, which computes HardNet8 descriptors of given grayscale patches of 32x32.

This is based on the original code from paper “Improving the HardNet Descriptor”. See [Pul20] for more details.

Parameters:: pretrained (bool, optional) – Download and set pretrained weights to the model. Default: False
Returns:: HardNet8 descriptor of the patches.
Return type:: torch.Tensor

Shape:

Input: \((B, 1, 32, 32)\)
Output: \((B, 128)\)

Examples

>>> input = torch.rand(16, 1, 32, 32)
>>> hardnet = HardNet8()
>>> descs = hardnet(input) # 16x128

class kornia.feature.HyNet(pretrained=False, is_bias=True, is_bias_FRN=True, dim_desc=128, drop_rate=0.3, eps_l2_norm=1e-10)[source]#

nn.Module, which computes HyNet descriptors of given grayscale patches of 32x32.

This is based on the original code from paper “HyNet: Learning Local Descriptor with Hybrid Similarity Measure and Triplet Loss”. See [TBLN+20] for more details.

Parameters:

pretrained (bool, optional) – Download and set pretrained weights to the model. Default: False
is_bias (bool, optional) – use bias in TLU layers Default: True
is_bias_FRN (bool, optional) – use bias in FRN layers Default: True
dim_desc (int, optional) – descriptor dimensionality, Default: 128
drop_rate (float, optional) – dropout rate, Default: 0.3
eps_l2_norm (float, optional) – to avoid div by zero Default: 1e-10

Returns:

HyNet descriptor of the patches.

Shape:

Input: \((B, 1, 32, 32)\)
Output: \((B, 128)\)

Examples

>>> input = torch.rand(16, 1, 32, 32)
>>> hynet = HyNet()
>>> descs = hynet(input) # 16x128

class kornia.feature.TFeat(pretrained=False)[source]#

Module, which computes TFeat descriptors of given grayscale patches of 32x32.

This is based on the original code from paper “Learning local feature descriptors with triplets and shallow convolutional neural networks”. See [BRPM16] for more details

Parameters:: pretrained (bool, optional) – Download and set pretrained weights to the model. Default: False
Returns:: TFeat descriptor of the patches.
Return type:: torch.Tensor

Shape:

Input: \((B, 1, 32, 32)\)
Output: \((B, 128)\)

Examples

>>> input = torch.rand(16, 1, 32, 32)
>>> tfeat = TFeat()
>>> descs = tfeat(input) # 16x128

class kornia.feature.SOSNet(pretrained=False)[source]#

128-dimensional SOSNet model definition for 32x32 patches.

This is based on the original code from paper “SOSNet:Second Order Similarity Regularization for Local Descriptor Learning”.

Parameters:: pretrained (bool, optional) – Download and set pretrained weights to the model. Default: False

Shape:

Input: \((B, 1, 32, 32)\)
Output: \((B, 128)\)

Examples

>>> input = torch.rand(8, 1, 32, 32)
>>> sosnet = SOSNet()
>>> descs = sosnet(input) # 8x128

class kornia.feature.LAFDescriptor(patch_descriptor_module=None, patch_size=32, grayscale_descriptor=True)[source]#

nn.Module to get local descriptors, corresponding to LAFs (keypoints).

Internally uses get_laf_descriptors().

Parameters:

patch_descriptor_module (Optional[Module], optional) – patch descriptor module, e.g. SIFTDescriptor or HardNet. Default: HardNet.
patch_size (int, optional) – patch size in pixels, which descriptor expects. Default: 32
grayscale_descriptor (bool, optional) – True if patch_descriptor expects single-channel image. Default: True

forward(img, lafs)[source]#

Three stage local feature detection.

First the location and scale of interest points are determined by detect function. Then affine shape and orientation.

Parameters:

img (Tensor) – image features with shape \((B,C,H,W)\).
lafs (Tensor) – local affine frames \((B,N,2,3)\).

Return type:

Tensor

Returns:

Local descriptors of shape \((B,N,D)\) where \(D\) is descriptor size.

class kornia.feature.SOLD2(pretrained=True, config=None)[source]#

nn.Module, which detects and describe line segments in an image.

This is based on the original code from the paper “SOLD²: Self-supervised Occlusion-aware Line Detector and Descriptor”. See [PautratLinL+21] for more details.

Parameters:

config (Optional[DetectorCfg], optional) – Dict specifying parameters. None will load the default parameters, which are tuned for images in the range 400~800 px. Default: None
pretrained (bool, optional) – If True, download and set pretrained weights to the model. Default: True

Returns:

The raw junction and line heatmaps, the semi-dense descriptor map, as well as the list of detected line segments (ij coordinates convention).

Example

>>> images = torch.rand(2, 1, 64, 64)
>>> sold2 = SOLD2()
>>> outputs = sold2(images)
>>> line_seg1 = outputs["line_segments"][0]
>>> line_seg2 = outputs["line_segments"][1]
>>> desc1 = outputs["dense_desc"][0]
>>> desc2 = outputs["dense_desc"][1]
>>> matches = sold2.match(line_seg1, line_seg2, desc1[None], desc2[None])

forward(img)[source]#

Run forward.

Parameters:: img (Tensor) – batched images with shape \((B, 1, H, W)\).
Returns:: list of N line segments in each of the B images \(List[(N, 2, 2)]\). junction_heatmap: raw junction heatmap of shape \((B, H, W)\). line_heatmap: raw line heatmap of shape \((B, H, W)\). dense_desc: the semi-dense descriptor map of shape \((B, 128, H/4, W/4)\).
Return type:: line_segments

kornia.feature.get_laf_descriptors(img, lafs, patch_descriptor, patch_size=32, grayscale_descriptor=True)[source]#

Get local descriptors, corresponding to LAFs (keypoints).

Parameters:

img (Tensor) – image features with shape \((B,C,H,W)\).
lafs (Tensor) – local affine frames \((B,N,2,3)\).
patch_descriptor (Module) – patch descriptor module, e.g. SIFTDescriptor or HardNet.
patch_size (int, optional) – patch size in pixels, which descriptor expects. Default: 32
grayscale_descriptor (bool, optional) – True if patch_descriptor expects single-channel image. Default: True

Return type:

Tensor

Returns:

Local descriptors of shape \((B,N,D)\) where \(D\) is descriptor size.