Local features (detector and descriptor together)#

Choosing a SIFT pipeline#

For new scale-space SIFT pipelines, start with SIFTFeatureScaleSpace and explicitly select descriptor_backend="pyramid":

import kornia.feature as KF

feature = KF.SIFTFeatureScaleSpace(
    num_features=2048, descriptor_backend="pyramid", rootsift=True
).to(grayscale_image.device)
lafs, responses, descriptors = feature(grayscale_image)

grayscale_image is a floating-point tensor of shape (B, 1, H, W) with values in [0, 1]. Outputs are LAFs (B, N, 2, 3), responses (B, N) and descriptors (B, N, 128). N is the requested feature budget; fewer detections leave padded slots. Use laf_is_filled() to identify real detections before matching. rootsift=True selects RootSIFT normalization. Use descriptor_layout="opencv" when feeding descriptors to LightGlueMatcher with "sift".

Both feature classes default to descriptor_backend="patch". Backend performance depends on device, image size, feature count and batching; the recipe is not a claim that pyramid extraction is always faster.

SIFT pipeline choices#
Pipeline	Detection and extraction	When to use it
`SIFTFeatureScaleSpace(..., descriptor_backend="pyramid")`	Dedicated DoG scale-space detector; orientation and description share the detector’s Gaussian layers.	Primary recipe for new scale-space SIFT use.
`SIFTFeatureScaleSpace(..., descriptor_backend="patch")`	Generic `ScaleSpaceDetector`; orientation and description operate on resampled patches.	Existing pipelines, or workloads where measurements favor patch extraction.
`SIFTFeature(..., descriptor_backend="patch")`	Multiresolution, per-level DoG detection; patch orientation and description.	Existing multiresolution pipelines; this is an approximation to scale-space SIFT.
`SIFTFeature(..., descriptor_backend="pyramid")`	The same multiresolution detector; a separate image pyramid supplies smoothed gradients for orientation and pooled histogram maps for description.	Measured multiresolution workloads; see the validation below.

The two feature classes have different detector algorithms. SIFTFeature applies a single-scale DoG response independently at each image resolution, with 2-D non-maximum suppression. It does not select extrema across scale or refine positions and scales to subpixel precision. SIFTFeatureScaleSpace searches 3-D DoG scale-space extrema and refines their positions and scales. Neither preset implements classical SIFT’s contrast and Hessian edge rejection, and each assigns only one orientation per detection (or preserves upright frames when requested).

descriptor_backend is not an interchangeable descriptor switch. In SIFTFeature it changes orientation and description but keeps detection; in SIFTFeatureScaleSpace it also selects a different detector implementation. Consequently, the scale-space backends can return different detections as well as different descriptors.

The scale-space pyramid path ranks valid refined extrema by absolute response. Precise doubling and integer octave decimation keep pixel coordinates aligned. The converged integer Gaussian layer and refined continuous scale are retained separately. Gradients are computed once per used layer and shared between orientation and description; no pyramid is retained between calls. Sampling and Gaussian support differ from patch extraction, so descriptors are not numerically interchangeable. For this backend, compile_modules accepts scale_pyr and subpix (or True for both); sparse orientation and description run eagerly. The patch backend passes compile_modules to ScaleSpaceDetector.

Matching with LightGlue#

Use descriptor_layout="opencv" with LightGlueMatcher in "sift" mode. Pass the oriented LAFs returned by the feature, filter zero-LAF padding from both LAFs and descriptor rows with laf_is_filled(), and supply the original image sizes as hw1=(height0, width0) and hw2=(height1, width1). The matcher converts the LAF angle and scale internally; do not apply those conversions twice.

The public adapter uses one sixth of the LAF scale. For the scale-space presets’ 6-sigma frames this is the detection sigma. SIFTFeature uses a different measurement-region convention, so its multiresolution frames do not imply the same detection sigma. Descriptor-layout compatibility alone does not guarantee equal matching accuracy across the four pipelines. Evaluate the complete feature/matcher combination at the intended feature budget.

Validation and possible deprecation#

The automatic orientation used by SIFTFeature(descriptor_backend="pyramid") integrates a 19x19 support of smoothed image gradients before angular binning, without extracting image patches. Its acceptance criteria, tracked in #5748, include performance at 1024 features and Oxford accuracy at 2048 features. Promotion of this convenience combination requires accuracy matching the patch backend and a demonstrated runtime or memory advantage on a stated workload. Validation should compare with the patch backend at 1024 and 2048 features on IMC PhotoTourism pose estimation and Oxford homography estimation, reporting matcher settings, device and timing methodology.

If validation does not establish those benefits, the plan is to deprecate this convenience combination and remove it only after the API stability policy window: at least one minor release of warnings and a documented replacement. No deprecation warning or removal version is set. This plan concerns the combination, not the standalone SIFTDescriptorFromPyramid descriptor-only interface; see Choosing a SIFT descriptor interface.

API reference#

class kornia.feature.LocalFeature(detector, descriptor, scaling_coef=1.0)[source]#

nn.Module, which combines local feature detector and descriptor.

Convention:

forward runs the detector, multiplies the shape A of every LAF by scaling_coef (scale_laf()) and describes the scaled LAFs. It returns (lafs, responses, descs) with shapes \((B, N, 2, 3)\), \((B, N)\) and \((B, N, D)\): the scaled LAFs, in the frame of laf_from_center_scale_ori() (see Local features), the detector’s responses and one descriptor per LAF. scaling_coef leaves the centres and the responses unchanged.
N comes from the detector. The detectors of every preset (ScaleSpaceDetector, MultiResolutionDetector, KeyNetDetector and the dedicated detector of SIFTFeatureScaleSpace(descriptor_backend="pyramid")) return num_features slots for every image, not the largest count in the batch.
A slot that the detector did not fill holds a zero LAF and a zero response, and all such slots of an image share one descriptor: with LAFDescriptor, that of a patch sampled at the origin. LocalFeatureMatcher drops these slots before matching. Drop them yourself before any other matcher, including match_nn(), match_mnn(), match_fginn(), match_adalam() and LightGlueMatcher, since identical descriptors match each other at zero distance and a mutual test does not reject a pair of them: keep the slots where laf_is_filled() is true, descs[b][laf_is_filled(lafs)[b]] for image b, rather than testing the response (see the Convention block of laf_is_filled()). The ratio tests of match_snn() and match_smnn() do not replace this filtering: when the other image has exactly one padded slot, the padded slots of the two images can match each other at zero distance.

Parameters:

detector (Module) – the detection module.
descriptor (Module) – the descriptor module.
scaling_coef (float, optional) – the multiplier of the detector’s LAFs before the description, see the Convention block. An int is converted to float. Must be positive. Default: 1.0

forward(img, mask=None)[source]#

Run forward.

Parameters:

img (Tensor) – image to extract features with shape \((B,C,H,W)\).
mask (Optional[Tensor], optional) – a mask saying where a detection may be, shape \((B,1,H,W)\) with the spatial size of the image. It is forwarded to the detector unchanged; see the detector for its semantics. Default: None

Return type:

Tuple[Tensor, Tensor, Tensor]

Returns:

Detected local affine frames with shape \((B,N,2,3)\).
Response function values for corresponding lafs with shape \((B,N)\).
Local descriptors of shape \((B,N,D)\) where \(D\) is descriptor size.

The Convention block of LocalFeature gives the frame, N and the slots the detector did not fill, which have to be dropped before matching.

class kornia.feature.SOLD2_detector(pretrained=True, config=None)[source]#

nn.Module, which detects line segments in an image.

This is based on the original code from the paper “SOLD²: Self-supervised Occlusion-aware Line Detector and Descriptor”. See [PautratLinL+21] for more details.

Convention:

The input is a grayscale image \((B, 1, H, W)\), in \([0, 1]\) for the pretrained weights, with H and W multiples of 8; other sizes raise an error or give heatmaps of another size. junction_heatmap and line_heatmap are \((B, H, W)\).
line_segments holds one \((N, 2, 2)\) tensor per image: the two endpoints of each segment as (row, col) in the pixel frame of laf_from_center_scale_ori() (see Local features), so flip(-1) gives (x, y). The endpoints of a segment, and the segments by their first endpoint, are in raster order: the smaller row first or, on the same row, the smaller column, whatever the junction scores.
The endpoints are junction pixels. The default junction refinement (use_junction_refinement of LineDetectorCfg) moves each of them by at most 1 pixel per axis, in 0.25 pixel steps.

Parameters:

config (DetectorCfg) – Configuration object containing all parameters. None will load the default parameters, which are tuned for images in the range 400~800 px. Default: None
pretrained (bool) – If True, download and set pretrained weights to the model. Default: True

Returns:

The raw junction and line heatmaps, as well as the list of detected line segments (ij coordinates convention).

Example

>>> img = torch.rand(1, 1, 128, 128)
>>> sold2_detector = SOLD2_detector(pretrained=False)
>>> line_segments = sold2_detector(img)["line_segments"]

forward(img)[source]#

Run forward.

Parameters:: img (Tensor) – batched images with shape \((B, 1, H, W)\).
Returns:: list of N line segments in each of the B images \(List[(N, 2, 2)]\). junction_heatmap: raw junction heatmap of shape \((B, H, W)\). line_heatmap: raw line heatmap of shape \((B, H, W)\).
Return type:: line_segments

class kornia.feature.ALIKED(model_name='aliked-n16', max_num_keypoints=-1, detection_threshold=0.2, nms_radius=2)[source]#

ALIKED local feature detector and descriptor.

ALIKED (Adaptive Local Image KEypoint Detection) combines a multi-scale ResNet backbone with deformable descriptor sampling (SDDH) and a differentiable keypoint detector (DKD).

See [ZWC+23] for details.

Convention:

ALIKED() starts in evaluation mode, like from_pretrained() and the reference implementation. In training mode BatchNorm would normalise with the statistics of the batch, so a keypoint or descriptor would depend on the rest of the batch, every call would update the running statistics, and a single small image would raise. Call .train() to opt in explicitly; the methods then behave as torch’s training-mode BatchNorm does.
images are RGB in \([0, 1]\), the range of the pretrained weights, and the network applies no normalisation; a \((B, 1, H, W)\) image is repeated to three channels. Any H and W work: the image is replicate-padded around its centre to multiples of 32 and the maps are cropped back, so the padding does not move the keypoints.
forward() returns one ALIKEDFeatures per image. Its keypoints are sub-pixel (x, y) in the pixel frame of laf_from_center_scale_ori() (see Local features): a strict maximum of the score map in its 2 * nms_radius + 1 window (see the note), at least nms_radius pixels from the border, moved by a soft-argmax over that window. Its score is the score map read bilinearly there, and the descriptors are L2-normalised. The keypoints, and the LAFs of forward_laf(), are float32 for a float16 or bfloat16 model and otherwise in the image’s dtype; scores and descriptors keep the model’s dtype.
With detection_threshold > 0 (threshold mode), the maxima strictly above it are kept, at most max_num_keypoints of them, and at most 20000 for a non-positive max_num_keypoints. An image with no maximum above the threshold keeps the maxima above its own mean score instead, whatever the rest of the batch. With detection_threshold <= 0 and max_num_keypoints > 0 (top-k mode), the max_num_keypoints strongest maxima are kept, or all of them when there are fewer; with both <= 0, the maxima above the image’s mean score.
image_size holds the valid (W, H) of each image of a batch padded at the bottom and right to a common size: maxima closer than nms_radius pixels to that image’s last valid row or column are dropped.
forward_laf() returns padded tensors like LocalFeature, but N is the largest keypoint count in the batch and the responses are \((B, N, 1)\). Its mask multiplies the score map, and it takes no image_size.
Known defects: on CPU, float16 and bfloat16 score and feature maps go to torch’s grid_sample without an upcast: the half-precision kernel can return wrong values, NaN or overflow, and where torch has none the call raises (#5810).

Parameters:

model_name (str, optional) – backbone configuration, one of 'aliked-t16', 'aliked-n16', 'aliked-n16rot', 'aliked-n32'. Default: "aliked-n16"
max_num_keypoints (int, optional) – the keypoint budget, a cap on the number of maxima kept in threshold mode and in top-k mode; see the Convention block. Default: -1
detection_threshold (float, optional) – a positive value selects threshold mode and is the score a maximum must exceed; zero or a negative value selects top-k mode when max_num_keypoints is positive. See the Convention block. Default: 0.2
nms_radius (int, optional) – NMS radius (kernel size = 2 * nms_radius + 1). The suppression keeps strict local maxima only; see the note below. Default: 2

Note

Keypoints are the strict local maxima of the score map: nms2d() keeps a position only if its score is greater than every other score in its 2 * nms_radius + 1 window. This departs on purpose from the reference implementations (Shiaoming/ALIKED and LightGlue’s port). Their simple_nms also keeps tied maxima, and in two further rounds it adds the positions that become the maximum of their window once the windows around the maxima already kept are zeroed. Those positions lie on the flank of a stronger peak and are not local maxima of the score map. In threshold mode, when neither the keypoint budget nor the fallback below decides the count, kornia therefore returns a subset of the reference keypoints: with the pretrained aliked-n16 and the default threshold it returns up to 10 % fewer keypoints than the reference on the Oxford img1 images, and each one left out has a higher score in its window. Otherwise the two keypoint sets need not be nested, because the reference ranks those extra positions with the maxima: in top-k mode, where it also fills top_k slots with positions that are not maxima when the score map has fewer maxima, and when the budget cuts the count in threshold mode. The reference falls back to the mean score only when no image of the batch has a maximum above the threshold; kornia returns maxima only and decides the fallback for each image, so an image’s keypoints do not depend on the rest of the batch.

Example

>>> aliked = ALIKED.from_pretrained('aliked-n16')
>>> images = torch.rand(1, 3, 256, 256)
>>> features = aliked(images)

forward(images, image_size=None)[source]#

Detect and describe local features in a batch of images.

Parameters:

images (Tensor) – (B, 3, H, W) float images (can be grayscale (B, 1, H, W) — will be broadcast to 3 channels automatically).
image_size (Optional[Tensor], optional) – optional (B, 2) tensor of valid (W, H) for border masking when images are padded to a common size. Default: None

Return type:

list[ALIKEDFeatures]

Returns:

A list of ALIKEDFeatures of length B, one per image. Keypoints are in pixel coordinates [x, y].

forward_laf(img, mask=None, compute_affine=True)[source]#

Detect and describe local features, returning results in kornia LAF format.

With compute_affine=True the 2x2 block A of each LAF comes from the covariance of the soft-argmax weights in the keypoint’s 2 * nms_radius + 1 detection window, computed inside DKD without any additional network parameters: A @ A.T is that covariance, so the axes of the frame are its principal axes, and det A > 0 (the frame is never mirrored). The frame describes how sharply the score map peaks, not an image structure. Its scale (get_laf_scale()) is about a pixel or less with the pretrained aliked-n16. That is much smaller than the region the descriptor samples, so rescale the frames, for example with scale_laf(), before using them as a region size.

All per-image tensors are zero-padded along the keypoint dimension to the largest keypoint count in the batch, so that the outputs are proper batched tensors.

Parameters:

img (Tensor) – image to extract features with shape \((B,C,H,W)\).
mask (Optional[Tensor], optional) – optional spatial mask (B, 1, H, W) with values in [0, 1]; the score map is multiplied by this mask before keypoint detection so that features are suppressed in masked regions. Default: None
compute_affine (bool, optional) – if True (default), estimate the 2x2 affine shape of each LAF using torch.linalg.eigh on the soft-argmax covariance. Set to False to skip the eigendecomposition and return identity affines, which is faster and avoids the linalg call entirely (useful when only keypoint positions are needed). Default: True

Return type:

Tuple[Tensor, Tensor, Tensor]

Returns:

Detected local affine frames with shape \((B,N,2,3)\).
Response function values for corresponding LAFs with shape \((B,N,1)\).
Local descriptors of shape \((B,N,D)\).

classmethod from_pretrained(model_name='aliked-n16', max_num_keypoints=-1, detection_threshold=0.2, nms_radius=2, device=None)[source]#

Load a pretrained ALIKED model from the official checkpoint repository.

Parameters:

model_name (str, optional) – one of 'aliked-t16', 'aliked-n16', 'aliked-n16rot', 'aliked-n32'. Default: "aliked-n16"
max_num_keypoints (int, optional) – passed to ALIKED constructor. Default: -1
detection_threshold (float, optional) – passed to ALIKED constructor. Default: 0.2
nms_radius (int, optional) – passed to ALIKED constructor. Default: 2
device (Optional[device], optional) – target device; defaults to CPU. Default: None

Return type:

ALIKED

Returns:

Pretrained ALIKED in eval mode.

class kornia.feature.ALIKEDFeatures(keypoints, descriptors, keypoint_scores)[source]#

Keypoints, descriptors and scores detected by ALIKED for a single image.

Since ALIKED detects a varying number of keypoints per image, ALIKEDFeatures is not batched.

Convention:: The keypoints, descriptors and scores that ALIKED returns for one image; its Convention block gives their frame and how many there are.

Parameters:

keypoints (Tensor) – pixel coordinates (N, 2) as [x, y].
descriptors (Tensor) – L2-normalised descriptors (N, D).
keypoint_scores (Tensor) – detection confidence scores (N,).

property n: int#: Number of detected keypoints.

to(*args, **kwargs)[source]#

Move all tensors to a new device / dtype.

Return type:: ALIKEDFeatures

class kornia.feature.DeDoDe(detector_model='L', descriptor_model='G', amp_dtype=torch.float16)[source]#

nn.Module which detects and/or describes local features in an image using the DeDode method.

See [EBWF24] for details.

Convention:

forward(), detect() and describe() take RGB images in the range \([0, 1]\) and apply the ImageNet normalization themselves; pass apply_imagenet_normalization=False for images that are already normalized. pad_if_not_divisible zero-pads the normalized image at the bottom and right to multiples of 14, so the padding is the ImageNet mean colour, and removes it from the maps again.
forward() returns pixel i as i + 0.5: subtract 0.5 for the pixel frame of laf_from_center_scale_ori() (see Local features). detect() returns it normalized, as (2 i + 1) / (W, H) - 1 in \([-1, 1]\) (align_corners=False), the form describe() takes; (k + 1) * (W, H) / 2 - 0.5 converts such a k to kornia’s frame. A description is read at the centre of pixel i.
Each image gives its n best keypoints by score, in descending order: \(N = \min(n, H W)\), with no threshold and no padding. Keypoints, and the descriptions of forward() and of describe(images, keypoints), are float32; descriptions are not normalized. The dense map of describe(images) follows the network dtype: amp_dtype on CUDA, the dtype of the model on CPU and MPS.
amp_dtype is CUDA mixed precision in every component: on CUDA tensors the networks run under torch.autocast in amp_dtype (and the DINOv2 backbone of the G descriptor is cast to it). On CPU and MPS it has no effect, so the default argument runs there with float32 weights and images.

Parameters:

detector_model (Literal['L'], optional) – The detector model kind. Available options are: L. Default: "L"
descriptor_model (Literal['G', 'B'], optional) – The descriptor model kind. Available options are: G or B Default: "G"
amp_dtype (dtype, optional) – The dtype of the automatic mixed precision used on CUDA; it has no effect on other devices. Default: torch.float16

Example

>>> dedode = DeDoDe.from_pretrained(detector_weights="L-C4-v2", descriptor_weights="B-upright")
>>> images = torch.rand(1, 3, 256, 256)
>>> keypoints, scores = dedode.detect(images)
>>> descriptions = dedode.describe(images, keypoints = keypoints)
>>> keypoints, scores, features = dedode(images) # alternatively do both

describe(images, keypoints=None, apply_imagenet_normalization=True, pad_if_not_divisible=True, crop_h=None, crop_w=None)[source]#

Describe keypoints in the input images. If keypoints are not provided, returns the dense descriptors.

Note

This method unconditionally sets the model to eval mode via self.train(False) so that BatchNorm and Dropout behave deterministically. This is intentional: the descriptor is only used at inference time and its statistics must be frozen.

Parameters:

images (Tensor) – A torch.Tensor of shape \((B, 3, H, W)\) containing the RGB input images in the range \([0, 1]\).
keypoints (Optional[Tensor], optional) – An optional torch.Tensor of shape \((B, N, 2)\) containing the detected keypoints. Default: None
apply_imagenet_normalization (bool, optional) – Whether to apply ImageNet normalization to the input images. Pass False if they are already ImageNet-normalized. Default: True
pad_if_not_divisible (bool, optional) – Zero-pad the (normalised) image on the bottom and right so that H and W are multiples of 14; the pad is the ImageNet mean colour, as in forward() and detect(). Required when using the G descriptor backed by DINOv2 (patch size 14). The padding is removed from the descriptor map, so the map has the size of the input image. Default: True
crop_h (Optional[int], optional) – The height of the crop to be used for description. If None, the full image is used. Default: None
crop_w (Optional[int], optional) – The width of the crop to be used for description. If None, the full image is used. Default: None

Returns:

A torch.Tensor of shape \((B, N, DIM)\) containing the descriptions: of the detected keypoints. If the dense descriptors are requested, the shape is \((B, DIM, H, W)\).

Return type:

descriptions

detect(images, n=10000, apply_imagenet_normalization=True, pad_if_not_divisible=True, crop_h=None, crop_w=None)[source]#

Detect keypoints in the input images.

Note

This method unconditionally sets the model to eval mode via self.train(False) so that BatchNorm and Dropout behave deterministically. This is intentional: the detector is only used at inference time and its statistics must be frozen.

Parameters:

images (Tensor) – A torch.Tensor of shape \((B, 3, H, W)\) containing the RGB input images in the range \([0, 1]\).
n (Optional[int], optional) – The number of keypoints to detect. Default: 10000
apply_imagenet_normalization (bool, optional) – Whether to apply ImageNet normalization to the input images. Pass False if they are already ImageNet-normalized. Default: True
pad_if_not_divisible (bool, optional) – Zero-pad the (normalised) image on the bottom and right so that H and W are multiples of 14; the pad is the ImageNet mean colour, as in forward() and describe(). The padding is removed from the scoremap, so keypoints are never reported in it. Default: True
crop_h (Optional[int], optional) – The height of the crop to be used for detection. If None, the full image is used. Default: None
crop_w (Optional[int], optional) – The width of the crop to be used for detection. If None, the full image is used. Default: None

Returns:

A torch.Tensor of shape \((B, N, 2)\) containing the detected keypoints, normalized to the range \([-1, 1]\). scores: A torch.Tensor of shape \((B, N)\) containing the scores of the detected keypoints.

Return type:

keypoints

forward(images, n=10_000, apply_imagenet_normalization=True, pad_if_not_divisible=True)[source]#

Detect and describe keypoints in the input images.

Parameters:

images (Tensor) – A torch.Tensor of shape \((B, 3, H, W)\) containing the RGB input images in the range \([0, 1]\).
n (Optional[int], optional) – The number of keypoints to detect. Default: 10_000
apply_imagenet_normalization (bool, optional) – Whether to apply ImageNet normalization to the input images. Pass False if they are already ImageNet-normalized. Default: True
pad_if_not_divisible (bool, optional) – Zero-pad the (normalised) image on the bottom and right so that H and W are multiples of 14; the pad is the ImageNet mean colour. The padding is removed from the scoremap, so keypoints are never reported in it. Default: True

Returns:

A torch.Tensor of shape \((B, N, 2)\) containing the detected keypoints in the image range,: unlike .detect() function.

scores: A torch.Tensor of shape \((B, N)\) containing the scores of the detected keypoints.

descriptions: A torch.Tensor of shape \((B, N, DIM)\) containing the descriptions: of the detected keypoints. DIM is 256 for both B and G.

Return type:

keypoints

classmethod from_pretrained(detector_weights='L-C4-v2', descriptor_weights='G-upright', amp_dtype=torch.float16)[source]#

Load a pretrained model.

Parameters:

detector_weights (str, optional) – The weights to load for the detector. One of ‘L-upright’ (original paper, https://arxiv.org/abs/2308.08479), ‘L-C4’, ‘L-SO2’ (from steerers, better for rotations, https://arxiv.org/abs/2312.02152), ‘L-C4-v2’ (from dedode v2, better at rotations, less clustering, https://arxiv.org/abs/2404.08928). Default is ‘L-C4-v2’. Default: "L-C4-v2"
descriptor_weights (str, optional) – The weights to load for the descriptor. One of ‘B-upright’,’G-upright’ (original paper, https://arxiv.org/abs/2308.08479), ‘B-C4’, ‘B-SO2’, ‘G-C4’, ‘G-SO2’ (from steerers, better for rotations, https://arxiv.org/abs/2312.02152). Default is ‘G-upright’. Default: "G-upright"
amp_dtype (dtype, optional) – the dtype of the automatic mixed precision used on CUDA, one of torch.float16 or torch.float32. It has no effect on CPU or MPS, where every component runs in the dtype of the input and the model. Default is torch.float16. Default: torch.float16

Return type:

Module

Returns:

The pretrained model.

class kornia.feature.DISK(desc_dim=128, unet=None)[source]#

nn.Module which detects and describes local features in an image using the DISK method.

See [TFT20] for details.

Convention:

images are RGB, \((B, 3, H, W)\), which the U-Net sees as given, without normalisation. With the default U-Net, a common positive gain of the input, such as \([0, 255]\) instead of \([0, 1]\), changes the outputs only through the instance norms’ eps, which weighs more the lower the image’s contrast. At ordinary contrast the maps barely change, though a keypoint can still swap where scores nearly tie or at the n cut; at low contrast the keypoints and descriptors change substantially. With the depth weights and n=2048, the \([0, 1]\) and \([0, 255]\) versions of Oxford graf, wall and bark img1 differ in 0 to 2 keypoints, in 2 to 6 % of them with the intensities multiplied by 0.1, and in about 30 % multiplied by 0.01. H and W must be multiples of 16, unless pad_if_not_divisible=True zero-pads the bottom and right, in which case the maps are cropped back to the image before the suppression and no keypoint lies in the padding.
Each image gives a DISKFeatures. Keypoints are the integer (x, y) pixels of the strict heatmap maxima (see the note), in the pixel frame of laf_from_center_scale_ori() (see Local features), listed in raster order (row by row, then column by column) rather than by score, in float32 for a float16 or bfloat16 network and otherwise in the dtype of the network’s output. detection_scores are the heatmap values there, in the network’s dtype, and each descriptor is the dense descriptor at the keypoint’s pixel, L2-normalised; heatmap_and_dense_descriptors() returns the dense descriptors without normalisation.
score_threshold keeps the scores strictly above it. n is an upper bound on the maxima above score_threshold in each image; the n argument of forward() says which of them are kept when there are more.

Parameters:

desc_dim (int, optional) – The dimension of the descriptor. Default: 128
unet (Module | None, optional) – The U-Net to use. If None, a default U-Net is used. Kornia doesn’t provide the training code for DISK so this is only useful when using a custom checkpoint trained using the code released with the paper. The unet should take as input a torch.Tensor of shape \((B, C, H, W)\) and output a torch.Tensor of shape \((B, \mathrm{desc\_dim} + 1, H, W)\). Default: None

Note

Keypoints are the strict local maxima of the heatmap: nms2d() keeps a position only if its score is greater than every other score in its full window_size window. Tied maxima, and the positions near the border whose window runs off the image (see nms2d()), are never keypoints. This departs on purpose from the reference implementation (cvlab-epfl/disk). Its max-pool NMS clips the window at the border, so it judges a border position against part of its window. Of several equal maxima it keeps the one the max-pool reports first, which depends on an order PyTorch does not specify. With the pretrained depth weights on Oxford graf and boat img1, every keypoint of the reference that kornia leaves out lies in that border strip. The reference’s n keeps the scores strictly above the (n + 1)-th largest, which drops the weakest maximum, with every maximum tied with it, of an image with n maxima or fewer, and every maximum tied across the cut. kornia’s n is an upper bound, so such an image keeps every maximum, and a tie at the cut is filled in raster order.

Example

>>> disk = DISK.from_pretrained('depth')
>>> images = torch.rand(1, 3, 256, 256)
>>> features = disk(images)

forward(images, n=None, window_size=5, score_threshold=0.0, pad_if_not_divisible=False)[source]#

Detect features in an image, returning keypoint locations, descriptors and detection scores.

Parameters:

images (Tensor) – The image to detect features in. Shape \((B, 3, H, W)\).
n (int | None, optional) – The maximum number of keypoints per image, counted among the maxima above score_threshold. An image with n such maxima or fewer keeps all of them; with more, its n strongest, where maxima tied at the n-th score are taken in raster order. If None, all keypoints are returned. Default: None
window_size (int, optional) – The size of the non-maxima suppression window used to filter detections. Default: 5
score_threshold (float, optional) – Detections must score strictly above it to be returned. See DISKFeatures for details. Default: 0.0
pad_if_not_divisible (bool, optional) – if True, an input whose height or width is not a multiple of 16 is zero-padded at the bottom and right to the next multiple of 16. Default: False

Return type:

list[DISKFeatures]

Returns:

A list of length \(B\) containing the detected features.

classmethod from_pretrained(checkpoint='depth', device=None)[source]#

Load a pretrained model.

Depth model was trained using depth map supervision and is slightly more precise but biased to detect keypoints only where SfM depth is available. Epipolar model was trained using epipolar geometry supervision and is less precise but detects keypoints everywhere where they are matchable. The difference is especially pronounced on thin structures and on edges of objects.

Parameters:

checkpoint (str, optional) – The checkpoint to load. One of ‘depth’ or ‘epipolar’. Default: "depth"
device (device | None, optional) – The device to load the model to. Default: None

Return type:

DISK

Returns:

The pretrained model.

heatmap_and_dense_descriptors(images)[source]#

Return the heatmap and the dense descriptors.

Parameters:: images (Tensor) – The image to detect features in. Shape \((B, 3, H, W)\).
Return type:: tuple[Tensor, Tensor]
Returns:: A tuple of dense detection scores and descriptors. Shapes are \((B, 1, H, W)\) and \((B, D, H, W)\), where \(D\) is the descriptor dimension.

class kornia.feature.XFeat(top_k=4096, detection_threshold=0.05)[source]#

XFeat sparse and semi-dense local feature extractor and matcher.

Try in browser

Wraps XFeatModel with NMS keypoint detection, descriptor interpolation, and mutual nearest-neighbour matching helpers.

Reference:: “XFeat: Accelerated Features for Lightweight Image Matching”, CVPR 2024. https://www.verlab.dcc.ufmg.br/descriptors/xfeat_cvpr24/

Convention:

XFeat defines no forward: call detectAndCompute() or detectAndComputeDense().
XFeat() starts in evaluation mode, like from_pretrained() and the reference implementation. The inference methods are run under torch.inference_mode; in training mode BatchNorm would normalise with the statistics of the batch, so a keypoint or match would depend on the rest of the batch, update its running statistics on every call, and raise on a single match. Call .train() to opt in explicitly; the methods then behave as torch’s training-mode BatchNorm does.
The image is \((B, C, H, W)\) with any channel count. Each extraction resizes the image it gets bilinearly to \((H', W')\): H and W rounded down to multiples of 32, with a minimum of 32. XFeatModel then averages the channels and instance-normalises the result, so an affine change of the intensities with a positive gain changes its maps only through the norm’s eps, which weighs more the lower the image’s contrast. With the pretrained weights and top_k=2048, the \([0, 1]\) and \([0, 255]\) versions of Oxford graf and wall img1 differ in at most 1 keypoint, in 1 to 2 % of them with the intensities multiplied by 0.1, and in 35 to 40 % multiplied by 0.01. Every output is float32.
detectAndCompute() keypoints are integer pixels i of the resized image, strict maxima of the keypoint heatmap in their 5 x 5 window (see the note), reported at i * (W / W', H / H'). That is the pixel frame of laf_from_center_scale_ori() (see Local features) when H and W are multiples of 32; otherwise the centre of the resized pixel is at (i + 0.5) * (W / W', H / H') - 0.5.
The heatmap value must be strictly above detection_threshold. scores is that value times the reliability map, and the top_k best keypoints come sorted by it. Each descriptor is 64-D, sampled bicubically from the dense map at the keypoint by InterpolateSparse2d, and L2-normalised.
With multiscale=False, detectAndComputeDense() reports cell (j, i) of the 1/8-resolution map at 8 * (j, i), times the same ratio: the top-left pixel of the cell, not its centre. It keeps the top_k cells of highest reliability, or every cell when there are fewer or top_k is below 1. With the default multiscale=True, it rescales the image bilinearly by 0.6 and by 1.3, runs that extraction on each with int(0.2 * top_k) and the remaining top_k - int(0.2 * top_k) cells, and divides each keypoint by its factor; scales is 1 / 0.6 or 1 / 1.3 for each keypoint. A share of 0 yields no cell from its scale, and a scale with fewer cells than its share returns all of them, so there are never more than top_k keypoints, and exactly top_k when both scales have enough cells. A top_k below 1 keeps every cell of both scales. Its descriptors are not normalised. The reference implementation gives the 1.3 scale int(0.8 * top_k) and reads a share of 0 as “every cell”; kornia does not, because that returns more keypoints than top_k asks for.
Known defects: converting the model to float16 or bfloat16 makes every extraction raise, because _preprocess_tensor passes float32 input to the half-precision weights (#5957).

Parameters:

top_k (int, optional) – maximum number of keypoints to keep per image. Default: 4096.
detection_threshold (float, optional) – the keypoint heatmap value a keypoint must exceed, see the Convention block. Default: 0.05.

Note

Keypoints are the strict local maxima of the keypoint heatmap: nms2d() keeps a position only if its score is greater than every other score in its full 5x5 window. Tied maxima, and the positions within 2 pixels of the border whose window runs off the image, are never keypoints. This departs on purpose from the reference implementation (verlab/accelerated_features). Its NMS keeps every position equal to the maximum of its window: all positions of a plateau survive, and whether a near-tie survives depends on rounding-level differences of the scores. Its window is also clipped at the border, so it judges a border position against part of its window. With the pretrained weights on Oxford graf and boat img1, every candidate of the reference that kornia leaves out lies in that border strip.

Example

>>> model = XFeat()
>>> img = torch.rand(1, 3, 256, 256)
>>> out = model.detectAndCompute(img)
>>> out[0]['keypoints'].shape
torch.Size([..., 2])

detectAndCompute(x, top_k=None, detection_threshold=None)[source]#

Detect sparse keypoints and compute descriptors.

Parameters:

x (Tensor) – image tensor of shape \((B, C, H, W)\).
top_k (Optional[int], optional) – number of keypoints to keep (overrides self.top_k). Default: None
detection_threshold (Optional[float], optional) – the heatmap value to exceed (overrides self.detection_threshold). Default: None

Returns:

'keypoints': \((N, 2)\) keypoints in (x, y) pixel coordinates.
'scores': \((N,)\) keypoint heatmap value times reliability, in descending order.
'descriptors': \((N, 64)\) L2-normalised descriptors.

Return type:

List of length B. Each element is a dict with

detectAndComputeDense(x, top_k=None, multiscale=True)[source]#

Detect keypoints and compute dense coarse descriptors.

Parameters:

x (Tensor) – image tensor of shape \((B, C, H, W)\).
top_k (Optional[int], optional) – number of features to keep (overrides self.top_k). A value below 1 keeps every cell. With multiscale=True the budget is split between the scales, int(0.2 * top_k) for the 0.6x extraction and the rest for the 1.3x one, so the result never exceeds a positive top_k; a scale with a budget of 0 contributes nothing. This departs from the reference implementation, which gives the 1.3x scale int(0.8 * top_k) and reads a budget that rounds down to 0 as “every cell”. Default: None
multiscale (bool, optional) – use dual-scale (0.6x and 1.3x) extraction. Default: True.

Returns:

'keypoints': \((B, K, 2)\) coarse keypoints.
'descriptors': \((B, K, 64)\) coarse descriptors.
'scales': \((B, K)\) extraction scale per keypoint.

Return type:

Dict with

classmethod from_pretrained(top_k=4096, detection_threshold=0.05)[source]#

Instantiate XFeat with pretrained weights downloaded from the official release.

Parameters:

top_k (int, optional) – maximum number of keypoints to keep. Default: 4096.
detection_threshold (float, optional) – the keypoint heatmap value a keypoint must exceed. Default: 0.05.

Return type:

XFeat

Returns:

XFeat model with pretrained weights loaded, set to eval mode.

match_xfeat(img1, img2, top_k=None, min_cossim=-1)[source]#

Detect, describe and mutually match keypoints from two images.

Parameters:

img1 (Tensor) – first image tensor of shape \((1, C, H, W)\).
img2 (Tensor) – second image tensor of shape \((1, C, H, W)\).
top_k (Optional[int], optional) – number of top keypoints to use. Default: None
min_cossim (float, optional) – minimum cosine similarity threshold. Use -1 to disable. Default: -1

Return type:

Tuple[Tensor, Tensor]

Returns:

Tuple (mkpts0, mkpts1) of matched keypoints, each \((N, 2)\).

match_xfeat_star(im_set1, im_set2, top_k=None)[source]#

Extract coarse features, match pairs and refine matches (XFeat*).

Parameters:

im_set1 (Tensor) – batch of images \((B, C, H, W)\).
im_set2 (Tensor) – batch of images \((B, C, H, W)\).
top_k (Optional[int], optional) – number of top features to use. Default: None

Returns:

list of \((N, 4)\) tensors with (x1, y1, x2, y2) matches. If B == 1: tuple of \((N, 2)\) matched keypoint tensors.

Return type:

If B > 1

class kornia.feature.XFeatModel[source]#

XFeat backbone: CNN feature extractor, keypoint and reliability heads.

Implements the architecture from “XFeat: Accelerated Features for Lightweight Image Matching”, CVPR 2024.

Input: float image tensor \((B, C, H, W)\) (grayscale or RGB, any channel count). Output:

feats: dense descriptors \((B, 64, H/8, W/8)\).
keypoints: keypoint logits \((B, 65, H/8, W/8)\).
heatmap: reliability map \((B, 1, H/8, W/8)\).

Convention:

The channels are averaged and the result is instance-normalised, so any channel count is accepted and an affine change a x + b of the intensities with a > 0 changes the outputs only through the norm’s eps (1e-5): negligibly while the variance of the channel mean is much larger than eps, and substantially for a low-contrast image, whose variance is close to it (see XFeat).
H and W must be multiples of 8; other sizes raise an error or give maps of mismatched sizes. XFeat resizes its input to multiples of 32.
keypoints holds 65 logits per 8 x 8 cell: channel (y % 8) * 8 + x % 8 of cell (y // 8, x // 8) is pixel (x, y), and channel 64 is the “no keypoint” dustbin.

Note

Image normalisation (InstanceNorm2d) is wrapped in torch.no_grad() following the original design; backpropagating through it is not supported.

forward(x)[source]#

Run the XFeat backbone.

Parameters:

x (Tensor) – image tensor of shape \((B, C, H, W)\).

Returns:

feats: dense descriptors \((B, 64, H/8, W/8)\).
keypoints: keypoint logits \((B, 65, H/8, W/8)\).
heatmap: reliability map \((B, 1, H/8, W/8)\).

Return type:

Tuple of

class kornia.feature.InterpolateSparse2d(mode='bicubic', align_corners=False)[source]#

Bilinearly or bicubically sample a dense feature map at sparse 2-D positions.

Convention:

pos holds (x, y) pixels of an image of height H and width W whose feature map has height h and width w. A position (px, py) is read at px * w / (W - 1) - 0.5 and py * h / (H - 1) - 0.5 in the map’s pixel frame: normgrid’s 2 p / (W - 1) - 1 sampled with align_corners=False, the convention the pretrained XFeat weights use. So a map of the image’s own size is not read at p itself.
Known defects: on CPU, float16 and bfloat16 maps go to torch’s grid_sample without an upcast: the half-precision kernel can return wrong values, NaN or overflow, and where torch has none the call raises (#5810).

Parameters:

mode (str, optional) – interpolation mode for torch.nn.functional.grid_sample(). Default: 'bicubic'.
align_corners (bool, optional) – stored and not used: grid_sample is always called with align_corners=False. Default: False.

Shape:

Input x: \((B, C, H, W)\).
Input pos: \((B, N, 2)\) integer or float (x, y) coordinates.
Output: \((B, N, C)\).

forward(x, pos, H, W)[source]#

Sample x at positions pos.

Parameters:

x (Tensor) – feature map \((B, C, H, W)\).
pos (Tensor) – sampling positions \((B, N, 2)\) in pixel coordinates.
H (int) – height used for coordinate normalisation.
W (int) – width used for coordinate normalisation.

Return type:

Tensor

Returns:

Sampled features \((B, N, C)\).

class kornia.feature.DISKFeatures(keypoints, descriptors, detection_scores)[source]#

A data structure holding DISK keypoints, descriptors and detection scores for an image.

Since DISK detects a varying number of keypoints per image, DISKFeatures is not batched.

Convention:: The keypoints, scores and descriptors that DISK returns for one image; its Convention block gives their frame and order.

Parameters:

keypoints (Tensor) – torch.Tensor of shape \((N, 2)\), where \(N\) is the number of keypoints.
descriptors (Tensor) – torch.Tensor of shape \((N, D)\), where \(D\) is the descriptor dimension.
detection_scores (Tensor) – torch.Tensor of shape \((N,)\), the detection heatmap value at each keypoint: a raw logit of the network, not a log-probability.

to(*args, **kwargs)[source]#

Call torch.Tensor.to() on each torch.tensor to move the keypoints, descriptors and detection scores to the specified device and/or data type.

Parameters:

*args (Any) – Arguments passed to torch.Tensor.to().
**kwargs (Any) – Keyword arguments passed to torch.Tensor.to().

Return type:

DISKFeatures

Returns:

A new DISKFeatures object with tensors of appropriate type and location.

property x: Tensor#: Accesses the x coordinates of keypoints (along image width).

property y: Tensor#: Accesses the y coordinates of keypoints (along image height).

class kornia.feature.SIFTFeatureScaleSpace(num_features=8000, upright=False, rootsift=True, device=None, compile_modules=False, descriptor_backend='patch', descriptor_layout='kornia')[source]#

DoG scale-space detection, position/scale refinement and (Root)SIFT description.

For new scale-space SIFT pipelines, explicitly select descriptor_backend="pyramid"; see Choosing a SIFT pipeline.

Convention:

A LocalFeature, whose Convention block gives the output contract, of grayscale images \((B, 1, H, W)\); the pyramid backend converts an RGB image to grayscale first. descriptor_backend="patch", the default, runs ScaleSpaceDetector with BlobDoG on ScalePyramid(3, 1.6, 32, double_image=True), keeping minima as well as maxima, then LAFOrienter(19) unless upright, and SIFTDescriptor on 41 x 41 patches. "pyramid" runs a dedicated DoG detector and reuses its Gaussian images and the octave and layer of each detection for orientation and description: it refines only sparse extrema, ranks all valid candidates by absolute response, and computes image gradients once per used layer.
Both backends search 3-D DoG extrema and refine position and scale. Neither applies classical SIFT’s contrast or Hessian edge rejection, and each assigns one orientation per detection unless upright=True. The backend selects a different detector as well as different orientation and description stages, so the backends can return different detections as well as different descriptors; the feature budget is the same.
On both backends the LAF scale is 6 times the detection sigma, and responses is the absolute DoG, the same for a bright blob as for a dark one. The LAF centres are in the frame of laf_from_center_scale_ori() (see Local features) on every octave. For a float16 or bfloat16 image the LAFs are float32 on both backends, and the responses and descriptors keep the image dtype.
Descriptors come in kornia’s SIFT layout; descriptor_layout="opencv" returns them in OpenCV’s, which LightGlueMatcher expects with "sift". convert_sift_descriptor_layout() converts between the two.

forward(img, mask=None)[source]#

Detect and describe sparse SIFT features, sharing scale space when selected.

Return type:: Tuple[Tensor, Tensor, Tensor]

class kornia.feature.SIFTFeature(num_features=8000, upright=False, rootsift=True, device=None, config=None, compile_model=False, score_threshold=0.0, descriptor_backend='patch', descriptor_layout='kornia')[source]#

Multiresolution DoG detection followed by a (Root)SIFT descriptor.

For new scale-space SIFT pipelines, start with SIFTFeatureScaleSpace and explicitly select descriptor_backend="pyramid"; see Choosing a SIFT pipeline.

Convention:

A LocalFeature, whose Convention block gives the output contract, of one grayscale image per call, \((1, 1, H, W)\). The detector is a MultiResolutionDetector on the magnitude of a single-scale BlobDoGSingle response. With descriptor_backend="patch", the default, LAFOrienter(19) orients the LAFs unless upright and SIFTDescriptor describes 41 x 41 patches. "pyramid" keeps the detector and orients and describes its LAFs with SIFTDescriptorFromPyramid, which integrates smoothed image gradients without extracting patches.
The detector applies the response independently at each image resolution, with 2-D non-maximum suppression. It does not select extrema across scale or refine positions and scales to subpixel precision, and it does not apply classical SIFT’s contrast or Hessian edge rejection. One orientation is assigned per detection unless upright=True.
Extrema of both polarities are kept: dark blobs are DoG maxima and bright blobs are DoG minima. On each image-pyramid level, a keypoint is a strict maximum of the response magnitude in its config["nms_size"] window (15 pixels by default), so a stronger extremum of either polarity suppresses a weaker one. responses holds the magnitude.
The frames are those of MultiResolutionDetector: a pixel of a pyramid level, with a scale of config["s_mult"] (22 by default) times that level’s pixel size whatever the size of the blob. Unlike the 6-sigma frames of SIFTFeatureScaleSpace, their scale is not a fixed multiple of the detection sigma.
Descriptors come in kornia’s SIFT layout; descriptor_layout="opencv" returns them in OpenCV’s, which LightGlueMatcher expects with "sift". convert_sift_descriptor_layout() converts between the two.
Known defects: descriptor_backend="pyramid" is less accurate than the patch backend on Oxford homographies at 2048 features, and not faster on CPU at 1024 features (#5748).

forward(img, mask=None)[source]#

Return sparse LAFs, detector responses, and one descriptor per LAF.

Return type:: Tuple[Tensor, Tensor, Tensor]

class kornia.feature.GFTTAffNetHardNet(num_features=8000, upright=False, device=None, compile_modules=False)[source]#

Convenience module, which implements GFTT detector + AffNet-HardNet descriptor.

Convention:: A LocalFeature, whose Convention block gives the output contract, of grayscale images \((B, 1, H, W)\): a ScaleSpaceDetector with the CornerGFTT response, its maxima only and mr_size=6, whose LAFs get their affine shape from LAFAffNetShapeEstimator, then their orientation from LAFOrienter(19) unless upright, and are described by HardNet on 32 x 32 patches.

forward(img, mask=None)[source]#

Run forward.

Parameters:

img (Tensor) – image to extract features with shape \((B,C,H,W)\).
mask (Optional[Tensor], optional) – a mask saying where a detection may be, shape \((B,1,H,W)\) with the spatial size of the image. It is forwarded to the detector unchanged; see the detector for its semantics. Default: None

Return type:

Tuple[Tensor, Tensor, Tensor]

Returns:

Detected local affine frames with shape \((B,N,2,3)\).
Response function values for corresponding lafs with shape \((B,N)\).
Local descriptors of shape \((B,N,D)\) where \(D\) is descriptor size.

The Convention block of LocalFeature gives the frame, N and the slots the detector did not fill, which have to be dropped before matching.

class kornia.feature.HesAffNetHardNet(num_features=2048, upright=False, device=None, compile_modules=False)[source]#

Convenience module, which implements Hessian detector + AffNet-HardNet descriptor.

Convention:: A LocalFeature, whose Convention block gives the output contract, of grayscale images \((B, 1, H, W)\): a ScaleSpaceDetector with the BlobHessian response, its maxima only and mr_size=6, whose LAFs get their affine shape from LAFAffNetShapeEstimator, then their orientation from LAFOrienter(19) unless upright, and are described by HardNet on 32 x 32 patches.

forward(img, mask=None)[source]#

Run forward.

Parameters:

img (Tensor) – image to extract features with shape \((B,C,H,W)\).
mask (Optional[Tensor], optional) – a mask saying where a detection may be, shape \((B,1,H,W)\) with the spatial size of the image. It is forwarded to the detector unchanged; see the detector for its semantics. Default: None

Return type:

Tuple[Tensor, Tensor, Tensor]

Returns:

Detected local affine frames with shape \((B,N,2,3)\).
Response function values for corresponding lafs with shape \((B,N)\).
Local descriptors of shape \((B,N,D)\) where \(D\) is descriptor size.

The Convention block of LocalFeature gives the frame, N and the slots the detector did not fill, which have to be dropped before matching.

class kornia.feature.DoGHardNet(num_features=2048, upright=False, device=None, compile_modules=False)[source]#

Convenience module, which implements DoG detector + HardNet descriptor.

Convention:

A LocalFeature, whose Convention block gives the output contract. The detector is the DoG ScaleSpaceDetector of SIFTFeatureScaleSpace’s "patch" backend, and the orientation is estimated on its 6-sigma frames. The returned LAFs are those frames scaled by two (scaling_coef=2), to 12 sigma or six times OpenCV’s KeyPoint.size, and HardNet describes them on 32 x 32 patches.
These are the frames the "doghardnet" weights of LightGlueMatcher expect: pass the LAFs and descriptors, without the padded slots, to LightGlueMatcher("doghardnet").
The image must have one channel, \((B, 1, H, W)\): convert a color image with rgb_to_grayscale() first.

Parameters:

num_features (int, optional) – the number of features to detect. Default: 2048
upright (bool, optional) – if True, skip the orientation estimation and return upright frames. Default: False
device (Union[str, device, None], optional) – the device of the detector and descriptor modules. Default: CPU.
compile_modules (Union[bool, List[str]], optional) – the detector sub-modules to wrap with torch.compile(), see ScaleSpaceDetector. Default: False

forward(img, mask=None)[source]#

Run forward.

Parameters:

img (Tensor) – image to extract features with shape \((B,C,H,W)\).
mask (Optional[Tensor], optional) – a mask saying where a detection may be, shape \((B,1,H,W)\) with the spatial size of the image. It is forwarded to the detector unchanged; see the detector for its semantics. Default: None

Return type:

Tuple[Tensor, Tensor, Tensor]

Returns:

Detected local affine frames with shape \((B,N,2,3)\).
Response function values for corresponding lafs with shape \((B,N)\).
Local descriptors of shape \((B,N,D)\) where \(D\) is descriptor size.

The Convention block of LocalFeature gives the frame, N and the slots the detector did not fill, which have to be dropped before matching.

class kornia.feature.DoGAffNetHardNet(num_features=2048, upright=False, device=None, compile_modules=False)[source]#

Convenience module, which implements DoG detector + AffNet-HardNet descriptor.

Convention:: As DoGHardNet, with the affine shape of each 6-sigma DoG frame estimated by LAFAffNetShapeEstimator before its orientation. The scale of the returned LAFs, the square root of their determinant, is again 12 sigma. Pass the LAFs and descriptors, without the padded slots, to LightGlueMatcher("dog_affnet_hardnet"). The image must have one channel, \((B, 1, H, W)\).

Parameters:

num_features (int, optional) – the number of features to detect. Default: 2048
upright (bool, optional) – if True, skip the orientation estimation and return upright frames. Default: False
device (Union[str, device, None], optional) – the device of the detector and descriptor modules. Default: CPU.
compile_modules (Union[bool, List[str]], optional) – the detector sub-modules to wrap with torch.compile(), see ScaleSpaceDetector. Default: False

forward(img, mask=None)[source]#

Run forward.

Parameters:

img (Tensor) – image to extract features with shape \((B,C,H,W)\).
mask (Optional[Tensor], optional) – a mask saying where a detection may be, shape \((B,1,H,W)\) with the spatial size of the image. It is forwarded to the detector unchanged; see the detector for its semantics. Default: None

Return type:

Tuple[Tensor, Tensor, Tensor]

Returns:

Detected local affine frames with shape \((B,N,2,3)\).
Response function values for corresponding lafs with shape \((B,N)\).
Local descriptors of shape \((B,N,D)\) where \(D\) is descriptor size.

The Convention block of LocalFeature gives the frame, N and the slots the detector did not fill, which have to be dropped before matching.

class kornia.feature.KeyNetAffNetHardNet(num_features=8000, upright=False, device=None, scale_laf=1.0, compile_model=False, score_threshold=0.0)[source]#

Convenience module, which implements KeyNet detector + AffNet + HardNet descriptor.

Convention:: As KeyNetHardNet, with the affine shape of each frame estimated by LAFAffNetShapeEstimator before its orientation.

forward(img, mask=None)[source]#

Run forward.

Parameters:

img (Tensor) – image to extract features with shape \((B,C,H,W)\).
mask (Optional[Tensor], optional) – a mask saying where a detection may be, shape \((B,1,H,W)\) with the spatial size of the image. It is forwarded to the detector unchanged; see the detector for its semantics. Default: None

Return type:

Tuple[Tensor, Tensor, Tensor]

Returns:

Detected local affine frames with shape \((B,N,2,3)\).
Response function values for corresponding lafs with shape \((B,N)\).
Local descriptors of shape \((B,N,D)\) where \(D\) is descriptor size.

The Convention block of LocalFeature gives the frame, N and the slots the detector did not fill, which have to be dropped before matching.

class kornia.feature.KeyNetHardNet(num_features=8000, upright=False, device=None, scale_laf=1.0, compile_model=False, score_threshold=0.0)[source]#

Convenience module, which implements KeyNet detector + HardNet descriptor.

Convention:: A LocalFeature, whose Convention block gives the output contract, of one grayscale image per call, \((1, 1, H, W)\): a KeyNetDetector, whose LAFs get their orientation from OriNet on 32 x 32 patches unless upright, and are described by HardNet on 32 x 32 patches. scale_laf is the scaling_coef of LocalFeature: it multiplies the detector’s LAFs before the description.

forward(img, mask=None)[source]#

Run forward.

Parameters:

img (Tensor) – image to extract features with shape \((B,C,H,W)\).
mask (Optional[Tensor], optional) – a mask saying where a detection may be, shape \((B,1,H,W)\) with the spatial size of the image. It is forwarded to the detector unchanged; see the detector for its semantics. Default: None

Return type:

Tuple[Tensor, Tensor, Tensor]

Returns:

Detected local affine frames with shape \((B,N,2,3)\).
Response function values for corresponding lafs with shape \((B,N)\).
Local descriptors of shape \((B,N,D)\) where \(D\) is descriptor size.

The Convention block of LocalFeature gives the frame, N and the slots the detector did not fill, which have to be dropped before matching.