Local affine frames (LAF)#

A local affine frame is a \((B, N, 2, 3)\) tensor describing, for each keypoint, an affine transformation from a canonical patch to the image. Most detectors and descriptors in kornia.feature exchange LAFs.

Functions#

kornia.feature.extract_patches_from_pyramid(img, laf, PS=32, normalize_lafs_before_extraction=True)[source]#

Extract patches defined by LAFs from image torch.Tensor.

Patches are extracted from the appropriate pyramid level. A LAF whose scale selects a level smaller than PS is sampled from the coarsest level that can still provide a full patch.

Parameters:

img (Tensor) – images, LAFs are detected in \((B, CH, H, W)\).
laf (Tensor) – \((B, N, 2, 3)\).
PS (int, optional) – patch size. Default: 32
normalize_lafs_before_extraction (bool, optional) – if True, lafs are normalized to image size. Default: True

Return type:

Tensor

Returns:

patches with shape \((B, N, CH, PS,PS)\).

kornia.feature.extract_patches_simple(img, laf, PS=32, normalize_lafs_before_extraction=True)[source]#

Extract patches defined by LAFs from image torch.Tensor.

No smoothing applied, huge aliasing (better use extract_patches_from_pyramid).

Parameters:

img (Tensor) – images, LAFs are detected in \((B, CH, H, W)\).
laf (Tensor) – \((B, N, 2, 3)\).
PS (int, optional) – patch size. Default: 32
normalize_lafs_before_extraction (bool, optional) – if True, lafs are normalized to image size. Default: True

Return type:

Tensor

Returns:

patches with shape \((B, N, CH, PS,PS)\).

kornia.feature.normalize_laf(LAF, images)[source]#

Normalize LAFs to [0,1] scale from pixel scale.

See below:: B,CH,H,W = images.size() MIN_SIZE = min(H - 1, W -1) [a11 a12 x] [a21 a22 y] becomes: [a11/MIN_SIZE a12/MIN_SIZE x/(W-1)] [a21/MIN_SIZE a22/MIN_SIZE y/(H-1)]

A singleton axis (H == 1 or W == 1) has no spatial extent and counts as one pixel, so the conversion stays finite instead of dividing by zero.

Parameters:

LAF (Tensor) – \((B, N, 2, 3)\)
images (Tensor) – \((B, CH, H, W)\)

Returns:

\((B, N, 2, 3)\), scale in image percentage (0, 1)

Return type:

the normalized LAF

kornia.feature.denormalize_laf(LAF, images)[source]#

Denormalize LAFs from the [0, 1] scale to image (pixel) scale.

The convention is that center of 5-pixel image (coordinates from 0 to 4) is 2, and not 2.5.

B,CH,H,W = images.size() MIN_SIZE = min(H - 1, W -1) [a11 a12 x] [a21 a22 y] becomes [a11*MIN_SIZE a12*MIN_SIZE x*(W-1)] [a21*MIN_SIZE a22*MIN_SIZE y*(H-1)]

A singleton axis (H == 1 or W == 1) has no spatial extent and counts as one pixel, so the conversion stays finite and round-trips with normalize_laf().

Parameters:

LAF (Tensor) – \((B, N, 2, 3)\)
images (Tensor) – \((B, CH, H, W)\)

Returns:

\((B, N, 2, 3)\), scale in pixels

Return type:

the denormalized LAF

kornia.feature.laf_to_boundary_points(LAF, n_pts=50)[source]#

Convert LAFs to boundary points of the regions + center.

Used for local features visualization, see visualize_laf function.

Parameters:

LAF (Tensor) – \((B, N, 2, 3)\)
n_pts (int, optional) – number of points to output. Default: 50

Returns:

\((B, N, n_pts, 2)\)

Return type:

torch.Tensor of boundary points LAF

kornia.feature.ellipse_to_laf(ells)[source]#

Convert ellipse regions to LAF format.

Ellipse (a, b, c) and upright covariance matrix [a11 a12; 0 a22] are connected by inverse matrix square root: A = invsqrt([a b; b c]).

See also vlfeat/vlfeat

Parameters:: ells (Tensor) – torch.Tensor \((B, N, 5)\) of ellipses in Oxford format [x y a b c].
Return type:: Tensor
Returns:: LAF \((B, N, 2, 3)\)

Note

A degenerate ellipse – one whose a or c is 0 after rounding to ells.dtype – describes an unbounded strip rather than a bounded region, and makes the matrix being inverted singular. Its LAF is non-finite: inf always appears on the diagonal, while nan appears only in the sub-case where the off-diagonal b is exactly 0 (0 * inf) – the generic degenerate ellipse is inf-only, so screen results with laf_is_valid() rather than an isnan test, which misses it. get_laf_scale() of such a LAF is non-finite as well. The conversion does not raise. Rounding is part of the condition: in float16 an a below roughly 3e-8 (half the smallest subnormal) rounds to 0, and a backend that flushes subnormals to zero raises that cutoff to the smallest normal, about 6e-5.

Example

>>> input = torch.ones(1, 10, 5)  # BxNx5
>>> output = ellipse_to_laf(input)  #  BxNx2x3

kornia.feature.make_upright(laf, eps=1e-9)[source]#

Rectify the affine matrix, so that it becomes upright.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
eps (float, optional) – for safe division. Default: 1e-9

Returns:

\((B, N, 2, 3)\)

Return type:

laf

Example

>>> input = torch.ones(1, 5, 2, 3)  # BxNx2x3
>>> output = make_upright(input)  #  BxNx2x3

kornia.feature.scale_laf(laf, scale_coef)[source]#

Multiplies region part of LAF ([:, :, :2, :2]) by a scale_coefficient.

So the center, shape and orientation of the local feature stays the same, but the region area changes.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
scale_coef (Union[float, Tensor]) – broadcastable torch.Tensor or float.

Return type:

Tensor

Returns:

LAF \((B, N, 2, 3)\)

Example

>>> input = torch.ones(1, 5, 2, 3)  # BxNx2x3
>>> scale = 0.5
>>> output = scale_laf(input, scale)  # BxNx2x3

kornia.feature.get_laf_scale(LAF)[source]#

Return a scale of the LAFs.

Parameters:: LAF (Tensor) – \((B, N, 2, 3)\)
Return type:: Tensor
Returns:: scale \((B, N, 1, 1)\)

Example

>>> input = torch.ones(1, 5, 2, 3)  # BxNx2x3
>>> output = get_laf_scale(input)  # BxNx1x1

kornia.feature.get_laf_center(LAF)[source]#

Return a center (keypoint) of the LAFs.

The convention is that center of 5-pixel image (coordinates from 0 to 4) is 2, and not 2.5.

Parameters:: LAF (Tensor) – \((B, N, 2, 3)\)
Return type:: Tensor
Returns:: xy \((B, N, 2)\)

Example

>>> input = torch.ones(1, 5, 2, 3)  # BxNx2x3
>>> output = get_laf_center(input)  # BxNx2

kornia.feature.rotate_laf(LAF, angles_degrees)[source]#

Apply additional rotation to the LAFs.

Compared to set_laf_orientation, the resulting rotation is original LAF orientation plus angles_degrees.

Parameters:

LAF (Tensor) – \((B, N, 2, 3)\)
angles_degrees (Tensor) – \((B, N, 1)\) in degrees.

Return type:

Tensor

Returns:

LAF oriented with angles \((B, N, 2, 3)\)

kornia.feature.get_laf_orientation(LAF)[source]#

Return orientation of the LAFs, in degrees.

Parameters:: LAF (Tensor) – \((B, N, 2, 3)\)
Return type:: Tensor
Returns:: angle in degrees \((B, N, 1)\)

Example

>>> input = torch.ones(1, 5, 2, 3)  # BxNx2x3
>>> output = get_laf_orientation(input)  # BxNx1

kornia.feature.set_laf_orientation(LAF, angles_degrees)[source]#

Change the orientation of the LAFs.

Parameters:

LAF (Tensor) – \((B, N, 2, 3)\)
angles_degrees (Tensor) – \((B, N, 1)\) in degrees.

Return type:

Tensor

Returns:

LAF oriented with angles \((B, N, 2, 3)\)

kornia.feature.laf_from_center_scale_ori(xy, scale=None, ori=None)[source]#

Create a LAF from keypoint center, scale and orientation.

Useful to create kornia LAFs from OpenCV keypoints.

Parameters:

xy (Tensor) – \((B, N, 2)\).
scale (Optional[Tensor], optional) – \((B, N, 1, 1)\). If not provided, scale = 1.0 is assumed Default: None
ori (Optional[Tensor], optional) – angle in degrees \((B, N, 1)\). If not provided orientation = 0 is assumed Default: None

Return type:

Tensor

Returns:

LAF \((B, N, 2, 3)\)

kornia.feature.laf_is_inside_image(laf, images, border=0)[source]#

Check if the LAF is touching or partly outside the image boundary.

Returns the mask of LAFs, which are fully inside the image, i.e. valid.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\).
images (Tensor) – images, lafs are detected in \((B, CH, H, W)\).
border (int, optional) – additional border. Default: 0

Return type:

Tensor

Returns:

mask with shape \((B, N)\).

kornia.feature.laf_is_valid(laf)[source]#

Check that each LAF is finite and has a finite, nonzero determinant.

Parameters:: laf (Tensor) – \((B, N, 2, 3)\).
Return type:: Tensor
Returns:: validity mask \((B, N)\).

Example

>>> laf = torch.eye(2, 3).view(1, 1, 2, 3)
>>> laf_is_valid(laf)
tensor([[True]])

kornia.feature.laf_to_three_points(laf)[source]#

Convert local affine frame(LAF) to alternative representation: coordinates of LAF center, LAF-x unit vector, LAF-y unit vector.

Parameters:: laf (Tensor) – \((B, N, 2, 3)\).
Return type:: Tensor
Returns:: threepts \((B, N, 2, 3)\).

kornia.feature.laf_from_three_points(threepts)[source]#

Convert three points to local affine frame.

Order is (0,0), (0, 1), (1, 0).

Parameters:: threepts (Tensor) – \((B, N, 2, 3)\).
Return type:: Tensor
Returns:: laf \((B, N, 2, 3)\).

kornia.feature.KORNIA_CHECK_LAF(laf, raises=True)[source]#

Check whether a Local Affine Frame (laf) has a valid shape.

Parameters:

laf (Tensor) – local affine frame tensor to evaluate.
raises (bool, optional) – bool indicating whether an exception should be raised upon failure. Default: True

Raises:

ShapeError – if the input laf does not have a shape \((B,N,2,3)\) and raises is True.

Return type:

bool

Note

Checks can be disabled in Python mode using disable_checks() or the KORNIA_CHECKS environment variable. In TorchScript-compiled code, checks always run (TorchScript cannot access module-level globals, but the validation logic is fast). When running with python -O, Python’s optimizer may eliminate some checks.

Example

>>> lafs = torch.rand(2, 10, 2, 3)
>>> KORNIA_CHECK_LAF(lafs)
True

kornia.feature.perspective_transform_lafs(trans_01, lafs_1)[source]#

Apply perspective transformations to a set of local affine frames (LAFs).

Parameters:

trans_01 (Tensor) – torch.Tensor for perspective transformations of shape \((B, 3, 3)\).
lafs_1 (Tensor) – torch.Tensor of lafs of shape \((B, N, 2, 3)\).

Return type:

Tensor

Returns:

torch.Tensor of N-dimensional points of shape \((B, N, 2, 3)\).

Examples

>>> rng = torch.manual_seed(0)
>>> lafs_1 = torch.rand(2, 4, 2, 3)  # BxNx2x3
>>> lafs_1
tensor([[[[0.4963, 0.7682, 0.0885],
          [0.1320, 0.3074, 0.6341]],

         [[0.4901, 0.8964, 0.4556],
          [0.6323, 0.3489, 0.4017]],

         [[0.0223, 0.1689, 0.2939],
          [0.5185, 0.6977, 0.8000]],

         [[0.1610, 0.2823, 0.6816],
          [0.9152, 0.3971, 0.8742]]],


        [[[0.4194, 0.5529, 0.9527],
          [0.0362, 0.1852, 0.3734]],

         [[0.3051, 0.9320, 0.1759],
          [0.2698, 0.1507, 0.0317]],

         [[0.2081, 0.9298, 0.7231],
          [0.7423, 0.5263, 0.2437]],

         [[0.5846, 0.0332, 0.1387],
          [0.2422, 0.8155, 0.7932]]]])
>>> trans_01 = torch.eye(3).repeat(2, 1, 1)  # Bx3x3
>>> trans_01.shape
torch.Size([2, 3, 3])
>>> lafs_0 = perspective_transform_lafs(trans_01, lafs_1)  # BxNx2x3

Orientation and affine shape estimation#

class kornia.feature.PassLAF(*args, **kwargs)[source]#

Dummy module to use instead of local feature orientation or affine shape estimator.

forward(laf, img)[source]#

Run forward.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
img (Tensor) – \((B, 1, H, W)\)

Return type:

Tensor

Returns:

LAF, unchanged \((B, N, 2, 3)\)

class kornia.feature.PatchAffineShapeEstimator(patch_size=19, eps=1e-10)[source]#

Module, which estimates the second moment matrix of the patch gradients.

The method determines the affine shape of the local feature as in [Baumberg00].

For float16 and bfloat16 inputs, the gradients, Gaussian weighting, moments, and normalization are computed in float32. The output keeps the input dtype.

Parameters:

patch_size (int, optional) – the input image patch size. Default: 19
eps (float, optional) – for safe division. Default: 1e-10

forward(patch)[source]#

Run forward.

Parameters:: patch (Tensor) – \((B, 1, H, W)\)
Returns:: ellipse_shape \((B, 1, 3)\)
Return type:: torch.Tensor

class kornia.feature.LAFAffineShapeEstimator(patch_size=32, affine_shape_detector=None, preserve_orientation=True)[source]#

Module, which extracts patches using input images and local affine frames (LAFs).

Then runs PatchAffineShapeEstimator on patches to estimate LAFs shape.

Then original LAF shape is replaced with estimated one. The original LAF orientation is not preserved, so it is recommended to first run LAFAffineShapeEstimator and then LAFOrienter,

Parameters:

patch_size (int, optional) – the input image patch size. Default: 32
affine_shape_detector (Optional[Module], optional) – Patch affine shape estimator, PatchAffineShapeEstimator. Default: None
preserve_orientation (bool, optional) – if True, the original orientation is preserved. Default: True

forward(laf, img)[source]#

Run forward.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
img (Tensor) – \((B, 1, H, W)\)

Returns:

\((B, N, 2, 3)\)

Return type:

LAF_out

class kornia.feature.LAFOrienter(patch_size=32, num_angular_bins=36, angle_detector=None)[source]#

Module, which extracts patches using input images and local affine frames (LAFs).

Then runs PatchDominantGradientOrientation or OriNet on patches and then rotates the LAFs by the estimated angles

Parameters:

patch_size (int, optional) – Default: 32
num_angular_bins (int, optional) – Default: 36
angle_detector (Optional[Module], optional) – Patch orientation estimator, e.g. PatchDominantGradientOrientation or OriNet. Default: None

forward(laf, img)[source]#

Run forward.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
img (Tensor) – \((B, 1, H, W)\)

Returns:

\((B, N, 2, 3)\)

Return type:

LAF_out

class kornia.feature.PatchDominantGradientOrientation(patch_size=32, num_angular_bins=36, eps=1e-8)[source]#

Module, which estimates the dominant gradient orientation of the given patches, in radians.

Zero angle points towards right.

Parameters:

patch_size (int, optional) – size of the (square) input patch. Default: 32
num_angular_bins (int, optional) – number of histogram bins. Default: 36
eps (float, optional) – for safe division, and arctan. Default: 1e-8

forward(patch)[source]#

Run forward.

Parameters:: patch (Tensor) – \((B, 1, H, W)\)
Returns:: \((B)\)
Return type:: angle in radians

class kornia.feature.OriNet(pretrained=False, eps=1e-8)[source]#

Network, which estimates the canonical orientation of the given 32x32 patches, in radians.

Zero angle points towards right. This is based on the original code from paper “Repeatability Is Not Enough: Learning Discriminative Affine Regions via Discriminability””. See [MRM18] for more details.

Parameters:

pretrained (bool, optional) – Download and set pretrained weights to the model. Default: False
eps (float, optional) – to avoid division by zero in atan2. Default: 1e-8

Returns:

Angle in radians.

Shape:

Input: (B, 1, 32, 32)
Output: (B)

Examples

>>> input = torch.rand(16, 1, 32, 32)
>>> orinet = OriNet()
>>> angle = orinet(input) # 16

forward(patch)[source]#

Run forward.

Parameters:: patch (Tensor) – \((B, 1, H, W)\)
Returns:: \((B)\)
Return type:: angle in radians

class kornia.feature.LAFAffNetShapeEstimator(pretrained=False, preserve_orientation=True)[source]#

Module, which extracts patches using input images and local affine frames (LAFs).

Then runs AffNet on patches to estimate LAFs shape. This is based on the original code from paper “Repeatability Is Not Enough: Learning Discriminative Affine Regions via Discriminability””. See [MRM18] for more details.

Then original LAF shape is replaced with estimated one. The original LAF orientation is not preserved, so it is recommended to first run LAFAffineShapeEstimator and then LAFOrienter.

Parameters:: pretrained (bool, optional) – Download and set pretrained weights to the model. Default: False

forward(laf, img)[source]#

Run forward.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
img (Tensor) – \((B, 1, H, W)\)

Returns:

\((B, N, 2, 3)\)

Return type:

LAF_out