Local affine frames (LAF)#

A local affine frame is a \((B, N, 2, 3)\) tensor describing, for each keypoint, an affine transformation from a canonical patch to the image. Most detectors and descriptors in kornia.feature exchange LAFs.

Functions#

kornia.feature.extract_patches_from_pyramid(img, laf, PS=32, normalize_lafs_before_extraction=True)[source]#

Extract patches defined by LAFs from image torch.Tensor.

Patches are extracted from the appropriate pyramid level.

Convention:

Cell (i, j) of a PS x PS patch samples the image at c + A @ (u_j, u_i), with u_k = (2 k + 1) / PS - 1 and the LAF [A | c] in pixels, pixel centres at integers. Row 0 is on the patch’s -y side, the top for an upright LAF, the columns run along the patch’s x axis, and the cells tile the square c + A [-1, 1]^2. For an odd PS, a LAF with A = (PS / 2) I centred on a pixel returns the image crop around that pixel.
The pyramid level is floor(log2(2 s / PS)) for the scale s = sqrt(|det A|), clamped to the levels that exist: pyrdown() builds them while the shorter side stays at least PS, so a larger LAF is sampled from the coarsest level that can still provide a full patch. Level 0, the image itself, always exists: an image whose shorter side is below PS has only that level, so even a large LAF is sampled from it. A level samples the same image positions as level 0, in its own pixels.
A cell outside the image reads the nearest border pixel of the level it samples.
The patches have the image’s dtype. The sampling runs in the promoted dtype of the image and the LAF, and in at least float32. A LAF with a non-finite entry gives a zero patch.

Parameters:

img (Tensor) – images, LAFs are detected in \((B, CH, H, W)\).
laf (Tensor) – \((B, N, 2, 3)\).
PS (int, optional) – patch size. Default: 32
normalize_lafs_before_extraction (bool, optional) – if True, lafs are normalized to image size; if False, laf must already be in the frame that normalize_laf() gives for img. Default: True

Return type:

Tensor

Returns:

patches with shape \((B, N, CH, PS,PS)\).

kornia.feature.extract_patches_simple(img, laf, PS=32, normalize_lafs_before_extraction=True)[source]#

Extract patches defined by LAFs from image torch.Tensor.

Convention:: See the Convention block on extract_patches_from_pyramid(). This function samples the same positions, always from img itself: a LAF larger than the patch is not smoothed first, so it aliases.

Parameters:

img (Tensor) – images, LAFs are detected in \((B, CH, H, W)\).
laf (Tensor) – \((B, N, 2, 3)\).
PS (int, optional) – patch size. Default: 32
normalize_lafs_before_extraction (bool, optional) – if True, lafs are normalized to image size; if False, laf must already be in the frame that normalize_laf() gives for img. Default: True

Return type:

Tensor

Returns:

patches with shape \((B, N, CH, PS,PS)\).

kornia.feature.normalize_laf(LAF, images)[source]#

Normalize LAFs to [0,1] scale from pixel scale.

See below:: B,CH,H,W = images.size() MIN_SIZE = min(H - 1, W -1) [a11 a12 x] [a21 a22 y] becomes: [a11/MIN_SIZE a12/MIN_SIZE x/(W-1)] [a21/MIN_SIZE a22/MIN_SIZE y/(H-1)]

A singleton axis (H == 1 or W == 1) has no spatial extent and counts as one pixel, so the conversion stays finite instead of dividing by zero.

Convention:

The centre becomes (x / max(W - 1, 1), y / max(H - 1, 1)), which lies in [0, 1] for 0 <= x <= W - 1 and 0 <= y <= H - 1. A is divided by the smaller of the two denominators as a whole, not per axis, and can exceed 1.
Taken to an image of another size with denormalize_laf(), x becomes x max(W' - 1, 1) / max(W - 1, 1), which maps corner to corner when neither width is 1. A resize with align_corners=False, the default of resize() in bilinear mode, maps pixel x to (x + 0.5) W' / W - 0.5 instead, so the two disagree away from the image centre.

Parameters:

LAF (Tensor) – \((B, N, 2, 3)\)
images (Tensor) – \((B, CH, H, W)\)

Returns:

\((B, N, 2, 3)\)

Return type:

the normalized LAF

kornia.feature.denormalize_laf(LAF, images)[source]#

Denormalize LAFs from the [0, 1] scale to image (pixel) scale.

B,CH,H,W = images.size() MIN_SIZE = min(H - 1, W -1) [a11 a12 x] [a21 a22 y] becomes [a11*MIN_SIZE a12*MIN_SIZE x*(W-1)] [a21*MIN_SIZE a22*MIN_SIZE y*(H-1)]

A singleton axis (H == 1 or W == 1) has no spatial extent and counts as one pixel, so the conversion stays finite and round-trips with normalize_laf().

Convention:: See the Convention block on normalize_laf(), which this inverts.

Parameters:

LAF (Tensor) – \((B, N, 2, 3)\)
images (Tensor) – \((B, CH, H, W)\)

Returns:

\((B, N, 2, 3)\), scale in pixels

Return type:

the denormalized LAF

kornia.feature.laf_to_boundary_points(LAF, n_pts=50)[source]#

Convert LAFs to boundary points of the regions + center.

Convention:

The first point is the centre c. The other n_pts - 1 points are c + A @ (sin t, cos t) for t from 0 to \(2 \pi\) in equal steps: they start and end at c + A[:, 1], the tip of the patch’s y axis, and so close the outline of the region.

Parameters:

LAF (Tensor) – \((B, N, 2, 3)\)
n_pts (int, optional) – number of points to output, the centre included. Default: 50

Returns:

\((B, N, n_pts, 2)\)

Return type:

torch.Tensor of boundary points LAF

kornia.feature.ellipse_to_laf(ells)[source]#

Convert ellipse regions to LAF format.

Convention:

An ellipse [x, y, a, b, c] is the region \(a u^2 + 2 b u v + c v^2 \le 1\) in the coordinates (u, v) relative to its centre (x, y), as in the Oxford affine-region files. (x, y) becomes the LAF centre unchanged.
A is the lower-triangular matrix with a positive diagonal and A @ A.T == inverse([[a, b], [b, c]]): the Cholesky factor of the inverse ellipse matrix, as VLFeat’s vl_frame2oell.m builds it, not the symmetric inverse square root (the two differ by a rotation). Either one maps the unit circle onto the ellipse; the lower-triangular one is what make_upright() returns, up to its stabilizers, for any LAF of that region.
The conversion checks only the shape \((B, N, 5)\); it does not check the ellipse values and does not raise on them. An ellipse that is degenerate or indefinite (a, c or a * c - b * b not positive) is not a bounded region; its LAF has inf or nan entries, or huge finite ones where rounding leaves a - b * b / c barely positive. Rounding also works the other way: a nearly singular valid ellipse can give inf or nan. laf_is_valid() screens out the inf and nan results, which an isnan test misses for inf; the huge finite ones pass it, so check the ellipse itself where that matters.

See also vlfeat/vlfeat

Parameters:: ells (Tensor) – torch.Tensor \((B, N, 5)\) of ellipses in Oxford format [x y a b c].
Return type:: Tensor
Returns:: LAF \((B, N, 2, 3)\)

Example

>>> input = torch.ones(1, 10, 5)  # BxNx5
>>> output = ellipse_to_laf(input)  #  BxNx2x3

kornia.feature.make_upright(laf, eps=1e-9)[source]#

Rectify the affine matrix, so that it becomes upright.

Convention:

The result is [U | c]: the centre is kept, and U is the lower-triangular matrix with a positive diagonal and \(U U^\top = A A^\top\). The region, the ellipse c + A u over the unit vectors u, and the scale are therefore kept, and the orientation becomes 0.
For \(\det A > 0\), A = U R with R the rotation matrix of the LAF’s orientation, as in laf_from_center_scale_ori(). For \(\det A < 0\) the reflection is dropped too: U has the determinant \(|\det A|\).
The equalities above hold up to two stabilizers: U is built from the norm of the first row of A plus eps, and from the scale of get_laf_scale(), which adds 1e-10 to the determinant. Both are negligible for a LAF in pixels, but a tiny frame such as A = diag(1e-12, 1) keeps neither its region nor its scale. The centre and the zero orientation are exact.
The all-zero LAF that pads an unfilled detector slot (see laf_is_filled()) is returned unchanged, so the padding stays recognizable.
A float16 LAF is rectified in float32, so that a LAF whose entry products leave the float16 range does not overflow; the result is returned in the LAF dtype.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
eps (float, optional) – for safe division. Default: 1e-9

Returns:

\((B, N, 2, 3)\)

Return type:

laf

Example

>>> input = torch.ones(1, 5, 2, 3)  # BxNx2x3
>>> output = make_upright(input)  #  BxNx2x3

kornia.feature.scale_laf(laf, scale_coef)[source]#

Multiplies region part of LAF ([:, :, :2, :2]) by a scale_coefficient.

Convention:

The result is [k A | c] for k = scale_coef, a product taken elementwise: the centre is kept. With one coefficient per LAF, the scale is multiplied by |k|, and a negative k also turns the frame by 180 degrees.
scale_coef is a float or a tensor that broadcasts against \((B, N, 2, 2)\), such as one coefficient per LAF in a \((B, N, 1, 1)\) tensor; an int is rejected.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
scale_coef (Union[float, Tensor]) – broadcastable torch.Tensor or float.

Return type:

Tensor

Returns:

LAF \((B, N, 2, 3)\)

Raises:

TypeError – if scale_coef is neither a float nor a tensor.

Example

>>> input = torch.ones(1, 5, 2, 3)  # BxNx2x3
>>> scale = 0.5
>>> output = scale_laf(input, scale)  # BxNx2x3

kornia.feature.get_laf_scale(LAF)[source]#

Return a scale of the LAFs.

Convention:

See the Convention block on laf_from_center_scale_ori(), which defines the scale.
The determinant of a float16 LAF is formed in float32, so that a LAF whose entry products leave the float16 range does not overflow; the scale is returned in the LAF dtype.

Parameters:: LAF (Tensor) – \((B, N, 2, 3)\)
Return type:: Tensor
Returns:: scale \((B, N, 1, 1)\)

Example

>>> input = torch.ones(1, 5, 2, 3)  # BxNx2x3
>>> output = get_laf_scale(input)  # BxNx1x1

kornia.feature.get_laf_center(LAF)[source]#

Return a center (keypoint) of the LAFs.

Convention:: The centre is (x, y) in pixels, with pixel centres at integers; see Local features.

Parameters:: LAF (Tensor) – \((B, N, 2, 3)\)
Return type:: Tensor
Returns:: xy \((B, N, 2)\)

Example

>>> input = torch.ones(1, 5, 2, 3)  # BxNx2x3
>>> output = get_laf_center(input)  # BxNx2

kornia.feature.rotate_laf(LAF, angles_degrees)[source]#

Apply additional rotation to the LAFs.

Convention:

The result is [A @ R | c], with R the rotation matrix of angles_degrees from angle_to_rotation_matrix(). The orientation grows by angles_degrees (set_laf_orientation() sets it instead), and the centre and the upright factor that make_upright() returns are kept.

Parameters:

LAF (Tensor) – \((B, N, 2, 3)\)
angles_degrees (Tensor) – \((B, N, 1)\) in degrees.

Return type:

Tensor

Returns:

LAF oriented with angles \((B, N, 2, 3)\)

kornia.feature.get_laf_orientation(LAF)[source]#

Return orientation of the LAFs, in degrees.

Convention:: See the Convention block on laf_from_center_scale_ori(), which defines the orientation.

Parameters:: LAF (Tensor) – \((B, N, 2, 3)\)
Return type:: Tensor
Returns:: angle in degrees \((B, N, 1)\)

Example

>>> input = torch.ones(1, 5, 2, 3)  # BxNx2x3
>>> output = get_laf_orientation(input)  # BxNx1

kornia.feature.set_laf_orientation(LAF, angles_degrees)[source]#

Change the orientation of the LAFs.

Convention:: See the Convention block on rotate_laf(). This rotates by angles_degrees minus the current orientation, so the orientation reads back as angles_degrees wrapped into [-180, 180].

Parameters:

LAF (Tensor) – \((B, N, 2, 3)\)
angles_degrees (Tensor) – \((B, N, 1)\) in degrees.

Return type:

Tensor

Returns:

LAF oriented with angles \((B, N, 2, 3)\)

kornia.feature.laf_from_center_scale_ori(xy, scale=None, ori=None)[source]#

Create a LAF from keypoint center, scale and orientation.

Convention:

A LAF is a \((B, N, 2, 3)\) tensor [A | c], with the centre c = (x, y) and A in pixels. A maps the patch frame onto the image: its first column is the image of the patch’s x axis and its second column that of the patch’s y axis. Local features gives the pixel frame, the patch that extract_patches_from_pyramid() samples and the conversion of OpenCV keypoints.
This function returns A = scale * R, with R = [[cos, sin], [-sin, cos]] of ori in degrees, the matrix of angle_to_rotation_matrix(). A positive ori turns the patch’s x axis counter-clockwise on screen, with image y pointing down.
For any LAF, the scale is \(\sqrt{|\det A|}\) (get_laf_scale(), which adds 1e-10 to the determinant) and the orientation is atan2(a12, a11) in degrees (get_laf_orientation()), within [-180, 180]: a LAF made with ori = 181 reads back -179. For \(\det A > 0\), A = U R with U the upright A that make_upright() returns, up to its stabilizers, and R the rotation matrix of that orientation.

Parameters:

xy (Tensor) – \((B, N, 2)\).
scale (Optional[Tensor], optional) – \((B, N, 1, 1)\). If not provided, scale = 1.0 is assumed Default: None
ori (Optional[Tensor], optional) – angle in degrees \((B, N, 1)\). If not provided orientation = 0 is assumed Default: None

Return type:

Tensor

Returns:

LAF \((B, N, 2, 3)\)

Example

>>> xy = torch.tensor([[[5.0, 7.0]]])
>>> laf_from_center_scale_ori(xy, torch.full((1, 1, 1, 1), 2.0), torch.tensor([[[30.0]]]))
tensor([[[[ 1.7321,  1.0000,  5.0000],
          [-1.0000,  1.7321,  7.0000]]]])

kornia.feature.laf_is_filled(laf)[source]#

Check which slots hold a detection rather than the zero-LAF padding.

Convention:

kornia’s LAF detectors pad the slots no detection filled with an all-zero LAF and a zero response, so that a batch is one tensor. Drop those slots before matching: the padding frames are identical to one another, so a mutual nearest-neighbour test does not reject them and they match each other at zero distance.
Only an all-zero frame is padding. Any other frame is filled: a detection at the image origin, which has a zero centre but a nonzero shape, and also a singular or non-finite frame, which laf_is_valid() screens out.
Occupancy cannot be read off the response instead, because a pluggable signed response may have a genuine maximum at exactly zero.

Parameters:: laf (Tensor) – \((B, N, 2, 3)\).
Return type:: Tensor
Returns:: mask of the slots a detection filled \((B, N)\).

Example

>>> laf = torch.tensor([[[[2., 0., 5.], [0., 3., 7.]], [[0., 0., 0.], [0., 0., 0.]]]])
>>> laf_is_filled(laf)
tensor([[ True, False]])

kornia.feature.laf_is_inside_image(laf, images, border=0)[source]#

Check which LAFs lie fully inside the image.

Convention:

The region of a LAF [A | c] is the ellipse c + A u over the unit vectors u. Along image axis i (0 for x, 1 for y) it spans exactly c_i - r_i to c_i + r_i, where r_i is the Euclidean norm of row i of A. A LAF is inside when both spans lie in border <= x <= w - 1 - border and border <= y <= h - 1 - border, so a region that touches the last valid pixel coordinate is inside.
The all-zero padding LAF is a point at the origin, so it counts as inside for border <= 0, the default included; combine the mask with laf_is_filled() to drop the padding.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\).
images (Tensor) – images, lafs are detected in \((B, CH, H, W)\).
border (int, optional) – additional border. Default: 0

Return type:

Tensor

Returns:

mask with shape \((B, N)\).

kornia.feature.laf_is_valid(laf)[source]#

Check that each LAF is finite and has a finite, nonzero determinant.

Convention:

A LAF with a negative determinant, a mirrored frame, is valid. The all-zero padding LAF (see laf_is_filled()), a rank-1 A and an A whose determinant overflows are not.
The determinant of a float16 LAF is formed in float32, so the LAF is not judged by an intermediate outside the float16 range: one whose entry products overflow float16 (for instance any LAF of scale 256 px or more) is valid, and so is one whose determinant is too small for float16. The determinant of a float32, float64 or bfloat16 LAF is formed in that dtype, where it can overflow.

Parameters:: laf (Tensor) – \((B, N, 2, 3)\).
Return type:: Tensor
Returns:: validity mask \((B, N)\).

Example

>>> laf = torch.eye(2, 3).view(1, 1, 2, 3)
>>> laf_is_valid(laf)
tensor([[True]])

kornia.feature.laf_to_three_points(laf)[source]#

Convert local affine frames (LAFs) to three points: the tips of the patch’s x and y axes, and the centre.

Convention:

The columns of the result are the points c + A[:, 0], c + A[:, 1] and c, in that order: the images of the patch points (1, 0), (0, 1) and (0, 0), at the frame’s scale rather than of unit length. laf_from_three_points() is the inverse.

Parameters:: laf (Tensor) – \((B, N, 2, 3)\).
Return type:: Tensor
Returns:: threepts \((B, N, 2, 3)\).

kornia.feature.laf_from_three_points(threepts)[source]#

Convert three points to local affine frame.

Convention:: See the Convention block on laf_to_three_points(), which this inverts: the columns of threepts are the images of the patch points (1, 0), (0, 1) and (0, 0), in that order.

Parameters:: threepts (Tensor) – \((B, N, 2, 3)\).
Return type:: Tensor
Returns:: laf \((B, N, 2, 3)\).

kornia.feature.KORNIA_CHECK_LAF(laf, raises=True)[source]#

Check whether a Local Affine Frame (laf) has a valid shape.

Convention:

Only the shape \((B, N, 2, 3)\) is checked: any dtype and any values pass, N = 0 included. Use laf_is_valid() to check the values.
With raises=False, a wrong shape returns False instead of raising.

Parameters:

laf (Tensor) – local affine frame tensor to evaluate.
raises (bool, optional) – bool indicating whether an exception should be raised upon failure. Default: True

Raises:

ShapeError – if the input laf does not have a shape \((B,N,2,3)\) and raises is True.

Return type:

bool

Note

Checks can be disabled in Python mode using disable_checks() or the KORNIA_CHECKS environment variable. In TorchScript-compiled code, checks always run (TorchScript cannot access module-level globals, but the validation logic is fast). When running with python -O, Python’s optimizer may eliminate some checks.

Example

>>> lafs = torch.rand(2, 10, 2, 3)
>>> KORNIA_CHECK_LAF(lafs)
True

kornia.feature.perspective_transform_lafs(trans_01, lafs_1)[source]#

Apply perspective transformations to a set of local affine frames (LAFs).

Convention:

trans_01 maps pixels of frame 1 to pixels of frame 0, as in transform_points(), and the centre maps through it exactly. The new A is the chord through the mapped tips of the patch axes: the three points of laf_to_three_points() are transformed and turned back into a LAF. For an affine trans_01 this equals the Jacobian of trans_01 at the centre times A; for a perspective one the two differ, the more so the larger the LAF.

Parameters:

trans_01 (Tensor) – torch.Tensor for perspective transformations of shape \((B, 3, 3)\).
lafs_1 (Tensor) – torch.Tensor of lafs of shape \((B, N, 2, 3)\).

Return type:

Tensor

Returns:

the transformed LAFs in frame 0, of shape \((B, N, 2, 3)\).

Examples

>>> rng = torch.manual_seed(0)
>>> lafs_1 = torch.rand(2, 4, 2, 3)  # BxNx2x3
>>> lafs_1
tensor([[[[0.4963, 0.7682, 0.0885],
          [0.1320, 0.3074, 0.6341]],

         [[0.4901, 0.8964, 0.4556],
          [0.6323, 0.3489, 0.4017]],

         [[0.0223, 0.1689, 0.2939],
          [0.5185, 0.6977, 0.8000]],

         [[0.1610, 0.2823, 0.6816],
          [0.9152, 0.3971, 0.8742]]],


        [[[0.4194, 0.5529, 0.9527],
          [0.0362, 0.1852, 0.3734]],

         [[0.3051, 0.9320, 0.1759],
          [0.2698, 0.1507, 0.0317]],

         [[0.2081, 0.9298, 0.7231],
          [0.7423, 0.5263, 0.2437]],

         [[0.5846, 0.0332, 0.1387],
          [0.2422, 0.8155, 0.7932]]]])
>>> trans_01 = torch.eye(3).repeat(2, 1, 1)  # Bx3x3
>>> trans_01.shape
torch.Size([2, 3, 3])
>>> lafs_0 = perspective_transform_lafs(trans_01, lafs_1)  # BxNx2x3

Orientation and affine shape estimation#

class kornia.feature.PassLAF(*args, **kwargs)[source]#

Dummy module to use instead of local feature orientation or affine shape estimator.

Convention:: Returns its laf argument itself, not a copy, and does not use img.

forward(laf, img)[source]#

Run forward.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
img (Tensor) – not used.

Return type:

Tensor

Returns:

LAF, unchanged \((B, N, 2, 3)\)

class kornia.feature.PatchAffineShapeEstimator(patch_size=19, eps=1e-10)[source]#

Module, which estimates the second moment matrix of the patch gradients.

The method determines the affine shape of the local feature as in [Baumberg00].

For float16 and bfloat16 inputs, the gradients, Gaussian weighting, moments, and normalization are computed in float32. The output keeps the input dtype.

Convention:

The output \((a, b, c)\) is \((\langle g_x^2 \rangle, \langle g_x g_y \rangle, \langle g_y^2 \rangle)\), the means over the patch of the products of its Sobel gradients, x right and y down, each gradient weighted by a Gaussian centred on the patch. It is divided by its largest entry, which is therefore 1. When two of the three entries are below eps before that division – a flat or low-contrast patch, or one whose gradient lies along an image axis – the output is \((1, 0, 1)\), a circle.
The patch must be patch_size pixels square.
Known defects: a negative \(b\) counts as below eps, so a patch and its mirror image can get a thin ellipse and a circle (#5824).

Parameters:

patch_size (int, optional) – the input image patch size. Default: 19
eps (float, optional) – for safe division. Default: 1e-10

forward(patch)[source]#

Run forward.

Parameters:: patch (Tensor) – \((B, 1, H, W)\)
Returns:: ellipse_shape \((B, 1, 3)\)
Return type:: torch.Tensor

class kornia.feature.LAFAffineShapeEstimator(patch_size=32, affine_shape_detector=None, preserve_orientation=True)[source]#

Module, which extracts patches using input images and local affine frames (LAFs).

Then runs PatchAffineShapeEstimator on patches to estimate LAFs shape.

Convention:

Each patch is sampled through the upright form of its LAF (make_upright()), so the shape is estimated in the coordinates of that frame, and the output region is the frame composed with the estimated shape. The estimated shape’s ellipse \(A A^\top\) is proportional to the inverse of the second moment matrix that affine_shape_detector returns. For an isotropic input LAF (a circle), the output shape is the estimated shape itself. An input whose shape already matches the local structure gives a nearly isotropic patch, so its shape changes little: one call is one step of the iterative shape adaptation of [Baumberg00].
The centre and the scale of the input LAF are kept. With preserve_orientation=True (the default) its orientation is kept too; otherwise the output is upright, and an orientation estimator such as LAFOrienter can be run after this module.

Parameters:

patch_size (int, optional) – the input image patch size. Default: 32
affine_shape_detector (Optional[Module], optional) – Patch affine shape estimator, PatchAffineShapeEstimator. Default: None
preserve_orientation (bool, optional) – if True, the original orientation is preserved. Default: True

forward(laf, img)[source]#

Run forward.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
img (Tensor) – \((B, 1, H, W)\)

Returns:

\((B, N, 2, 3)\)

Return type:

LAF_out

class kornia.feature.LAFOrienter(patch_size=32, num_angular_bins=36, angle_detector=None)[source]#

Module, which extracts patches using input images and local affine frames (LAFs).

Then runs PatchDominantGradientOrientation or OriNet on patches and then rotates the LAFs by the estimated angles

Convention:

Each LAF’s patch (extract_patches_from_pyramid()) gives an angle, measured in the patch, and the output orientation is the input orientation plus that angle, converted to degrees, modulo 360 degrees. get_laf_orientation() reads it back in [-180, 180]. The centre, the scale and the upright shape (make_upright()) of the LAF are kept.
For an isotropic LAF, the default angle detector therefore turns the LAF’s x axis onto the dominant gradient, whatever the input orientation: content turned 30 degrees counter-clockwise on screen turns the output orientation by +30, the sign of laf_from_center_scale_ori() (see Local features).

Parameters:

patch_size (int, optional) – side of the square patch sampled for each LAF. Default: 32
num_angular_bins (int, optional) – number of histogram bins of the default angle detector; not used when angle_detector is given. Default: 36
angle_detector (Optional[Module], optional) – Patch orientation estimator, e.g. PatchDominantGradientOrientation or OriNet. Default: None

forward(laf, img)[source]#

Run forward.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
img (Tensor) – \((B, 1, H, W)\)

Returns:

\((B, N, 2, 3)\)

Return type:

LAF_out

class kornia.feature.PatchDominantGradientOrientation(patch_size=32, num_angular_bins=36, eps=1e-8)[source]#

Module, which estimates the direction of the dominant gradient of the given patches, as an angle in radians.

Zero angle points towards right.

Convention:

The angle is that of the dominant gradient, which points toward brighter values, counter-clockwise on screen with image y pointing down: \(-\operatorname{atan2}(g_y, g_x)\) in pixel coordinates. A gradient pointing down the patch gives \(-\pi / 2\). LAFOrienter adds the angle to the LAF orientation.
The angle is not wrapped into \((-\pi, \pi]\): it can exceed \(\pi\) by up to pi / num_angular_bins.

Parameters:

patch_size (int, optional) – size of the (square) input patch. Default: 32
num_angular_bins (int, optional) – number of histogram bins. Default: 36
eps (float, optional) – for safe division, and arctan. Default: 1e-8

Note

Each gradient pixel is accumulated into its two neighbouring histogram bins with scatter_add_. On CUDA that accumulation uses atomics, so two calls on identical input can differ at the ulp level, and a patch whose two strongest bins are nearly tied can return the neighbouring peak. torch.use_deterministic_algorithms(True) selects the deterministic kernel and restores run-to-run reproducibility.

forward(patch)[source]#

Run forward.

Parameters:: patch (Tensor) – \((B, 1, H, W)\)
Returns:: \((B)\)
Return type:: angle in radians

class kornia.feature.OriNet(pretrained=True, eps=1e-8)[source]#

Network, which estimates the canonical orientation of the given 32x32 patches, in radians.

Zero angle points towards right. This is based on the original code from paper “Repeatability Is Not Enough: Learning Discriminative Affine Regions via Discriminability””. See [MRM18] for more details.

Convention:

With the pretrained weights, the angle turns with the patch content, in the sense of PatchDominantGradientOrientation: content turned counter-clockwise on screen increases the angle. See LAFOrienter for its use.
Each patch is normalised first: its mean is subtracted and the result is divided by its standard deviation plus 1e-6. An offset of its intensity therefore does not change the angle, and a positive gain changes it only through that 1e-6: little on a textured patch, more as the patch flattens, up to pi rad on a nearly flat one. An inverted patch generally gets a different angle.

Parameters:

pretrained (bool, optional) – Download and set pretrained weights to the model. Default: True
eps (float, optional) – to avoid division by zero in atan2. Default: 1e-8

Returns:

Angle in radians.

Shape:

Input: (B, 1, 32, 32)
Output: (B)

Examples

>>> input = torch.rand(16, 1, 32, 32)
>>> orinet = OriNet(pretrained=False).eval()
>>> angle = orinet(input) # 16

forward(patch)[source]#

Run forward.

Parameters:: patch (Tensor) – \((B, 1, H, W)\)
Returns:: \((B)\)
Return type:: angle in radians

class kornia.feature.LAFAffNetShapeEstimator(pretrained=True, preserve_orientation=True)[source]#

Module, which extracts patches using input images and local affine frames (LAFs).

Then runs AffNet on patches to estimate LAFs shape. This is based on the original code from paper “Repeatability Is Not Enough: Learning Discriminative Affine Regions via Discriminability””. See [MRM18] for more details.

Convention:: As LAFAffineShapeEstimator, with AffNet’s prediction on a 32 x 32 patch in place of the second moment estimate; see its Convention block for the frame the patch is sampled through and for what the output keeps of the input LAF.

Parameters:

pretrained (bool, optional) – Download and set pretrained weights to the model. Default: True
preserve_orientation (bool, optional) – if True, the original orientation is preserved. Default: True

forward(laf, img)[source]#

Run forward.

Parameters:

laf (Tensor) – \((B, N, 2, 3)\)
img (Tensor) – \((B, 1, H, W)\)

Returns:

\((B, N, 2, 3)\)

Return type:

LAF_out