kornia.geometry.epipolar#

Module with useful functionalities for epipolar geometry used by Structure from Motion

Essential#

kornia.geometry.epipolar.find_essential(points1, points2, weights=None)#

Parameters:

points1 (Tensor) – A set of points in the first image with a tensor shape \((B, N, 2), N>=5\).
points2 (Tensor) – A set of points in the second image with a tensor shape \((B, N, 2), N>=5\).
weights (Optional[Tensor], optional) – Tensor containing the weights per point correspondence with a shape of \((5, N)\). Default: None

Return type:

Tensor

Returns:

the computed essential matrix with shape \((B, 3, 3)\), one model for each batch selected out of ten solutions by Sampson distances.

kornia.geometry.epipolar.essential_from_fundamental(F_mat, K1, K2)#

Get Essential matrix from Fundamental and Camera matrices.

Uses the method from Hartley/Zisserman 9.6 pag 257 (formula 9.12).

Parameters:

F_mat (Tensor) – The fundamental matrix with shape of \((*, 3, 3)\).
K1 (Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).
K2 (Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).

Return type:

Tensor

Returns:

The essential matrix with shape \((*, 3, 3)\).

kornia.geometry.epipolar.essential_from_Rt(R1, t1, R2, t2)#

Get the Essential matrix from Camera motion (Rs and ts).

Reference: Hartley/Zisserman 9.6 pag 257 (formula 9.12)

Parameters:

R1 (Tensor) – The first camera rotation matrix with shape \((*, 3, 3)\).
t1 (Tensor) – The first camera translation vector with shape \((*, 3, 1)\).
R2 (Tensor) – The second camera rotation matrix with shape \((*, 3, 3)\).
t2 (Tensor) – The second camera translation vector with shape \((*, 3, 1)\).

Return type:

Tensor

Returns:

The Essential matrix with the shape \((*, 3, 3)\).

kornia.geometry.epipolar.decompose_essential_matrix(E_mat)#

Decompose an essential matrix to possible rotations and translation.

This function decomposes the essential matrix E using svd decomposition [96] and give the possible solutions: \(R1, R2, t\).

Parameters:: E_mat (Tensor) – The essential matrix in the form of \((*, 3, 3)\).
Return type:: Tuple[Tensor, Tensor, Tensor]
Returns:: A tuple containing the first and second possible rotation matrices and the translation vector. The shape of the tensors with be same input \([(*, 3, 3), (*, 3, 3), (*, 3, 1)]\).

kornia.geometry.epipolar.motion_from_essential(E_mat)#

Get Motion (R’s and t’s ) from Essential matrix.

Computes and return four possible poses exist for the decomposition of the Essential matrix. The possible solutions are \([R1,t], [R1,-t], [R2,t], [R2,-t]\).

Parameters:: E_mat (Tensor) – The essential matrix in the form of \((*, 3, 3)\).
Return type:: Tuple[Tensor, Tensor]
Returns:: The rotation and translation containing the four possible combination for the retrieved motion. The tuple is as following \([(*, 4, 3, 3), (*, 4, 3, 1)]\).

kornia.geometry.epipolar.motion_from_essential_choose_solution(E_mat, K1, K2, x1, x2, mask=None)#

Recover the relative camera rotation and the translation from an estimated essential matrix.

The method checks the corresponding points in two images and also returns the triangulated 3d points. Internally uses decompose_essential_matrix() and then chooses the best solution based on the combination that gives more 3d points in front of the camera plane from triangulate_points().

Parameters:

E_mat (Tensor) – The essential matrix in the form of \((*, 3, 3)\).
K1 (Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).
K2 (Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).
x1 (Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).
x2 (Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).
mask (Optional[Tensor], optional) – A boolean mask which can be used to exclude some points from choosing the best solution. This is useful for using this function with sets of points of different cardinality (for instance after filtering with RANSAC) while keeping batch semantics. Mask is of shape \((*, N)\). Default: None

Return type:

Tuple[Tensor, Tensor, Tensor]

Returns:

The rotation and translation plus the 3d triangulated points. The tuple is as following \([(*, 3, 3), (*, 3, 1), (*, N, 3)]\).

kornia.geometry.epipolar.relative_camera_motion(R1, t1, R2, t2)#

Compute the relative camera motion between two cameras.

Given the motion parameters of two cameras, computes the motion parameters of the second one assuming the first one to be at the origin. If \(T1\) and \(T2\) are the camera motions, the computed relative motion is \(T = T_{2}T^{-1}_{1}\).

Parameters:

R1 (Tensor) – The first camera rotation matrix with shape \((*, 3, 3)\).
t1 (Tensor) – The first camera translation vector with shape \((*, 3, 1)\).
R2 (Tensor) – The second camera rotation matrix with shape \((*, 3, 3)\).
t2 (Tensor) – The second camera translation vector with shape \((*, 3, 1)\).

Return type:

Tuple[Tensor, Tensor]

Returns:

A tuple with the relative rotation matrix and translation vector with the shape of \([(*, 3, 3), (*, 3, 1)]\).

Fundamental#

kornia.geometry.epipolar.find_fundamental(points1, points2, weights=None, method='8POINT')#

Parameters:

points1 (Tensor) – A set of points in the first image with a tensor shape \((B, N, 2), N>=8\).
points2 (Tensor) – A set of points in the second image with a tensor shape \((B, N, 2), N>=8\).
weights (Optional[Tensor], optional) – Tensor containing the weights per point correspondence with a shape of \((B, N)\). Default: None
method (Literal['8POINT', '7POINT'], optional) – The method to use for computing the fundamental matrix. Supported methods are “7POINT” and “8POINT”. Default: "8POINT"

Return type:

Tensor

Returns:

the computed fundamental matrix with shape \((B, 3*m, 3)\), where m number of fundamental matrix.

Raises:

ValueError – If an invalid method is provided.

kornia.geometry.epipolar.fundamental_from_essential(E_mat, K1, K2)#

Get the Fundamental matrix from Essential and camera matrices.

Uses the method from Hartley/Zisserman 9.6 pag 257 (formula 9.12).

Parameters:

E_mat (Tensor) – The essential matrix with shape of \((*, 3, 3)\).
K1 (Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).
K2 (Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).

Return type:

Tensor

Returns:

The fundamental matrix with shape \((*, 3, 3)\).

kornia.geometry.epipolar.fundamental_from_projections(P1, P2)#

Get the Fundamental matrix from Projection matrices.

Parameters:

P1 (Tensor) – The projection matrix from first camera with shape \((*, 3, 4)\).
P2 (Tensor) – The projection matrix from second camera with shape \((*, 3, 4)\).

Return type:

Tensor

Returns:

The fundamental matrix with shape \((*, 3, 3)\).

kornia.geometry.epipolar.compute_correspond_epilines(points, F_mat)#

Compute the corresponding epipolar line for a given set of points.

Parameters:

points (Tensor) – tensor containing the set of points to project in the shape of \((*, N, 2)\) or \((*, N, 3)\).
F_mat (Tensor) – the fundamental to use for projection the points in the shape of \((*, 3, 3)\).

Return type:

Tensor

Returns:

a tensor with shape \((*, N, 3)\) containing a vector of the epipolar lines corresponding to the points to the other image. Each line is described as \(ax + by + c = 0\) and encoding the vectors as \((a, b, c)\).

kornia.geometry.epipolar.normalize_points(points, eps=1e-8)#

Normalizes points (isotropic).

Computes the transformation matrix such that the two principal moments of the set of points are equal to unity, forming an approximately symmetric circular cloud of points of radius 1 about the origin. Reference: Hartley/Zisserman 4.4.4 pag.107

This operation is an essential step before applying the DLT algorithm in order to consider the result as optimal.

Parameters:

points (Tensor) – Tensor containing the points to be normalized with shape \((B, N, 2)\).
eps (float, optional) – epsilon value to avoid numerical instabilities. Default: 1e-8

Return type:

Tuple[Tensor, Tensor]

Returns:

tuple containing the normalized points in the shape \((B, N, 2)\) and the transformation matrix in the shape \((B, 3, 3)\).

kornia.geometry.epipolar.normalize_transformation(M, eps=1e-8)#

Normalize a given transformation matrix.

The function trakes the transformation matrix and normalize so that the value in the last row and column is one.

Parameters:

M (Tensor) – The transformation to be normalized of any shape with a minimum size of 2x2.
eps (float, optional) – small value to avoid unstabilities during the backpropagation. Default: 1e-8

Return type:

Tensor

Returns:

the normalized transformation matrix with same shape as the input.

kornia.geometry.epipolar.get_perpendicular(lines, points)#

Compute the perpendicular to a line, through the point.

Parameters:

lines (Tensor) – tensor containing the set of lines \((*, N, 3)\).
points (Tensor) – tensor containing the set of points \((*, N, 2)\).

Return type:

Tensor

Returns:

a tensor with shape \((*, N, 3)\) containing a vector of the epipolar perpendicular lines. Each line is described as \(ax + by + c = 0\) and encoding the vectors as \((a, b, c)\).

kornia.geometry.epipolar.get_closest_point_on_epipolar_line(pts1, pts2, Fm)#

Return closest point on the epipolar line to the correspondence, given the fundamental matrix.

Parameters:

pts1 (Tensor) – correspondences from the left images with shape \((*, N, (2|3))\). If they are not homogeneous, converted automatically.
pts2 (Tensor) – correspondences from the right images with shape \((*, N, (2|3))\). If they are not homogeneous, converted automatically.
Fm (Tensor) – Fundamental matrices with shape \((*, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.

Return type:

Tensor

Returns:

point on epipolar line \((*, N, 2)\).

Metrics#

kornia.geometry.epipolar.sampson_epipolar_distance(pts1, pts2, Fm, squared=True, eps=1e-8)#

Return Sampson distance for correspondences given the fundamental matrix.

Parameters:

pts1 (Tensor) – correspondences from the left images with shape \((*, N, (2|3))\). If they are not homogeneous, converted automatically.
pts2 (Tensor) – correspondences from the right images with shape \((*, N, (2|3))\). If they are not homogeneous, converted automatically.
Fm (Tensor) – Fundamental matrices with shape \((*, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.
squared (bool, optional) – if True (default), the squared distance is returned. Default: True
eps (float, optional) – Small constant for safe sqrt. Default: 1e-8

Return type:

Tensor

Returns:

the computed Sampson distance with shape \((*, N)\).

kornia.geometry.epipolar.symmetrical_epipolar_distance(pts1, pts2, Fm, squared=True, eps=1e-8)#

Return symmetrical epipolar distance for correspondences given the fundamental matrix.

Parameters:

pts1 (Tensor) – correspondences from the left images with shape \((*, N, (2|3))\). If they are not homogeneous, converted automatically.
pts2 (Tensor) – correspondences from the right images with shape \((*, N, (2|3))\). If they are not homogeneous, converted automatically.
Fm (Tensor) – Fundamental matrices with shape \((*, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.
squared (bool, optional) – if True (default), the squared distance is returned. Default: True
eps (float, optional) – Small constant for safe sqrt. Default: 1e-8

Return type:

Tensor

Returns:

the computed Symmetrical distance with shape \((*, N)\).

kornia.geometry.epipolar.left_to_right_epipolar_distance(pts1, pts2, Fm)#

Return one-sided epipolar distance for correspondences given the fundamental matrix.

This method measures the distance from points in the right images to the epilines of the corresponding points in the left images as they reflect in the right images.

Parameters:

pts1 (Tensor) – correspondences from the left images with shape \((*, N, 2 or 3)\). If they are not homogeneous, converted automatically.
pts2 (Tensor) – correspondences from the right images with shape \((*, N, 2 or 3)\). If they are not homogeneous, converted automatically.
Fm (Tensor) – Fundamental matrices with shape \((*, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.

Return type:

Tensor

Returns:

the computed Symmetrical distance with shape \((*, N)\).

kornia.geometry.epipolar.right_to_left_epipolar_distance(pts1, pts2, Fm)#

Return one-sided epipolar distance for correspondences given the fundamental matrix.

This method measures the distance from points in the left images to the epilines of the corresponding points in the right images as they reflect in the left images.

Parameters:

pts1 (Tensor) – correspondences from the left images with shape \((*, N, 2 or 3)\). If they are not homogeneous, converted automatically.
pts2 (Tensor) – correspondences from the right images with shape \((*, N, 2 or 3)\). If they are not homogeneous, converted automatically.
Fm (Tensor) – Fundamental matrices with shape \((*, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.

Return type:

Tensor

Returns:

the computed Symmetrical distance with shape \((*, N)\).

Projection#

kornia.geometry.epipolar.projection_from_KRt(K, R, t)#

Get the projection matrix P from K, R and t.

This function estimate the projection matrix by solving the following equation: \(P = K * [R|t]\).

Parameters:

K (Tensor) – the camera matrix with the intrinsics with shape \((B, 3, 3)\).
R (Tensor) – The rotation matrix with shape \((B, 3, 3)\).
t (Tensor) – The translation vector with shape \((B, 3, 1)\).

Return type:

Tensor

Returns:

The projection matrix P with shape \((B, 4, 4)\).

kornia.geometry.epipolar.projections_from_fundamental(F_mat)#

Get the projection matrices from the Fundamental Matrix.

Parameters:: F_mat (Tensor) – the fundamental matrix with the shape \((B, 3, 3)\).
Return type:: Tensor
Returns:: The projection matrices with shape \((B, 3, 4, 2)\).

kornia.geometry.epipolar.intrinsics_like(focal, input)#

Return a 3x3 intrinsics matrix, with same size as the input.

The center of projection will be based in the input image size.

Parameters:

focal (float) – the focal length for the camera matrix.
input (Tensor) – image tensor that will determine the batch size and image height and width. It is assumed to be a tensor in the shape of \((B, C, H, W)\).

Return type:

Tensor

Returns:

The camera matrix with the shape of \((B, 3, 3)\).

kornia.geometry.epipolar.scale_intrinsics(camera_matrix, scale_factor)#

Scale a camera matrix containing the intrinsics.

Applies the scaling factor to the focal length and center of projection.

Parameters:

camera_matrix (Tensor) – the camera calibration matrix containing the intrinsic parameters. The expected shape for the tensor is \((B, 3, 3)\).
scale_factor (Union[float, Tensor]) – the scaling factor to be applied.

Return type:

Tensor

Returns:

The scaled camera matrix with shame shape as input \((B, 3, 3)\).

kornia.geometry.epipolar.random_intrinsics(low, high)#

Generate a random camera matrix based on a given uniform distribution.

Parameters:

low (Union[float, Tensor]) – lower range (inclusive).
high (Union[float, Tensor]) – upper range (exclusive).

Return type:

Tensor

Returns:

the random camera matrix with the shape of \((1, 3, 3)\).

Numeric#

kornia.geometry.epipolar.cross_product_matrix(x)#

Return the cross_product_matrix symmetric matrix of a vector.

Parameters:: x (Tensor) – The input vector to construct the matrix in the shape \((*, 3)\).
Return type:: Tensor
Returns:: The constructed cross_product_matrix symmetric matrix with shape \((*, 3, 3)\).

Triangulation#

kornia.geometry.epipolar.triangulate_points(P1, P2, points1, points2)#

Reconstructs a bunch of points by triangulation.

Triangulates the 3d position of 2d correspondences between several images. Reference: Internally it uses DLT method from Hartley/Zisserman 12.2 pag.312

The input points are assumed to be in homogeneous coordinate system and being inliers correspondences. The method does not perform any robust estimation.

Parameters:

P1 (Tensor) – The projection matrix for the first camera with shape \((*, 3, 4)\).
P2 (Tensor) – The projection matrix for the second camera with shape \((*, 3, 4)\).
points1 (Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).
points2 (Tensor) – The set of points seen from the second camera frame in the camera plane coordinates with shape \((*, N, 2)\).

Return type:

Tensor

Returns:

The reconstructed 3d points in the world frame with shape \((*, N, 3)\).