kornia.geometry.epipolar¶
Module to with useful functionalities for epipolar geometry used by Structure from Motion
Essential¶

essential_from_fundamental
(F_mat: torch.Tensor, K1: torch.Tensor, K2: torch.Tensor) → torch.Tensor[source]¶ Get Essential matrix from Fundamental and Camera matrices.
Uses the method from Hartley/Zisserman 9.6 pag 257 (formula 9.12).
 Parameters
F_mat (torch.Tensor) – The fundamental matrix with shape of \((*, 3, 3)\).
K1 (torch.Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).
K2 (torch.Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).
 Returns
The essential matrix with shape \((*, 3, 3)\).
 Return type

essential_from_Rt
(R1: torch.Tensor, t1: torch.Tensor, R2: torch.Tensor, t2: torch.Tensor) → torch.Tensor[source]¶ Get the Essential matrix from Camera motion (Rs and ts).
Reference: Hartley/Zisserman 9.6 pag 257 (formula 9.12)
 Parameters
R1 (torch.Tensor) – The first camera rotation matrix with shape \((*, 3, 3)\).
t1 (torch.Tensor) – The first camera translation vector with shape \((*, 3, 1)\).
R2 (torch.Tensor) – The second camera rotation matrix with shape \((*, 3, 3)\).
t2 (torch.Tensor) – The second camera translation vector with shape \((*, 3, 1)\).
 Returns
The Essential matrix with the shape \((*, 3, 3)\).
 Return type

decompose_essential_matrix
(E_mat: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor, torch.Tensor][source]¶ Decompose an essential matrix to possible rotations and translation.
This function decomposes the essential matrix E using svd decomposition [96] and give the possible solutions: \(R1, R2, t\).
 Parameters
E_mat (torch.Tensor) – The essential matrix in the form of \((*, 3, 3)\).
 Returns
A tuple containing the first and second possible rotation matrices and the translation vector. The shape of the tensors with be same input \([(*, 3, 3), (*, 3, 3), (*, 3, 1)]\).
 Return type
Tuple[torch.Tensor, torch.Tensor, torch.Tensor]

motion_from_essential
(E_mat: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor][source]¶ Get Motion (R’s and t’s ) from Essential matrix.
Computes and return four possible poses exist for the decomposition of the Essential matrix. The posible solutions are \([R1,t], [R1,−t], [R2,t], [R2,−t]\).
 Parameters
E_mat (torch.Tensor) – The essential matrix in the form of \((*, 3, 3)\).
 Returns
The rotation and translation containing the four possible combination for the retrieved motion. The tuple is as following \([(*, 4, 3, 3), (*, 4, 3, 1)]\).
 Return type
Tuple[torch.Tensor, torch.Tensor]

motion_from_essential_choose_solution
(E_mat: torch.Tensor, K1: torch.Tensor, K2: torch.Tensor, x1: torch.Tensor, x2: torch.Tensor, mask: Optional[torch.Tensor] = None) → Tuple[torch.Tensor, torch.Tensor, torch.Tensor][source]¶ Recover the relative camera rotation and the translation from an estimated essential matrix.
The method checks the corresponding points in two images and also returns the triangulated 3d points. Internally uses
decompose_essential_matrix()
and then chooses the best solution based on the combination that gives more 3d points in front of the camera plane fromtriangulate_points()
. Parameters
E_mat (torch.Tensor) – The essential matrix in the form of \((*, 3, 3)\).
K1 (torch.Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).
K2 (torch.Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).
x1 (torch.Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).
x2 (torch.Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).
mask (torch.Tensor) – A boolean mask which can be used to exclude some points from choosing the best solution. This is useful for using this function with sets of points of different cardinality (for instance after filtering with RANSAC) while keeping batch semantics. Mask is of shape \((*, N)\).
 Returns
The rotation and translation plus the 3d triangulated points. The tuple is as following \([(*, 3, 3), (*, 3, 1), (*, N, 3)]\).
 Return type
Tuple[torch.Tensor, torch.Tensor, torch.Tensor]

relative_camera_motion
(R1: torch.Tensor, t1: torch.Tensor, R2: torch.Tensor, t2: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor][source]¶ Computes the relative camera motion between two cameras.
Given the motion parameters of two cameras, computes the motion parameters of the second one assuming the first one to be at the origin. If \(T1\) and \(T2\) are the camera motions, the computed relative motion is \(T = T_{2}T^{−1}_{1}\).
 Parameters
R1 (torch.Tensor) – The first camera rotation matrix with shape \((*, 3, 3)\).
t1 (torch.Tensor) – The first camera translation vector with shape \((*, 3, 1)\).
R2 (torch.Tensor) – The second camera rotation matrix with shape \((*, 3, 3)\).
t2 (torch.Tensor) – The second camera translation vector with shape \((*, 3, 1)\).
 Returns
A tuple with the relative rotation matrix and translation vector with the shape of \([(*, 3, 3), (*, 3, 1)]\).
 Return type
Tuple[torch.Tensor, torch.Tensor]
Fundamental¶

find_fundamental
(points1: torch.Tensor, points2: torch.Tensor, weights: torch.Tensor) → torch.Tensor[source]¶ Computes the fundamental matrix using the DLT formulation.
The linear system is solved by using the Weighted Least Squares Solution for the 8 Points algorithm.
 Parameters
points1 (torch.Tensor) – A set of points in the first image with a tensor shape \((B, N, 2)\).
points2 (torch.Tensor) – A set of points in the second image with a tensor shape \((B, N, 2)\).
weights (torch.Tensor) – Tensor containing the weights per point correspondence with a shape of \((B, N)\).
 Returns
the computed fundamental matrix with shape \((B, 3, 3)\).
 Return type

fundamental_from_essential
(E_mat: torch.Tensor, K1: torch.Tensor, K2: torch.Tensor) → torch.Tensor[source]¶ Get the Fundamental matrix from Essential and camera matrices.
Uses the method from Hartley/Zisserman 9.6 pag 257 (formula 9.12).
 Parameters
E_mat (torch.Tensor) – The essential matrix with shape of \((*, 3, 3)\).
K1 (torch.Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).
K2 (torch.Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).
 Returns
The fundamental matrix with shape \((*, 3, 3)\).
 Return type

fundamental_from_projections
(P1: torch.Tensor, P2: torch.Tensor) → torch.Tensor[source]¶ Get the Fundamental matrix from Projection matrices.
 Parameters
P1 (torch.Tensor) – The projection matrix from first camera with shape \((*, 3, 4)\).
P2 (torch.Tensor) – The projection matrix from second camera with shape \((*, 3, 4)\).
 Returns
The fundamental matrix with shape \((*, 3, 3)\).
 Return type

compute_correspond_epilines
(points: torch.Tensor, F_mat: torch.Tensor) → torch.Tensor[source]¶ Computes the corresponding epipolar line for a given set of points.
 Parameters
points (torch.Tensor) – tensor containing the set of points to project in the shape of \((B, N, 2)\).
F_mat (torch.Tensor) – the fundamental to use for projection the points in the shape of \((B, 3, 3)\).
 Returns
a tensor with shape \((B, N, 3)\) containing a vector of the epipolar lines corresponding to the points to the other image. Each line is described as \(ax + by + c = 0\) and encoding the vectors as \((a, b, c)\).
 Return type

normalize_points
(points: torch.Tensor, eps: float = 1e08) → Tuple[torch.Tensor, torch.Tensor][source]¶ Normalizes points (isotropic).
Computes the transformation matrix such that the two principal moments of the set of points are equal to unity, forming an approximately symmetric circular cloud of points of radius 1 about the origin. Reference: Hartley/Zisserman 4.4.4 pag.107
This operation is an essential step before applying the DLT algorithm in order to consider the result as optimal.
 Parameters
points (torch.Tensor) – Tensor containing the points to be normalized with shape \((B, N, 2)\).
eps (float) – epsilon value to avoid numerical instabilities. Default: 1e8.
 Returns
tuple containing the normalized points in the shape \((B, N, 2)\) and the transformation matrix in the shape \((B, 3, 3)\).
 Return type
Tuple[torch.Tensor, torch.Tensor]

normalize_transformation
(M: torch.Tensor, eps: float = 1e08) → torch.Tensor[source]¶ Normalizes a given transformation matrix.
The function trakes the transformation matrix and normalize so that the value in the last row and column is one.
 Parameters
M (torch.Tensor) – The transformation to be normalized of any shape with a minimum size of 2x2.
eps (float) – small value to avoid unstabilities during the backpropagation.
 Returns
the normalized transformation matrix with same shape as the input.
 Return type
Metrics¶

sampson_epipolar_distance
(pts1: torch.Tensor, pts2: torch.Tensor, Fm: torch.Tensor, squared: bool = True, eps: float = 1e08) → torch.Tensor[source]¶ Returns Sampson distance for correspondences given the fundamental matrix.
 Parameters
pts1 (torch.Tensor) – correspondences from the left images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.
pts2 (torch.Tensor) – correspondences from the right images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.
Fm (torch.Tensor) – Fundamental matrices with shape \((B, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.
squared (bool) – if True (default), the squared distance is returned.
eps (float) – Small constant for safe sqrt. Default 1e9.
 Returns
the computed Sampson distance with shape \((B, N)\).
 Return type

symmetrical_epipolar_distance
(pts1: torch.Tensor, pts2: torch.Tensor, Fm: torch.Tensor, squared: bool = True, eps: float = 1e08) → torch.Tensor[source]¶ Returns symmetrical epipolar distance for correspondences given the fundamental matrix.
 Parameters
pts1 (torch.Tensor) – correspondences from the left images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.
pts2 (torch.Tensor) – correspondences from the right images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.
Fm (torch.Tensor) – Fundamental matrices with shape \((B, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.
squared (bool) – if True (default), the squared distance is returned.
eps (float) – Small constant for safe sqrt. Default 1e9.
 Returns
the computed Symmetrical distance with shape \((B, N)\).
 Return type
Projection¶

projection_from_KRt
(K: torch.Tensor, R: torch.Tensor, t: torch.Tensor) → torch.Tensor[source]¶ Get the projection matrix P from K, R and t.
This function estimate the projection matrix by solving the following equation: \(P = K * [Rt]\).
 Parameters
K (torch.Tensor) – the camera matrix with the instrinsics with shape \((B, 3, 3)\).
R (torch.Tensor) – The rotation matrix with shape \((B, 3, 3)\).
t (torch.Tensor) – The translation vector with shape \((B, 3, 1)\).
 Returns
The projection matrix P with shape \((B, 4, 4)\).
 Return type

projections_from_fundamental
(F_mat: torch.Tensor) → torch.Tensor[source]¶ Get the projection matrices from the Fundamental Matrix.
 Parameters
F_mat (torch.Tensor) – the fundamental matrix with the shape \((*, 3, 3)\).
 Returns
The projection matrices with shape \((*, 4, 4, 2)\).
 Return type

intrinsics_like
(focal: float, input: torch.Tensor) → torch.Tensor[source]¶ Returns a 3x3 instrinsics matrix, with same size as the input.
The center of projection will be based in the input image size.
 Parameters
focal (float) – the focal length for tha camera matrix.
input (torch.Tensor) – image tensor that will determine the batch size and image height and width. It is assumed to be a tensor in the shape of \((B, C, H, W)\).
 Returns
The camera matrix with the shape of \((B, 3, 3)\).
 Return type

scale_intrinsics
(camera_matrix: torch.Tensor, scale_factor: Union[float, torch.Tensor]) → torch.Tensor[source]¶ Scale a camera matrix containing the intrinsics.
Applies the scaling factor to the focal length and center of projection.
 Parameters
camera_matrix (torch.Tensor) – the camera calibration matrix containing the intrinsic parameters. The expected shape for the tensor is \((B, 3, 3)\).
scale_factor (Union[float, torch.Tensor]) – the scaling factor to be applied.
 Returns
The scaled camera matrix with shame shape as input \((B, 3, 3)\).
 Return type

random_intrinsics
(low: Union[float, torch.Tensor], high: Union[float, torch.Tensor]) → torch.Tensor[source]¶ Generates a random camera matrix based on a given uniform distribution.
 Parameters
low (Union[float, torch.Tensor]) – lower range (inclusive).
high (Union[float, torch.Tensor]) – upper range (exclusive).
 Returns
The random camera matrix with the shape of \((1, 3, 3)\).
 Return type
Numeric¶

cross_product_matrix
(x: torch.Tensor) → torch.Tensor[source]¶ Returns the cross_product_matrix symmetric matrix of a vector.
 Parameters
x (torch.Tensor) – The input vector to construct the matrix in the shape \((B, 3)\).
 Returns
The constructed cross_product_matrix symmetric matrix with shape \((B, 3, 3)\).
 Return type

eye_like
(n: int, input: torch.Tensor) → torch.Tensor[source]¶ Returns a 2D tensor with ones on the diagonal and zeros elsewhere with same size as the input.
 Parameters
n (int) – the number of rows \((N)\).
input (torch.Tensor) – image tensor that will determine the batch size of the output matrix. The expected shape is \((B, *)\).
 Returns
The identity matrix with same size as input \((*, N, N)\).
 Return type

vec_like
(n, tensor)[source]¶ Returns a 2D tensor with a vector containing zeros with same size as the input.
 Parameters
n (int) – the number of rows \((N)\).
input (torch.Tensor) – image tensor that will determine the batch size of the output matrix. The expected shape is \((B, *)\).
 Returns
The vector with same size as input \((*, N, 1)\).
 Return type
Triangulation¶

triangulate_points
(P1: torch.Tensor, P2: torch.Tensor, points1: torch.Tensor, points2: torch.Tensor) → torch.Tensor[source]¶ Reconstructs a bunch of points by triangulation.
Triangulates the 3d position of 2d correspondences between several images. Reference: Internally it uses DLT method from Hartley/Zisserman 12.2 pag.312
The input points are assumed to be in homogeneous coordinate system and being inliers correspondences. The method does not perform any robust estimation.
 Parameters
P1 (torch.Tensor) – The projection matrix for the first camera with shape \((*, 3, 4)\).
P2 (torch.Tensor) – The projection matrix for the second camera with shape \((*, 3, 4)\).
points1 (torch.Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).
points2 (torch.Tensor) – The set of points seen from the second camera frame in the camera plane coordinates with shape \((*, N, 2)\).
 Returns
The reconstructed 3d points in the world frame with shape \((*, N, 3)\).
 Return type