kornia.geometry.epipolar

Module to with useful functionalities for epipolar geometry used by Structure from Motion

_images/epipolar_geometry.svg.png

Essential

essential_from_fundamental(F_mat: torch.Tensor, K1: torch.Tensor, K2: torch.Tensor) → torch.Tensor[source]

Get Essential matrix from Fundamental and Camera matrices.

Uses the method from Hartley/Zisserman 9.6 pag 257 (formula 9.12).

Parameters

  • F_mat (torch.Tensor) – The fundamental matrix with shape of \((*, 3, 3)\).

  • K1 (torch.Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).

  • K2 (torch.Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).

Returns

The essential matrix with shape \((*, 3, 3)\).

Return type

torch.Tensor

essential_from_Rt(R1: torch.Tensor, t1: torch.Tensor, R2: torch.Tensor, t2: torch.Tensor) → torch.Tensor[source]

Get the Essential matrix from Camera motion (Rs and ts).

Reference: Hartley/Zisserman 9.6 pag 257 (formula 9.12)

Parameters

  • R1 (torch.Tensor) – The first camera rotation matrix with shape \((*, 3, 3)\).

  • t1 (torch.Tensor) – The first camera translation vector with shape \((*, 3, 1)\).

  • R2 (torch.Tensor) – The second camera rotation matrix with shape \((*, 3, 3)\).

  • t2 (torch.Tensor) – The second camera translation vector with shape \((*, 3, 1)\).

Returns

The Essential matrix with the shape \((*, 3, 3)\).

Return type

torch.Tensor

decompose_essential_matrix(E_mat: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor, torch.Tensor][source]

Decompose an essential matrix to possible rotations and translation.

This function decomposes the essential matrix E using svd decomposition [96] and give the possible solutions: \(R1, R2, t\).

Parameters

E_mat (torch.Tensor) – The essential matrix in the form of \((*, 3, 3)\).

Returns

A tuple containing the first and second possible rotation matrices and the translation vector. The shape of the tensors with be same input \([(*, 3, 3), (*, 3, 3), (*, 3, 1)]\).

Return type

Tuple[torch.Tensor, torch.Tensor, torch.Tensor]

motion_from_essential(E_mat: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor][source]

Get Motion (R’s and t’s ) from Essential matrix.

Computes and return four possible poses exist for the decomposition of the Essential matrix. The posible solutions are \([R1,t], [R1,−t], [R2,t], [R2,−t]\).

Parameters

E_mat (torch.Tensor) – The essential matrix in the form of \((*, 3, 3)\).

Returns

The rotation and translation containing the four possible combination for the retrieved motion. The tuple is as following \([(*, 4, 3, 3), (*, 4, 3, 1)]\).

Return type

Tuple[torch.Tensor, torch.Tensor]

motion_from_essential_choose_solution(E_mat: torch.Tensor, K1: torch.Tensor, K2: torch.Tensor, x1: torch.Tensor, x2: torch.Tensor, mask: Optional[torch.Tensor] = None) → Tuple[torch.Tensor, torch.Tensor, torch.Tensor][source]

Recover the relative camera rotation and the translation from an estimated essential matrix.

The method checks the corresponding points in two images and also returns the triangulated 3d points. Internally uses decompose_essential_matrix() and then chooses the best solution based on the combination that gives more 3d points in front of the camera plane from triangulate_points().

Parameters

  • E_mat (torch.Tensor) – The essential matrix in the form of \((*, 3, 3)\).

  • K1 (torch.Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).

  • K2 (torch.Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).

  • x1 (torch.Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).

  • x2 (torch.Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).

  • mask (torch.Tensor) – A boolean mask which can be used to exclude some points from choosing the best solution. This is useful for using this function with sets of points of different cardinality (for instance after filtering with RANSAC) while keeping batch semantics. Mask is of shape \((*, N)\).

Returns

The rotation and translation plus the 3d triangulated points. The tuple is as following \([(*, 3, 3), (*, 3, 1), (*, N, 3)]\).

Return type

Tuple[torch.Tensor, torch.Tensor, torch.Tensor]

relative_camera_motion(R1: torch.Tensor, t1: torch.Tensor, R2: torch.Tensor, t2: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor][source]

Computes the relative camera motion between two cameras.

Given the motion parameters of two cameras, computes the motion parameters of the second one assuming the first one to be at the origin. If \(T1\) and \(T2\) are the camera motions, the computed relative motion is \(T = T_{2}T^{−1}_{1}\).

Parameters

  • R1 (torch.Tensor) – The first camera rotation matrix with shape \((*, 3, 3)\).

  • t1 (torch.Tensor) – The first camera translation vector with shape \((*, 3, 1)\).

  • R2 (torch.Tensor) – The second camera rotation matrix with shape \((*, 3, 3)\).

  • t2 (torch.Tensor) – The second camera translation vector with shape \((*, 3, 1)\).

Returns

A tuple with the relative rotation matrix and translation vector with the shape of \([(*, 3, 3), (*, 3, 1)]\).

Return type

Tuple[torch.Tensor, torch.Tensor]

Fundamental

find_fundamental(points1: torch.Tensor, points2: torch.Tensor, weights: torch.Tensor) → torch.Tensor[source]

Computes the fundamental matrix using the DLT formulation.

The linear system is solved by using the Weighted Least Squares Solution for the 8 Points algorithm.

Parameters

  • points1 (torch.Tensor) – A set of points in the first image with a tensor shape \((B, N, 2)\).

  • points2 (torch.Tensor) – A set of points in the second image with a tensor shape \((B, N, 2)\).

  • weights (torch.Tensor) – Tensor containing the weights per point correspondence with a shape of \((B, N)\).

Returns

the computed fundamental matrix with shape \((B, 3, 3)\).

Return type

torch.Tensor

fundamental_from_essential(E_mat: torch.Tensor, K1: torch.Tensor, K2: torch.Tensor) → torch.Tensor[source]

Get the Fundamental matrix from Essential and camera matrices.

Uses the method from Hartley/Zisserman 9.6 pag 257 (formula 9.12).

Parameters

  • E_mat (torch.Tensor) – The essential matrix with shape of \((*, 3, 3)\).

  • K1 (torch.Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).

  • K2 (torch.Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).

Returns

The fundamental matrix with shape \((*, 3, 3)\).

Return type

torch.Tensor

fundamental_from_projections(P1: torch.Tensor, P2: torch.Tensor) → torch.Tensor[source]

Get the Fundamental matrix from Projection matrices.

Parameters

  • P1 (torch.Tensor) – The projection matrix from first camera with shape \((*, 3, 4)\).

  • P2 (torch.Tensor) – The projection matrix from second camera with shape \((*, 3, 4)\).

Returns

The fundamental matrix with shape \((*, 3, 3)\).

Return type

torch.Tensor

compute_correspond_epilines(points: torch.Tensor, F_mat: torch.Tensor) → torch.Tensor[source]

Computes the corresponding epipolar line for a given set of points.

Parameters

  • points (torch.Tensor) – tensor containing the set of points to project in the shape of \((B, N, 2)\).

  • F_mat (torch.Tensor) – the fundamental to use for projection the points in the shape of \((B, 3, 3)\).

Returns

a tensor with shape \((B, N, 3)\) containing a vector of the epipolar lines corresponding to the points to the other image. Each line is described as \(ax + by + c = 0\) and encoding the vectors as \((a, b, c)\).

Return type

torch.Tensor

normalize_points(points: torch.Tensor, eps: float = 1e-08) → Tuple[torch.Tensor, torch.Tensor][source]

Normalizes points (isotropic).

Computes the transformation matrix such that the two principal moments of the set of points are equal to unity, forming an approximately symmetric circular cloud of points of radius 1 about the origin. Reference: Hartley/Zisserman 4.4.4 pag.107

This operation is an essential step before applying the DLT algorithm in order to consider the result as optimal.

Parameters

  • points (torch.Tensor) – Tensor containing the points to be normalized with shape \((B, N, 2)\).

  • eps (float) – epsilon value to avoid numerical instabilities. Default: 1e-8.

Returns

tuple containing the normalized points in the shape \((B, N, 2)\) and the transformation matrix in the shape \((B, 3, 3)\).

Return type

Tuple[torch.Tensor, torch.Tensor]

normalize_transformation(M: torch.Tensor, eps: float = 1e-08) → torch.Tensor[source]

Normalizes a given transformation matrix.

The function trakes the transformation matrix and normalize so that the value in the last row and column is one.

Parameters

  • M (torch.Tensor) – The transformation to be normalized of any shape with a minimum size of 2x2.

  • eps (float) – small value to avoid unstabilities during the backpropagation.

Returns

the normalized transformation matrix with same shape as the input.

Return type

torch.Tensor

Metrics

sampson_epipolar_distance(pts1: torch.Tensor, pts2: torch.Tensor, Fm: torch.Tensor, squared: bool = True, eps: float = 1e-08) → torch.Tensor[source]

Returns Sampson distance for correspondences given the fundamental matrix.

Parameters

  • pts1 (torch.Tensor) – correspondences from the left images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.

  • pts2 (torch.Tensor) – correspondences from the right images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.

  • Fm (torch.Tensor) – Fundamental matrices with shape \((B, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.

  • squared (bool) – if True (default), the squared distance is returned.

  • eps (float) – Small constant for safe sqrt. Default 1e-9.

Returns

the computed Sampson distance with shape \((B, N)\).

Return type

torch.Tensor

symmetrical_epipolar_distance(pts1: torch.Tensor, pts2: torch.Tensor, Fm: torch.Tensor, squared: bool = True, eps: float = 1e-08) → torch.Tensor[source]

Returns symmetrical epipolar distance for correspondences given the fundamental matrix.

Parameters

  • pts1 (torch.Tensor) – correspondences from the left images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.

  • pts2 (torch.Tensor) – correspondences from the right images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.

  • Fm (torch.Tensor) – Fundamental matrices with shape \((B, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.

  • squared (bool) – if True (default), the squared distance is returned.

  • eps (float) – Small constant for safe sqrt. Default 1e-9.

Returns

the computed Symmetrical distance with shape \((B, N)\).

Return type

torch.Tensor

Projection

projection_from_KRt(K: torch.Tensor, R: torch.Tensor, t: torch.Tensor) → torch.Tensor[source]

Get the projection matrix P from K, R and t.

This function estimate the projection matrix by solving the following equation: \(P = K * [R|t]\).

Parameters

  • K (torch.Tensor) – the camera matrix with the instrinsics with shape \((B, 3, 3)\).

  • R (torch.Tensor) – The rotation matrix with shape \((B, 3, 3)\).

  • t (torch.Tensor) – The translation vector with shape \((B, 3, 1)\).

Returns

The projection matrix P with shape \((B, 4, 4)\).

Return type

torch.Tensor

projections_from_fundamental(F_mat: torch.Tensor) → torch.Tensor[source]

Get the projection matrices from the Fundamental Matrix.

Parameters

F_mat (torch.Tensor) – the fundamental matrix with the shape \((*, 3, 3)\).

Returns

The projection matrices with shape \((*, 4, 4, 2)\).

Return type

torch.Tensor

intrinsics_like(focal: float, input: torch.Tensor) → torch.Tensor[source]

Returns a 3x3 instrinsics matrix, with same size as the input.

The center of projection will be based in the input image size.

Parameters

  • focal (float) – the focal length for tha camera matrix.

  • input (torch.Tensor) – image tensor that will determine the batch size and image height and width. It is assumed to be a tensor in the shape of \((B, C, H, W)\).

Returns

The camera matrix with the shape of \((B, 3, 3)\).

Return type

torch.Tensor

scale_intrinsics(camera_matrix: torch.Tensor, scale_factor: Union[float, torch.Tensor]) → torch.Tensor[source]

Scale a camera matrix containing the intrinsics.

Applies the scaling factor to the focal length and center of projection.

Parameters

  • camera_matrix (torch.Tensor) – the camera calibration matrix containing the intrinsic parameters. The expected shape for the tensor is \((B, 3, 3)\).

  • scale_factor (Union[float, torch.Tensor]) – the scaling factor to be applied.

Returns

The scaled camera matrix with shame shape as input \((B, 3, 3)\).

Return type

torch.Tensor

random_intrinsics(low: Union[float, torch.Tensor], high: Union[float, torch.Tensor]) → torch.Tensor[source]

Generates a random camera matrix based on a given uniform distribution.

Parameters
Returns

The random camera matrix with the shape of \((1, 3, 3)\).

Return type

torch.Tensor

Numeric

cross_product_matrix(x: torch.Tensor) → torch.Tensor[source]

Returns the cross_product_matrix symmetric matrix of a vector.

Parameters

x (torch.Tensor) – The input vector to construct the matrix in the shape \((B, 3)\).

Returns

The constructed cross_product_matrix symmetric matrix with shape \((B, 3, 3)\).

Return type

torch.Tensor

eye_like(n: int, input: torch.Tensor) → torch.Tensor[source]

Returns a 2-D tensor with ones on the diagonal and zeros elsewhere with same size as the input.

Parameters

  • n (int) – the number of rows \((N)\).

  • input (torch.Tensor) – image tensor that will determine the batch size of the output matrix. The expected shape is \((B, *)\).

Returns

The identity matrix with same size as input \((*, N, N)\).

Return type

torch.Tensor

vec_like(n, tensor)[source]

Returns a 2-D tensor with a vector containing zeros with same size as the input.

Parameters

  • n (int) – the number of rows \((N)\).

  • input (torch.Tensor) – image tensor that will determine the batch size of the output matrix. The expected shape is \((B, *)\).

Returns

The vector with same size as input \((*, N, 1)\).

Return type

torch.Tensor

Triangulation

triangulate_points(P1: torch.Tensor, P2: torch.Tensor, points1: torch.Tensor, points2: torch.Tensor) → torch.Tensor[source]

Reconstructs a bunch of points by triangulation.

Triangulates the 3d position of 2d correspondences between several images. Reference: Internally it uses DLT method from Hartley/Zisserman 12.2 pag.312

The input points are assumed to be in homogeneous coordinate system and being inliers correspondences. The method does not perform any robust estimation.

Parameters

  • P1 (torch.Tensor) – The projection matrix for the first camera with shape \((*, 3, 4)\).

  • P2 (torch.Tensor) – The projection matrix for the second camera with shape \((*, 3, 4)\).

  • points1 (torch.Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).

  • points2 (torch.Tensor) – The set of points seen from the second camera frame in the camera plane coordinates with shape \((*, N, 2)\).

Returns

The reconstructed 3d points in the world frame with shape \((*, N, 3)\).

Return type

torch.Tensor