kornia.geometry.epipolar

Module with useful functionalities for epipolar geometry used by Structure from Motion

_images/epipolar_geometry.svg.png

Essential

kornia.geometry.epipolar.essential_from_fundamental(F_mat, K1, K2)[source]

Get Essential matrix from Fundamental and Camera matrices.

Uses the method from Hartley/Zisserman 9.6 pag 257 (formula 9.12).

Parameters
  • F_mat (Tensor) – The fundamental matrix with shape of \((*, 3, 3)\).

  • K1 (Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).

  • K2 (Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).

Return type

Tensor

Returns

The essential matrix with shape \((*, 3, 3)\).

kornia.geometry.epipolar.essential_from_Rt(R1, t1, R2, t2)[source]

Get the Essential matrix from Camera motion (Rs and ts).

Reference: Hartley/Zisserman 9.6 pag 257 (formula 9.12)

Parameters
  • R1 (Tensor) – The first camera rotation matrix with shape \((*, 3, 3)\).

  • t1 (Tensor) – The first camera translation vector with shape \((*, 3, 1)\).

  • R2 (Tensor) – The second camera rotation matrix with shape \((*, 3, 3)\).

  • t2 (Tensor) – The second camera translation vector with shape \((*, 3, 1)\).

Return type

Tensor

Returns

The Essential matrix with the shape \((*, 3, 3)\).

kornia.geometry.epipolar.decompose_essential_matrix(E_mat)[source]

Decompose an essential matrix to possible rotations and translation.

This function decomposes the essential matrix E using svd decomposition [96] and give the possible solutions: \(R1, R2, t\).

Parameters

E_mat (Tensor) – The essential matrix in the form of \((*, 3, 3)\).

Return type

Tuple[Tensor, Tensor, Tensor]

Returns

A tuple containing the first and second possible rotation matrices and the translation vector. The shape of the tensors with be same input \([(*, 3, 3), (*, 3, 3), (*, 3, 1)]\).

kornia.geometry.epipolar.motion_from_essential(E_mat)[source]

Get Motion (R’s and t’s ) from Essential matrix.

Computes and return four possible poses exist for the decomposition of the Essential matrix. The possible solutions are \([R1,t], [R1,−t], [R2,t], [R2,−t]\).

Parameters

E_mat (Tensor) – The essential matrix in the form of \((*, 3, 3)\).

Return type

Tuple[Tensor, Tensor]

Returns

The rotation and translation containing the four possible combination for the retrieved motion. The tuple is as following \([(*, 4, 3, 3), (*, 4, 3, 1)]\).

kornia.geometry.epipolar.motion_from_essential_choose_solution(E_mat, K1, K2, x1, x2, mask=None)[source]

Recover the relative camera rotation and the translation from an estimated essential matrix.

The method checks the corresponding points in two images and also returns the triangulated 3d points. Internally uses decompose_essential_matrix() and then chooses the best solution based on the combination that gives more 3d points in front of the camera plane from triangulate_points().

Parameters
  • E_mat (Tensor) – The essential matrix in the form of \((*, 3, 3)\).

  • K1 (Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).

  • K2 (Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).

  • x1 (Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).

  • x2 (Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).

  • mask (Optional[Tensor], optional) – A boolean mask which can be used to exclude some points from choosing the best solution. This is useful for using this function with sets of points of different cardinality (for instance after filtering with RANSAC) while keeping batch semantics. Mask is of shape \((*, N)\). Default: None

Return type

Tuple[Tensor, Tensor, Tensor]

Returns

The rotation and translation plus the 3d triangulated points. The tuple is as following \([(*, 3, 3), (*, 3, 1), (*, N, 3)]\).

kornia.geometry.epipolar.relative_camera_motion(R1, t1, R2, t2)[source]

Compute the relative camera motion between two cameras.

Given the motion parameters of two cameras, computes the motion parameters of the second one assuming the first one to be at the origin. If \(T1\) and \(T2\) are the camera motions, the computed relative motion is \(T = T_{2}T^{−1}_{1}\).

Parameters
  • R1 (Tensor) – The first camera rotation matrix with shape \((*, 3, 3)\).

  • t1 (Tensor) – The first camera translation vector with shape \((*, 3, 1)\).

  • R2 (Tensor) – The second camera rotation matrix with shape \((*, 3, 3)\).

  • t2 (Tensor) – The second camera translation vector with shape \((*, 3, 1)\).

Return type

Tuple[Tensor, Tensor]

Returns

A tuple with the relative rotation matrix and translation vector with the shape of \([(*, 3, 3), (*, 3, 1)]\).

Fundamental

kornia.geometry.epipolar.find_fundamental(points1, points2, weights)[source]

Compute the fundamental matrix using the DLT formulation.

The linear system is solved by using the Weighted Least Squares Solution for the 8 Points algorithm.

Parameters
  • points1 (Tensor) – A set of points in the first image with a tensor shape \((B, N, 2)\).

  • points2 (Tensor) – A set of points in the second image with a tensor shape \((B, N, 2)\).

  • weights (Tensor) – Tensor containing the weights per point correspondence with a shape of \((B, N)\).

Return type

Tensor

Returns

the computed fundamental matrix with shape \((B, 3, 3)\).

kornia.geometry.epipolar.fundamental_from_essential(E_mat, K1, K2)[source]

Get the Fundamental matrix from Essential and camera matrices.

Uses the method from Hartley/Zisserman 9.6 pag 257 (formula 9.12).

Parameters
  • E_mat (Tensor) – The essential matrix with shape of \((*, 3, 3)\).

  • K1 (Tensor) – The camera matrix from first camera with shape \((*, 3, 3)\).

  • K2 (Tensor) – The camera matrix from second camera with shape \((*, 3, 3)\).

Return type

Tensor

Returns

The fundamental matrix with shape \((*, 3, 3)\).

kornia.geometry.epipolar.fundamental_from_projections(P1, P2)[source]

Get the Fundamental matrix from Projection matrices.

Parameters
  • P1 (Tensor) – The projection matrix from first camera with shape \((*, 3, 4)\).

  • P2 (Tensor) – The projection matrix from second camera with shape \((*, 3, 4)\).

Return type

Tensor

Returns

The fundamental matrix with shape \((*, 3, 3)\).

kornia.geometry.epipolar.compute_correspond_epilines(points, F_mat)[source]

Compute the corresponding epipolar line for a given set of points.

Parameters
  • points (Tensor) – tensor containing the set of points to project in the shape of \((B, N, 2)\).

  • F_mat (Tensor) – the fundamental to use for projection the points in the shape of \((B, 3, 3)\).

Return type

Tensor

Returns

a tensor with shape \((B, N, 3)\) containing a vector of the epipolar lines corresponding to the points to the other image. Each line is described as \(ax + by + c = 0\) and encoding the vectors as \((a, b, c)\).

kornia.geometry.epipolar.normalize_points(points, eps=1e-08)[source]

Normalizes points (isotropic).

Computes the transformation matrix such that the two principal moments of the set of points are equal to unity, forming an approximately symmetric circular cloud of points of radius 1 about the origin. Reference: Hartley/Zisserman 4.4.4 pag.107

This operation is an essential step before applying the DLT algorithm in order to consider the result as optimal.

Parameters
  • points (Tensor) – Tensor containing the points to be normalized with shape \((B, N, 2)\).

  • eps (float, optional) – epsilon value to avoid numerical instabilities. Default: 1e-08

Return type

Tuple[Tensor, Tensor]

Returns

tuple containing the normalized points in the shape \((B, N, 2)\) and the transformation matrix in the shape \((B, 3, 3)\).

kornia.geometry.epipolar.normalize_transformation(M, eps=1e-08)[source]

Normalize a given transformation matrix.

The function trakes the transformation matrix and normalize so that the value in the last row and column is one.

Parameters
  • M (Tensor) – The transformation to be normalized of any shape with a minimum size of 2x2.

  • eps (float, optional) – small value to avoid unstabilities during the backpropagation. Default: 1e-08

Return type

Tensor

Returns

the normalized transformation matrix with same shape as the input.

Metrics

kornia.geometry.epipolar.sampson_epipolar_distance(pts1, pts2, Fm, squared=True, eps=1e-08)[source]

Return Sampson distance for correspondences given the fundamental matrix.

Parameters
  • pts1 (Tensor) – correspondences from the left images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.

  • pts2 (Tensor) – correspondences from the right images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.

  • Fm (Tensor) – Fundamental matrices with shape \((B, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.

  • squared (bool, optional) – if True (default), the squared distance is returned. Default: True

  • eps (float, optional) – Small constant for safe sqrt. Default: 1e-08

Return type

Tensor

Returns

the computed Sampson distance with shape \((B, N)\).

kornia.geometry.epipolar.symmetrical_epipolar_distance(pts1, pts2, Fm, squared=True, eps=1e-08)[source]

Return symmetrical epipolar distance for correspondences given the fundamental matrix.

Parameters
  • pts1 (Tensor) – correspondences from the left images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.

  • pts2 (Tensor) – correspondences from the right images with shape (B, N, 2 or 3). If they are not homogeneous, converted automatically.

  • Fm (Tensor) – Fundamental matrices with shape \((B, 3, 3)\). Called Fm to avoid ambiguity with torch.nn.functional.

  • squared (bool, optional) – if True (default), the squared distance is returned. Default: True

  • eps (float, optional) – Small constant for safe sqrt. Default: 1e-08

Return type

Tensor

Returns

the computed Symmetrical distance with shape \((B, N)\).

Projection

kornia.geometry.epipolar.projection_from_KRt(K, R, t)[source]

Get the projection matrix P from K, R and t.

This function estimate the projection matrix by solving the following equation: \(P = K * [R|t]\).

Parameters
  • K (Tensor) – the camera matrix with the instrinsics with shape \((B, 3, 3)\).

  • R (Tensor) – The rotation matrix with shape \((B, 3, 3)\).

  • t (Tensor) – The translation vector with shape \((B, 3, 1)\).

Return type

Tensor

Returns

The projection matrix P with shape \((B, 4, 4)\).

kornia.geometry.epipolar.projections_from_fundamental(F_mat)[source]

Get the projection matrices from the Fundamental Matrix.

Parameters

F_mat (Tensor) – the fundamental matrix with the shape \((B, 3, 3)\).

Return type

Tensor

Returns

The projection matrices with shape \((B, 3, 4, 2)\).

kornia.geometry.epipolar.intrinsics_like(focal, input)[source]

Return a 3x3 instrinsics matrix, with same size as the input.

The center of projection will be based in the input image size.

Parameters
  • focal (float) – the focal length for the camera matrix.

  • input (Tensor) – image tensor that will determine the batch size and image height and width. It is assumed to be a tensor in the shape of \((B, C, H, W)\).

Return type

Tensor

Returns

The camera matrix with the shape of \((B, 3, 3)\).

kornia.geometry.epipolar.scale_intrinsics(camera_matrix, scale_factor)[source]

Scale a camera matrix containing the intrinsics.

Applies the scaling factor to the focal length and center of projection.

Parameters
  • camera_matrix (Tensor) – the camera calibration matrix containing the intrinsic parameters. The expected shape for the tensor is \((B, 3, 3)\).

  • scale_factor (Union[float, Tensor]) – the scaling factor to be applied.

Return type

Tensor

Returns

The scaled camera matrix with shame shape as input \((B, 3, 3)\).

kornia.geometry.epipolar.random_intrinsics(low, high)[source]

Generate a random camera matrix based on a given uniform distribution.

Parameters
Return type

Tensor

Returns

the random camera matrix with the shape of \((1, 3, 3)\).

Numeric

kornia.geometry.epipolar.cross_product_matrix(x)[source]

Return the cross_product_matrix symmetric matrix of a vector.

Parameters

x (Tensor) – The input vector to construct the matrix in the shape \((B, 3)\).

Return type

Tensor

Returns

The constructed cross_product_matrix symmetric matrix with shape \((B, 3, 3)\).

kornia.geometry.epipolar.eye_like(n, input)[source]

Return a 2-D tensor with ones on the diagonal and zeros elsewhere with same size as the input.

Parameters
  • n (int) – the number of rows \((N)\).

  • input (Tensor) – image tensor that will determine the batch size of the output matrix. The expected shape is \((B, *)\).

Return type

Tensor

Returns

The identity matrix with same size as input \((B, N, N)\).

kornia.geometry.epipolar.vec_like(n, tensor)[source]

Return a 2-D tensor with a vector containing zeros with same size as the input.

Parameters
  • n – the number of rows \((N)\).

  • input – image tensor that will determine the batch size of the output matrix. The expected shape is \((B, *)\).

Returns

The vector with same size as input \((B, N, 1)\).

Triangulation

kornia.geometry.epipolar.triangulate_points(P1, P2, points1, points2)[source]

Reconstructs a bunch of points by triangulation.

Triangulates the 3d position of 2d correspondences between several images. Reference: Internally it uses DLT method from Hartley/Zisserman 12.2 pag.312

The input points are assumed to be in homogeneous coordinate system and being inliers correspondences. The method does not perform any robust estimation.

Parameters
  • P1 (Tensor) – The projection matrix for the first camera with shape \((*, 3, 4)\).

  • P2 (Tensor) – The projection matrix for the second camera with shape \((*, 3, 4)\).

  • points1 (Tensor) – The set of points seen from the first camera frame in the camera plane coordinates with shape \((*, N, 2)\).

  • points2 (Tensor) – The set of points seen from the second camera frame in the camera plane coordinates with shape \((*, N, 2)\).

Return type

Tensor

Returns

The reconstructed 3d points in the world frame with shape \((*, N, 3)\).