kornia.geometry.camera#

Projections#

kornia.geometry.camera.project_points_z1(points_in_camera)[source]#

Project one or more points from the camera frame into the canonical z=1 plane through perspective division.

\[\begin{split}\begin{bmatrix} u \\ v \\ w \end{bmatrix} = \begin{bmatrix} x \\ y \\ z \end{bmatrix} / z\end{split}\]

Note

This function has a precondition that the points are in front of the camera, i.e. z > 0. If this is not the case, the points will be projected to the canonical plane, but the resulting points will be behind the camera and causing numerical issues for z == 0.

Convention:

the input is a camera-frame point and the output its position on the canonical z = 1 plane, a normalized coordinate rather than a pixel; applying K with denormalize_points_with_intrinsics() gives the pixel.
the z > 0 precondition above is not validated.

Warning

The divide is plain at every z: at z = 0 a component is infinite, or nan for a zero numerator. project_points() instead skips its divide at abs(z) <= 1e-8 and returns finite pixels there, so the two disagree below that threshold: #4267.

Parameters:: points_in_camera (Tensor) – torch.Tensor representing the points to project with shape (…, 3).
Return type:: Tensor
Returns:: torch.Tensor representing the projected points with shape (…, 2).

Example

>>> points = torch.tensor([1., 2., 3.])
>>> project_points_z1(points)
tensor([0.3333, 0.6667])

kornia.geometry.camera.unproject_points_z1(points_in_cam_canonical, extension=None)[source]#

Unproject one or more points from the canonical z=1 plane into the camera frame.

\[\begin{split}\begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} u \\ v \end{bmatrix} \cdot w\end{split}\]

Convention:

extension is the camera-frame z of the unprojected point: the canonical point is multiplied by it and it becomes the third component. unproject() names the same argument depth. Both (...,) and (..., 1) extensions are accepted.

Parameters:

points_in_cam_canonical (Tensor) – torch.Tensor representing the points to unproject with shape (…, 2).
extension (Optional[Tensor], optional) – torch.Tensor representing the extension (depth) of the points to unproject with shape (…, 1) or (…), matching the points’ leading dimensions. Defaults to unit depth. Default: None

Return type:

Tensor

Returns:

torch.Tensor representing the unprojected points with shape (…, 3).

Example

>>> points = torch.tensor([1., 2.])
>>> extension = torch.tensor([3.])
>>> unproject_points_z1(points, extension)
tensor([3., 6., 3.])

kornia.geometry.camera.dx_project_points_z1(points_in_camera)[source]#

Compute the derivative of the x projection with respect to the x coordinate.

Returns point derivative of inverse depth point projection with respect to the x coordinate.

\[\begin{split}\frac{\partial \pi}{\partial x} = \begin{bmatrix} \frac{1}{z} & 0 & -\frac{x}{z^2} \\ 0 & \frac{1}{z} & -\frac{y}{z^2} \end{bmatrix}\end{split}\]

Note

This function has a precondition that the points are in front of the camera, i.e. z > 0. If this is not the case, the points will be projected to the canonical plane, but the resulting points will be behind the camera and causing numerical issues for z == 0.

Convention:

the result is the full Jacobian of project_points_z1() with shape (..., 2, 3), laid out row-major in the output index: the u row then the v row, each holding the derivatives with respect to x, y and z.
the same z > 0 precondition applies, and it is not validated either.

Parameters:: points_in_camera (Tensor) – torch.Tensor representing the points to project with shape (…, 3).
Return type:: Tensor
Returns:: torch.Tensor representing the derivative of the x projection with respect to the x coordinate with shape (…, 2, 3).

Example

>>> points = torch.tensor([1., 2., 3.])
>>> dx_project_points_z1(points)
tensor([[ 0.3333,  0.0000, -0.1111],
        [ 0.0000,  0.3333, -0.2222]])

kornia.geometry.camera.project_points_orthographic(points_in_camera)[source]#

Project points from the camera frame into the canonical z=1 plane through orthographic projection.

\[\begin{split}\begin{bmatrix} u \\ v \end{bmatrix} = \begin{bmatrix} x \\ y \\ z \end{bmatrix}\end{split}\]

Parameters:: points_in_camera (Tensor) – torch.Tensor representing the points to project.
Return type:: Tensor
Returns:: torch.Tensor representing the projected points.

Example

>>> points = torch.tensor([1., 2., 3.])
>>> project_points_orthographic(points)
tensor([1., 2.])

kornia.geometry.camera.unproject_points_orthographic(points_in_camera, extension)[source]#

Unproject one or more points from the canonical z=1 plane into the camera frame.

\[\begin{split}\begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} u \\ v \\ w \end{bmatrix}\end{split}\]

Convention:

extension becomes the z component of the result; both a (N,) and a (N, 1) extension are accepted.

Parameters:

points_in_camera (Tensor) – torch.Tensor representing the points to unproject with shape (…, 2).
extension (Tensor) – torch.Tensor representing the extension of the points to unproject with shape (…, 1).

Return type:

Tensor

Returns:

torch.Tensor representing the unprojected points with shape (…, 3).

Example

>>> points = torch.tensor([1., 2.])
>>> extension = torch.tensor([3.])
>>> unproject_points_orthographic(points, extension)
tensor([1., 2., 3.])

kornia.geometry.camera.dx_project_points_orthographic(points_in_camera)[source]#

Compute the derivative of the x projection with respect to the x coordinate.

\[\frac{\partial u}{\partial x} = 1\]

Parameters:: points_in_camera (Tensor) – torch.Tensor representing the points to project.
Return type:: Tensor
Returns:: torch.Tensor representing the derivative of the x projection with respect to the x coordinate.

Example

>>> points = torch.tensor([1., 2., 3.])
>>> dx_project_points_orthographic(points)
tensor([1.])

Distortion#

kornia.geometry.camera.distort_points_affine(projected_points_in_camera_z1_plane, params)[source]#

Distort one or more points from the canonical z=1 plane into the camera frame.

\[\begin{split}\begin{bmatrix} u \\ v \end{bmatrix} = \begin{bmatrix} f_x & 0 \\ 0 & f_y \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} + \begin{bmatrix} c_x \\ c_y \end{bmatrix}\end{split}\]

Convention:

projected_points_in_camera_z1_plane is a point on the normalized \(z = 1\) plane, not a pixel, and params is the flat vector [fx, fy, cx, cy]; the result is in pixels (integer pixel centres; see PinholeCamera).
distort_points() is the same camera parametrized by a \((*, 3, 3)\) K and an OpenCV coefficient vector; it takes a pixel input instead.
undistort_points_affine() is the closed-form inverse.

Parameters:

projected_points_in_camera_z1_plane (Tensor) – torch.Tensor representing the points to distort with shape (…, 2).
params (Tensor) – torch.Tensor representing the parameters of the affine distortion model with shape (…, 4).

Return type:

Tensor

Returns:

torch.Tensor representing the distorted points with shape (…, 2).

Example

>>> points = torch.tensor([319.5, 239.5])  # center of a 640x480 image
>>> params = torch.tensor([600., 600., 319.5, 239.5])
>>> distort_points_affine(points, params)
tensor([192019.5000, 143939.5000])

kornia.geometry.camera.undistort_points_affine(distorted_points_in_camera, params)[source]#

Undistort one or more points from the camera frame into the canonical z=1 plane.

\[\begin{split}\begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} f_x & 0 \\ 0 & f_y \end{bmatrix}^{-1} \left( \begin{bmatrix} u \\ v \end{bmatrix} - \begin{bmatrix} c_x \\ c_y \end{bmatrix} \right)\end{split}\]

Convention:

distorted_points_in_camera is a pixel coordinate and the result is a point on the normalized \(z = 1\) plane; params is the same [fx, fy, cx, cy] vector that distort_points_affine() takes, and this function is its closed-form inverse.
params values are not validated (a design choice shared with the other intrinsics consumers): a zero fx or fy gives inf or nan on that axis.

Parameters:

distorted_points_in_camera (Tensor) – torch.Tensor representing the points to undistort with shape (…, 2).
params (Tensor) – torch.Tensor representing the parameters of the affine distortion model with shape (…, 4).

Return type:

Tensor

Returns:

torch.Tensor representing the undistorted points with shape (…, 2).

Example

>>> points = torch.tensor([319.5, 239.5])  # center of a 640x480 image
>>> params = torch.tensor([600., 600., 319.5, 239.5])
>>> undistort_points_affine(points, params)
tensor([0., 0.])

kornia.geometry.camera.dx_distort_points_affine(projected_points_in_camera_z1_plane, params)[source]#

Compute the Jacobian of the affine distortion with respect to the point.

\[\begin{split}\frac{\partial (u, v)}{\partial (x, y)} = \begin{bmatrix} f_x & 0 \\ 0 & f_y \end{bmatrix}\end{split}\]

Convention:

the result is the \((..., 2, 2)\) Jacobian of distort_points_affine() with respect to the point: rows are the output components (u, v), columns the input components (x, y).
the map is affine, so that Jacobian is diag(fx, fy) and does not depend on the point.

Parameters:

projected_points_in_camera_z1_plane (Tensor) – torch.Tensor representing the points to distort with shape (…, 2).
params (Tensor) – torch.Tensor representing the parameters of the affine distortion model with shape (…, 4).

Return type:

Tensor

Returns:

torch.Tensor representing the Jacobian of the distortion with respect to the point with shape (…, 2, 2).

Example

>>> points = torch.tensor([319.5, 239.5])  # center of a 640x480 image
>>> params = torch.tensor([600., 600., 319.5, 239.5])
>>> dx_distort_points_affine(points, params)
tensor([[600.,   0.],
        [  0., 600.]])

kornia.geometry.camera.distort_points_kannala_brandt(projected_points_in_camera_z1_plane, params)[source]#

Distort points from the canonical z=1 plane into the camera frame using the Kannala-Brandt model.

Convention:

projected_points_in_camera_z1_plane is a point on the normalized \(z = 1\) plane, not a pixel, and the result is in pixels (integer pixel centres; see PinholeCamera).
params is the flat vector [fx, fy, cx, cy, k0, k1, k2, k3]: the first four are the affine part that distort_points_affine() takes on its own, and k0 to k3 multiply \(\theta^2\), \(\theta^4\), \(\theta^6\) and \(\theta^8\) in the fish-eye polynomial.
undistort_points_kannala_brandt() is the inverse map.

Parameters:

projected_points_in_camera_z1_plane (Tensor) – torch.Tensor representing the points to distort with shape (…, 2).
params (Tensor) – torch.Tensor representing the parameters of the Kannala-Brandt distortion model with shape (…, 8).

Return type:

Tensor

Returns:

torch.Tensor representing the distorted points with shape (…, 2).

Example

>>> points = torch.tensor([319.5, 239.5])  # center of a 640x480 image
>>> params = torch.tensor([1000.0, 1000.0, 320.0, 280.0, 0.1, 0.01, 0.001, 0.0001])
>>> distort_points_kannala_brandt(points, params)
tensor([1982.6832, 1526.3619])

kornia.geometry.camera.undistort_points_kannala_brandt(distorted_points_in_camera, params)[source]#

Undistort points from the camera frame into the canonical z=1 plane using the Kannala-Brandt model.

Convention:

distorted_points_in_camera is a pixel coordinate and the result is a point on the normalized \(z = 1\) plane; params is the same 8-element vector documented on distort_points_kannala_brandt().
the inverse is a fixed number of Gauss-Newton steps with no convergence test and no caller-visible step count, so the round trip through distort_points_kannala_brandt() closes only to the accuracy that iteration reaches.
a zero distorted radius maps to the origin.

Parameters:

distorted_points_in_camera (Tensor) – torch.Tensor representing the points to undistort with shape (…, 2).
params (Tensor) – torch.Tensor representing the parameters of the Kannala-Brandt distortion model with shape (…, 8).

Return type:

Tensor

Returns:

torch.Tensor representing the undistorted points with shape (…, 2).

Example

>>> points = torch.tensor([319.5, 239.5])  # center of a 640x480 image
>>> params = torch.tensor([1000.0, 1000.0, 320.0, 280.0, 0.1, 0.01, 0.001, 0.0001])
>>> undistort_points_kannala_brandt(points, params).shape
torch.Size([2])

kornia.geometry.camera.dx_distort_points_kannala_brandt(projected_points_in_camera_z1_plane, params)[source]#

Return the analytic Jacobian of the Kannala-Brandt distortion model.

Convention:

the result has shape \((..., 2, 2)\) and is laid out like the Jacobian that dx_distort_points_affine() returns: rows are the output components (u, v), columns the input components (x, y).
for squared radii <= 1e-8 the forward function uses its affine branch, so the Jacobian there is diag(fx, fy).

Parameters:

projected_points_in_camera_z1_plane (Tensor) – torch.Tensor representing the points to distort with shape (…, 2).
params (Tensor) – torch.Tensor representing the parameters of the Kannala-Brandt distortion model with shape (…, 8).

Return type:

Tensor

Returns:

torch.Tensor representing the derivative of the distortion with respect to the point with shape (…, 2, 2).

Example

>>> points = torch.tensor([1., 2.])
>>> params = torch.tensor([1000.0, 1000.0, 320.0, 280.0, 0.1, 0.01, 0.001, 0.0001])
>>> dx_distort_points_kannala_brandt(points, params)
tensor([[ 524.3779, -136.9029],
        [-136.9029,  319.0236]])