kornia.sensors.camera¶

Warning

kornia.sensors.camera is an experimental API and is subject to change. Once finished, it will subsume kornia.geometry.camera

The objective of kornia.sensors.camera is to express well-known camera models such as Pinhole, Kannala Brandt, and others in terms of distortion and projection types while ensuring differentiability. We also aim to equip the user with tools to define custom camera models. As of now, only the Pinhole model is supported.

Defining a Pinhole camera model is as simple as:

from kornia.image import ImageSize
from kornia.sensors.camera import CameraModel, CameraModelType

params = torch.tensor([328., 328., 320., 240.]) # fx, fy, cx, cy
cam = CameraModel(ImageSize(480, 640), CameraModelType.PINHOLE, params)

To define a custom camera model based on distortion and projection types, one can use the CameraModelBase api:

from kornia.image import ImageSize
from kornia.sensors.camera import CameraModelBase
from kornia.sensors.camera.distortion_model import AffineTransform
from kornia.sensors.camera.projection_model import Z1Projection

params = torch.tensor([328., 328., 320., 240.])
cam = CameraModelBase(AffineTransform(), Z1Projection(), ImageSize(480, 640), params)

Note

At the moment, the only supported model is Pinhole. However, we plan to add Kannala Brandt, Orthographic, and other models in the near future.

class kornia.sensors.camera.CameraModelBase(distortion, projection, image_size, params)¶

Base class to represent camera models based on distortion and projection types.

Distortion is of 3 types:

Affine
Brown Conrady
Kannala Brandt K3

Projection is of 2 types:

Z1
Orthographic

Example

>>> params = torch.Tensor([328., 328., 320., 240.])
>>> cam = CameraModelBase(BrownConradyTransform(), Z1Projection(), ImageSize(480, 640), params)
>>> cam.params
tensor([328., 328., 320., 240.])

K()¶

Returns the camera matrix.

Return type:: Tensor

property cx: Tensor¶: Returns the principal point in x direction.

property cy: Tensor¶: Returns the principal point in y direction.

property fx: Tensor¶: Returns the focal length in x direction.

property fy: Tensor¶: Returns the focal length in y direction.

property height: int | Tensor¶: Returns the height of the image.

property image_size: ImageSize¶: Returns the image size of the camera model.

matrix()¶

Returns the camera matrix.

Return type:: Tensor

property params: Tensor¶: Returns the camera parameters.

project(points)¶

Projects 3D points to 2D camera plane.

Parameters:: points (Vector3) – Vector3 representing 3D points.
Return type:: Vector2
Returns:: Vector2 representing the projected 2D points.

Example

>>> points = Vector3(torch.Tensor([1.0, 1.0, 1.0]))
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.PINHOLE, torch.Tensor([328., 328., 320., 240.]))
>>> cam.project(points)
x: 648.0
y: 568.0

unproject(points, depth)¶

Unprojects 2D points from camera plane to 3D.

Parameters:

points (Vector2) – Vector2 representing 2D points.
depth (Tensor) – Depth of the points.

Return type:

Vector3

Returns:

Vector3 representing the unprojected 3D points.

Example

>>> points = Vector2(torch.Tensor([1.0, 1.0]))
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.PINHOLE, torch.Tensor([328., 328., 320., 240.]))
>>> cam.unproject(points, torch.Tensor([1.0]))
x: tensor([-0.9726])
y: tensor([-0.7287])
z: tensor([1.])

property width: int | Tensor¶: Returns the width of the image.

class kornia.sensors.camera.CameraModel(image_size, model_type, params)¶

Class to represent camera models.

Example

>>> # Pinhole Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.PINHOLE, torch.Tensor([328., 328., 320., 240.]))
>>> # Brown Conrady Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.BROWN_CONRADY, torch.Tensor([1.0, 1.0, 1.0, 1.0,
... 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]))
>>> # Kannala Brandt K3 Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.KANNALA_BRANDT_K3, torch.Tensor([1.0, 1.0, 1.0,
... 1.0, 1.0, 1.0, 1.0, 1.0]))
>>> # Orthographic Camera Model
>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.ORTHOGRAPHIC, torch.Tensor([328., 328., 320., 240.]))
>>> cam.params
tensor([328., 328., 320., 240.])

class kornia.sensors.camera.CameraModelType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)¶

class kornia.sensors.camera.PinholeModel(image_size, params)¶

Class to represent Pinhole Camera Model.

The pinhole camera model describes the mathematical relationship between the coordinates of a point in three-dimensional space and its projection onto the image plane of an ideal pinhole camera, where the camera aperture is described as a point and no lenses are used to focus light. See more: https://en.wikipedia.org/wiki/Pinhole_camera_model

Example

>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.PINHOLE, torch.Tensor([328., 328., 320., 240.]))
>>> cam
CameraModel(ImageSize(height=480, width=640), PinholeModel, tensor([328., 328., 320., 240.]))

matrix()¶

Returns the camera matrix.

The matrix is of the form: :rtype: Tensor

\[\begin{split}\begin{bmatrix} fx & 0 & cx \\ 0 & fy & cy \\ 0 & 0 & 1\end{bmatrix}\end{split}\]

Example

>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.PINHOLE, torch.Tensor([1.0, 2.0, 3.0, 4.0]))
>>> cam.matrix()
tensor([[1., 0., 3.],
        [0., 2., 4.],
        [0., 0., 1.]])

scale(scale_factor)¶

Scales the camera model by a scale factor.

Parameters:: scale_factor (Tensor) – Scale factor to scale the camera model.
Return type:: PinholeModel
Returns:: Scaled camera model.

Example

>>> cam = CameraModel(ImageSize(480, 640), CameraModelType.PINHOLE, torch.Tensor([328., 328., 320., 240.]))
>>> cam_scaled = cam.scale(2)
>>> cam_scaled.params
tensor([656., 656., 640., 480.])

Distortions¶

class kornia.sensors.camera.distortion_model.AffineTransform¶

distort(params, points)¶

Distort one or more Vector2 points using the affine transform.

Parameters:

params (Tensor) – Tensor representing the affine transform parameters.
points (Vector2) – Vector2 representing the points to distort.

Return type:

Vector2

Returns:

Vector2 representing the distorted points.

Example

>>> params = Tensor([1., 2., 3., 4.])
>>> points = Vector2.from_coords(1., 2.)
>>> AffineTransform().distort(params, points)
x: 4.0
y: 8.0

undistort(params, points)¶

Undistort one or more Vector2 points using the affine transform.

Parameters:

params (Tensor) – Tensor representing the affine transform parameters.
points (Vector2) – Vector2 representing the points to undistort.

Return type:

Vector2

Returns:

Vector2 representing the undistorted points.

Example

>>> params = Tensor([1., 2., 3., 4.])
>>> points = Vector2.from_coords(1., 2.)
>>> AffineTransform().undistort(params, points)
x: -2.0
y: -1.0

Projections¶

class kornia.sensors.camera.projection_model.Z1Projection¶

project(points)¶

Project one or more Vector3 from the camera frame into the canonical z=1 plane through perspective division.

Parameters:: points (Vector3) – Vector3 representing the points to project.
Return type:: Vector2
Returns:: Vector2 representing the projected points.

Example

>>> points = Vector3.from_coords(1., 2., 3.)
>>> Z1Projection().project(points)
x: 0.3333333432674408
y: 0.6666666865348816

unproject(points, depth)¶

Unproject one or more Vector2 from the canonical z=1 plane into the camera frame.

Parameters:

points (Vector2) – Vector2 representing the points to unproject.
depth (Tensor | float) – Tensor representing the depth of the points to unproject.

Return type:

Vector3

Returns:

Vector3 representing the unprojected points.

Example

>>> points = Vector2.from_coords(1., 2.)
>>> Z1Projection().unproject(points, 3)
x: tensor([3.])
y: tensor([6.])
z: tensor([3.])