Camera and world conventions across the ecosystem#
Kornia’s camera stack follows the OpenCV pinhole model: pixel coordinates are (u, v) = (x, y) =
(column, row) with integer pixel centres, extrinsics is the world-to-camera transform, and depth
is the camera-frame z. Several of the ecosystems below differ on at least one of those, and the difference
is silent: the shapes still match, the images still look plausible, and the reconstruction is merely half a
pixel off, or mirrored.
Read Conventions & Pitfalls first — it holds the library-wide tensor, coordinate and align_corners rules. This
page covers what changes when a calibration or a pose crosses a library boundary.
Pixel-centre conventions#
Convention |
Pixel (0, 0) center at |
Principal point of centered W×H image |
Used by |
|---|---|---|---|
OpenCV / “integer” |
|
|
OpenCV and Kalibr (as COLMAP’s FAQ states), Kornia |
COLMAP / “half-pixel” |
|
|
COLMAP |
|
normalized: corner pixels map to ±1 exactly |
— |
torch |
|
normalized: pixel areas span [−1, 1]; centers at ±(1−1/N) |
— |
torch default |
The “Used by” column names only projects whose own documentation states the convention; COLMAP’s FAQ covers OpenCV, Kalibr and COLMAP itself. Learned-reconstruction code often
fits neither row, so read the code: DUSt3R and VGGT enumerate integer pixel coordinates but place the principal
point at (W/2, H/2) (DUSt3R,
VGGT),
half a pixel off the first row. MoGe returns intrinsics normalized to the unit square on a half-pixel grid (MoGe),
which the third rule below converts.
Three rules follow from the first two rows:
cx_colmap = cx_opencv + 0.5, and the same forcy. Nothing else in the calibration moves: the focal lengths and the distortion coefficients are unaffected.Resizing an image by a scale
srescales the principal point differently in each convention — OpenCVcx' = s * cx + (s - 1) / 2, COLMAPcx' = s * cx. The two agree only ats = 1. Both formulas assume a resampler that keeps the image’s outer boundaries in place, which is whattorch.nn.functional.interpolate(align_corners=False)and thereforekornia.geometry.transform.resize()at its default do. Analign_corners=Trueresize keeps the corner pixel centres in place instead; the integer-centre rule is thencx' = cx * (W_out - 1) / (W_in - 1)with no offset, andfxscales by the same factor. For 5 → 3 columns,cx = 1becomes0.4under the first geometry and0.5under the second.Intrinsics normalized to the unit square with the first pixel centre at
(0.5/W, 0.5/H)are the half-pixel convention scaled by the image size. They convert to Kornia’s integer-centre pixels asfx_px = W * fx_n,cx_px = W * cx_n - 0.5, and the same forfy,cywithH(utils3d denormalize_intrinsics is that matrix). Multiplying by the size alone leaves the half-pixel offset in place.
Kornia’s grids are integer-centre: kornia.geometry.grid.create_meshgrid() with
normalized_coordinates=False enumerates 0 .. W-1 along x and 0 .. H-1 along y, so a centred
image has its principal point at cx = (W - 1) / 2, cy = (H - 1) / 2. That is what
kornia.geometry.camera.perspective.project_points(), kornia.geometry.depth.depth_to_3d() and the
rest of the functional camera API assume.
Warning
kornia.geometry.camera.pinhole.PinholeCamera.scale() (and scale_, and
kornia.sensors.camera.PinholeModel.scale()) rescale the principal point as cx' = s * cx — the COLMAP
rule — which disagrees with the integer pixel centres the rest of the library uses:
#4263.
Camera and world frames#
Each non-Kornia cell below is taken from, or derived from, the upstream documentation linked in the same row; a cell reads “not stated” when that documentation does not say.
Convention |
Camera axes (right / up / forward) |
Handedness |
World up |
Pixel centre |
Upstream reference |
Kornia converter |
|---|---|---|---|---|---|---|
OpenCV (Kornia’s own frame) |
+X right, +Y down, +Z forward |
right-handed |
not stated |
integer |
native — no conversion |
|
COLMAP |
+X right, +Y bottom, +Z front |
right-handed |
not stated |
half-pixel |
|
|
OpenGL, ARCore |
+X right, +Y up, −Z forward |
right-handed |
OpenGL: set by the |
OpenGL: half-pixel with a lower-left window origin: the lower-left pixel is centred at |
gluLookAt, glFrustum, gl_FragCoord, glViewport; ARCore Camera, ARCore Pose (which also states OpenGL’s handedness) |
for the pose only: |
ARKit |
+X right, +Y up, +Z away from the device on the screen side — the camera looks along −Z |
right-handed |
oriented by the session configuration |
not stated |
||
PyTorch3D |
+X left, +Y up, +Z from us to the scene |
right-handed |
not stated |
corner-based: |
none — converter candidate |
Extrinsics semantics#
extrinsics in Kornia is the world-to-camera [R | t], the OpenCV and COLMAP semantics:
kornia.geometry.camera.pinhole.PinholeCamera.project() takes world points and computes K (R X + t).
COLMAP’s images.txt stores the same direction — “the projection from world to the camera coordinate system
of an image” — so its QW QX QY QZ TX TY TZ needs no inversion, only the quaternion-to-matrix step.
The inverse is the camera-to-world pose, whose translation column is the camera centre in world
coordinates. Convert in either direction with kornia.geometry.conversions.worldtocam_to_camtoworld_Rt() and
kornia.geometry.conversions.camtoworld_to_worldtocam_Rt() — their Convention: blocks document which
t means what on each side, and that both compute the rigid inverse (R^T, -R^T t) by transposition.
The graphics/vision converters take a camera-to-world pose, not a world-to-camera one; feeding the wrong direction in flips the wrong side of the product and is a silent error rather than an exception. Their docstrings carry the details.
Intrinsics layout#
The functional API (
kornia.geometry.camera.perspective.project_points(),kornia.geometry.depth.depth_to_3d(),kornia.geometry.calibration.undistort_points(), …) takes a row-major3x3Kwithfx = K[0, 0],fy = K[1, 1],cx = K[0, 2],cy = K[1, 2], andkornia.sensors.camera.CameraModelBaseexposes that same layout through itsK()method. The functional pinhole model has no skew, like OpenCV’s: the skew entryK[0, 1]is ignored bykornia.geometry.conversions.normalize_points_with_intrinsics(), its inverse andkornia.geometry.camera.perspective.project_points()alike. With a skewedK, applyinv(K)to homogeneous pixel coordinates, orKto normalized ones, directly.kornia.geometry.camera.pinhole.PinholeCamerainstead stores a4x4intrinsicswhose canonical form is the homogeneous embedding[[fx, 0, cx, 0], [0, fy, cy, 0], [0, 0, 1, 0], [0, 0, 0, 1]]. Only the top-left3x3block is read:project()andunproject()build their projection from it,intrinsics_inverse()returns the homogeneous4x4embedding of its inverse, andDepthWarperanddepth_warp()use both. So a non-zerointrinsics[0, 3]never enters a projection, and a3x3Kzero-padded withoutintrinsics[3, 3] = 1behaves exactly like its homogeneous embedding (#4771). Its class docstring documents the layout and the zero-padded case.Depth means two different things. It is the camera-frame
zby default, and the Euclidean ray length whenkornia.geometry.camera.perspective.unproject_points()is called withnormalize=True(thenormalize_pointsflags ofkornia.geometry.depth.depth_to_3d()andkornia.geometry.depth.depth_to_3d_v2()read it the same way). Monocular-depth networks disagree on which one they predict; the choice moves every unprojected point that is not on the principal ray.
grid_sample bridge#
Pixel coordinates reach torch.nn.functional.grid_sample() through normalized [-1, 1] coordinates, and
the last two rows of the pixel-centre table are the two ways to build them. Kornia’s normalized coordinates are
the align_corners=True reading — kornia.geometry.grid.create_meshgrid() returns that grid by default — so
a camera calibration expressed with integer pixel centres maps onto it without an offset, while the
align_corners=False reading shifts by half a pixel at the image borders.
OpenGL’s normalized device coordinates are the align_corners=False reading with the y axis reversed,
because the window origin is the lower-left corner: in a 5 × 3 image the centre of the top-left pixel is
(-0.8, +2/3) in OpenGL NDC, (-0.8, -2/3) under align_corners=False and (-1, -1) under
align_corners=True. The pose converters in the frames table change the camera basis only. A principal point
expressed in OpenGL window coordinates is half-pixel with y up: flip it with cy → H - cy, then subtract
0.5 from both coordinates — the COLMAP rule — to obtain Kornia’s integer-centre principal point.
Three camera-adjacent warps sample with align_corners=True:
kornia.geometry.calibration.undistort_image()has noalign_cornersparameter and remaps withalign_corners=True.kornia.geometry.depth.warp_frame_depth()has noalign_cornersparameter and samples withalign_corners=True.kornia.geometry.depth.DepthWarperandkornia.geometry.depth.depth_warp()do expose it, defaulting toTrue.
The library-wide table of align_corners defaults is on Conventions & Pitfalls.