Camera and world conventions across the ecosystem#

Kornia’s camera stack follows the OpenCV pinhole model: pixel coordinates are (u, v) = (x, y) = (column, row) with integer pixel centres, extrinsics is the world-to-camera transform, and depth is the camera-frame z. Several of the ecosystems below differ on at least one of those, and the difference is silent: the shapes still match, the images still look plausible, and the reconstruction is merely half a pixel off, or mirrored.

Read Conventions & Pitfalls first — it holds the library-wide tensor, coordinate and align_corners rules. This page covers what changes when a calibration or a pose crosses a library boundary.

Pixel-centre conventions#

Convention

Pixel (0, 0) center at

Principal point of centered W×H image

Used by

OpenCV / “integer”

(0.0, 0.0)

((W-1)/2, (H-1)/2)

OpenCV and Kalibr (as COLMAP’s FAQ states), Kornia

COLMAP / “half-pixel”

(0.5, 0.5)

(W/2, H/2)

COLMAP

grid_sample, align_corners=True

normalized: corner pixels map to ±1 exactly

—

torch grid_sample, kornia create_meshgrid(normalized_coordinates=True)

grid_sample, align_corners=False

normalized: pixel areas span [−1, 1]; centers at ±(1−1/N)

—

torch default

The “Used by” column names only projects whose own documentation states the convention; COLMAP’s FAQ covers OpenCV, Kalibr and COLMAP itself. Learned-reconstruction code often fits neither row, so read the code: DUSt3R and VGGT enumerate integer pixel coordinates but place the principal point at (W/2, H/2) (DUSt3R, VGGT), half a pixel off the first row. MoGe returns intrinsics normalized to the unit square on a half-pixel grid (MoGe), which the third rule below converts.

Three rules follow from the first two rows:

  • cx_colmap = cx_opencv + 0.5, and the same for cy. Nothing else in the calibration moves: the focal lengths and the distortion coefficients are unaffected.

  • Resizing an image by a scale s rescales the principal point differently in each convention — OpenCV cx' = s * cx + (s - 1) / 2, COLMAP cx' = s * cx. The two agree only at s = 1. Both formulas assume a resampler that keeps the image’s outer boundaries in place, which is what torch.nn.functional.interpolate(align_corners=False) and therefore kornia.geometry.transform.resize() at its default do. An align_corners=True resize keeps the corner pixel centres in place instead; the integer-centre rule is then cx' = cx * (W_out - 1) / (W_in - 1) with no offset, and fx scales by the same factor. For 5 → 3 columns, cx = 1 becomes 0.4 under the first geometry and 0.5 under the second.

  • Intrinsics normalized to the unit square with the first pixel centre at (0.5/W, 0.5/H) are the half-pixel convention scaled by the image size. They convert to Kornia’s integer-centre pixels as fx_px = W * fx_n, cx_px = W * cx_n - 0.5, and the same for fy, cy with H (utils3d denormalize_intrinsics is that matrix). Multiplying by the size alone leaves the half-pixel offset in place.

Kornia’s grids are integer-centre: kornia.geometry.grid.create_meshgrid() with normalized_coordinates=False enumerates 0 .. W-1 along x and 0 .. H-1 along y, so a centred image has its principal point at cx = (W - 1) / 2, cy = (H - 1) / 2. That is what kornia.geometry.camera.perspective.project_points(), kornia.geometry.depth.depth_to_3d() and the rest of the functional camera API assume.

Warning

kornia.geometry.camera.pinhole.PinholeCamera.scale() (and scale_, and kornia.sensors.camera.PinholeModel.scale()) rescale the principal point as cx' = s * cx — the COLMAP rule — which disagrees with the integer pixel centres the rest of the library uses: #4263.

Camera and world frames#

Each non-Kornia cell below is taken from, or derived from, the upstream documentation linked in the same row; a cell reads “not stated” when that documentation does not say.

Convention

Camera axes (right / up / forward)

Handedness

World up

Pixel centre

Upstream reference

Kornia converter

OpenCV (Kornia’s own frame)

+X right, +Y down, +Z forward

right-handed

not stated

integer

OpenCV calib3d

native — no conversion

COLMAP

+X right, +Y bottom, +Z front

right-handed

not stated

half-pixel

COLMAP output format, camera models, FAQ

kornia.geometry.conversions.camtoworld_to_worldtocam_Rt() and kornia.geometry.conversions.worldtocam_to_camtoworld_Rt() for the pose; kornia.geometry.conversions.ARKitQTVecs_to_ColmapQTVecs() from ARKit; the principal-point rule above has no helper

OpenGL, ARCore

+X right, +Y up, −Z forward

right-handed

OpenGL: set by the gluLookAt up vector; ARCore: not stated

OpenGL: half-pixel with a lower-left window origin: the lower-left pixel is centred at (0.5, 0.5), and NDC ±1 are the viewport edges, so OpenGL NDC is the align_corners=False row above with y reversed; ARCore: not stated

gluLookAt, glFrustum, gl_FragCoord, glViewport; ARCore Camera, ARCore Pose (which also states OpenGL’s handedness)

for the pose only: kornia.geometry.conversions.camtoworld_graphics_to_vision_4x4() / kornia.geometry.conversions.camtoworld_graphics_to_vision_Rt() and the inverse pair kornia.geometry.conversions.camtoworld_vision_to_graphics_4x4() / kornia.geometry.conversions.camtoworld_vision_to_graphics_Rt(); the image-coordinate change is under the grid_sample bridge below

ARKit

+X right, +Y up, +Z away from the device on the screen side — the camera looks along −Z

right-handed

oriented by the session configuration

not stated

ARCamera.transform

kornia.geometry.conversions.ARKitQTVecs_to_ColmapQTVecs()

PyTorch3D

+X left, +Y up, +Z from us to the scene

right-handed

not stated

corner-based: (0, 0) is the top-left corner of the top-left pixel

PyTorch3D cameras

none — converter candidate

Extrinsics semantics#

extrinsics in Kornia is the world-to-camera [R | t], the OpenCV and COLMAP semantics: kornia.geometry.camera.pinhole.PinholeCamera.project() takes world points and computes K (R X + t). COLMAP’s images.txt stores the same direction — “the projection from world to the camera coordinate system of an image” — so its QW QX QY QZ TX TY TZ needs no inversion, only the quaternion-to-matrix step.

The inverse is the camera-to-world pose, whose translation column is the camera centre in world coordinates. Convert in either direction with kornia.geometry.conversions.worldtocam_to_camtoworld_Rt() and kornia.geometry.conversions.camtoworld_to_worldtocam_Rt() — their Convention: blocks document which t means what on each side, and that both compute the rigid inverse (R^T, -R^T t) by transposition.

The graphics/vision converters take a camera-to-world pose, not a world-to-camera one; feeding the wrong direction in flips the wrong side of the product and is a silent error rather than an exception. Their docstrings carry the details.

Intrinsics layout#

grid_sample bridge#

Pixel coordinates reach torch.nn.functional.grid_sample() through normalized [-1, 1] coordinates, and the last two rows of the pixel-centre table are the two ways to build them. Kornia’s normalized coordinates are the align_corners=True reading — kornia.geometry.grid.create_meshgrid() returns that grid by default — so a camera calibration expressed with integer pixel centres maps onto it without an offset, while the align_corners=False reading shifts by half a pixel at the image borders.

OpenGL’s normalized device coordinates are the align_corners=False reading with the y axis reversed, because the window origin is the lower-left corner: in a 5 × 3 image the centre of the top-left pixel is (-0.8, +2/3) in OpenGL NDC, (-0.8, -2/3) under align_corners=False and (-1, -1) under align_corners=True. The pose converters in the frames table change the camera basis only. A principal point expressed in OpenGL window coordinates is half-pixel with y up: flip it with cy → H - cy, then subtract 0.5 from both coordinates — the COLMAP rule — to obtain Kornia’s integer-centre principal point.

Three camera-adjacent warps sample with align_corners=True:

The library-wide table of align_corners defaults is on Conventions & Pitfalls.