kornia.geometry.ransac#

Robust model fitting with RANSAC (Random Sample Consensus), used to estimate affine transforms, homographies, fundamental and essential matrices from noisy correspondences.

Affine transforms#

model_type="affine" estimates a full 2D affine transform, including translation, from three non-collinear source points. It returns a source-to-target (3, 3) matrix with bottom row [0, 0, 1] and a boolean (N,) inlier mask. inl_th is a Euclidean distance in the target points’ units. Affine fitting and residual scoring compute in at least float32, even inside an autocast context, and the returned matrix retains the input dtype. At least four inliers are required to accept a model; otherwise the result is the all-zero failure matrix and an all-false mask.

from kornia.geometry import RANSAC

estimator = RANSAC("affine", inl_th=2.0, seed=0)
matrix, inliers = estimator(points1, points2)
warp_matrix = matrix[:2]  # 2x3 form accepted by warp_affine after batching

Affine estimation supports uniform and PROSAC sampling, MSAC and inlier-count scoring, and the existing local_optimization="dlt" loop. Here the refit is linear least squares with free translation using find_affine(); "lm", compile=True and DEGENSAC are not supported. max_lo_iters=0 disables local optimization. Like the other model types, each call estimates one model for one correspondence set; batch_size batches hypotheses, not independent problems.

AdaLAM shares the linear fitting primitive but retains its specialized loop: its local 2x2 models are constrained by a matched seed, and its variable-size neighborhoods, sampling and confidence-based inlier selection are unchanged.

Batches, scores, and refinement#

Hypotheses are generated and verified in batches. The sampling budget is max_samples minimal samples when given, otherwise batch_size * max_iter (2048 * max_iter, the historical default, with batch_size="auto"); the last batch is truncated to it. Seven-point fundamental and five-point essential solvers may produce several candidate matrices from each set. Early stopping is checked between batches, and confidence=1 runs the whole budget.

model_type="fundamental" draws seven-point samples and completes each with up to three rank-2 matrices, the roots of the cubic det F = 0, like "fundamental_7pt"; "fundamental_8pt" draws eight-point samples. Seven correspondences make an all-inlier sample more likely than eight, so fewer samples reach the same confidence.

The default batch_size="auto" picks the batches per call. With the default local_optimization="lm" (see below), a CPU call starts with 256 samples, 512 for homographies, and doubles the batch after each one up to 2048, 4096 for homographies: inputs with many inliers stop after a small first batch, and the others pay the per-batch overhead only a few times. On CUDA and MPS the batch is the whole budget up to 8192 samples. Essential matrices start smaller on every device, with 64 samples up to 1024 on CPU and 256 up to 8192 on CUDA and MPS: a five-point sample needs few draws at high inlier ratios, and its eigenvalue solve runs on the host (on MPS, the whole solve). All shrink for many correspondences, down to 64 samples, so that above that floor a batch scores at most 2**22 (CPU) or 2**25 residuals. The rest of this paragraph describes local_optimization="dlt": on CUDA and MPS a homography batch costs about the same from a few hundred up to 8192 hypotheses, so the budget is drawn in batches of 8192; the epipolar solvers are compute-bound past 2048 hypotheses, so their batches stop there. Verification holds a batch x N residual matrix, so past 2**27 entries (about 1 GiB at peak in float32) the batch shrinks with N, down to 2048. Other devices retain the historical 2048-sample batch. A PROSAC batch of either size spans most of the growth schedule, whereas certifying a model after a small first batch drawn from a short prefix can stop on a poorly conditioned fit. On CPU the cost is linear in batch * N residuals and the eight-point solver costs about ten DLTs, so the batch aims at a millisecond or so of work: 256 to 2048 hypotheses for homographies and 128 to 512 for the epipolar models, fewer for more correspondences, which lets early stopping pay off. An integer batch_size fixes the batch on every device.

The default score_type="msac" minimizes the sum of squared residuals truncated at inl_th ** 2; "ransac" counts inliers instead. The returned internal score is normalized to increase with quality. Confidence stopping always uses the number of inliers, not this score. Fundamental and essential models use Sampson residuals. Essential estimation expects camera-normalized coordinates, so its threshold is not in pixels. Line-segment homographies use the squared mean distance from the transferred endpoints to the target segment’s line; zero-length target segments are outliers. Their local optimization re-weights segments by the same distance.

A model is accepted only with more inliers than its minimal sample (three correspondences for affine transforms, four for homographies, five for essential and seven or eight for fundamental matrices): a model fitted to a sample always fits that sample, so only further inliers show a consensus. Without one, the estimator returns the all-zero matrix and an empty mask.

prosac_sampling=True enables the per-sample growth schedule of Chum and Matas (CVPR 2005). Order the correspondences from best to worst before calling the estimator. For example, when smaller descriptor ratios indicate better matches:

order = ratios.argsort()
estimator = RANSAC("fundamental", score_type="msac", prosac_sampling=True, seed=0)
F, sorted_inliers = estimator(points1[order], points2[order])
inliers = sorted_inliers.new_empty(sorted_inliers.shape)
inliers[order] = sorted_inliers

The prefix grows within a batch, and each growth sample includes the newest correspondence. Stopping follows the paper’s termination-length test: a ranked prefix certifies the incumbent when its support there is non-random (a Chernoff bound on accidental inliers with probability 0.05, at significance 0.05) and the draws it needs for confidence fit inside the prefix’s growth interval, so that only draws from that prefix count. As in OpenCV’s USAC, a prefix shorter than min(N / 2, 100) or supporting fewer than 20% of all correspondences cannot terminate; the whole set is always a candidate, so PROSAC never runs longer than the uniform bound. Stopping is checked between batches. Uninformative or reversed rankings can hurt accuracy. The weights argument is not used to rank correspondences.

PROSAC pays off when the ranking tracks inlier-ness and the inlier ratio is low, where uniform sampling cannot stop early: on SIFT correspondences with a ratio test (HEB), it adds about 0.1 mAA at equal time for homographies. On learned matchers with 85% or more inliers, uniform sampling already stops after one batch, and the confidence-ranked prefixes can be spatially clustered, so the model PROSAC certifies first may fit them well and the pose poorly. Prefer uniform sampling there, or run PROSAC with confidence=1 when the extra draws are affordable.

Local optimization#

Homographies, fundamental and essential matrices default to local_optimization="lm". The sampling loop keeps the eight best-scoring minimal models of the whole run, in coordinates normalized once per call (the caller’s calibrated ones for essential matrices, see below). After sampling, all eight are refined together with max_lo_iters Levenberg-Marquardt iterations that minimize the squared Sampson distance or one-way transfer error of every correspondence, truncated at inl_th. The best-scoring model among the refits and the minimal models is then refined on its own inliers with refine_iters iterations of a Cauchy loss of scale inl_th / 3, the noise level of a threshold at three standard deviations, and the returned mask holds the inliers of the returned model after conversion to the input dtype. If rounding removes sufficient support, the call returns the all-zero model and empty inlier mask used for estimation failure. Fundamental matrices keep rank two through the parametrization of Bartoli and Sturm (TPAMI 2004); the refinement follows PoseLib’s refine_fundamental and refine_homography. Refining several models once sampling ends, rather than each new incumbent, is the batched form of PoseLib’s rule of refining every minimal model that improves on the best one so far. The homography and fundamental-matrix solvers take their null spaces from a partial-pivoted LU factorization, the five-point solver from Householder reflections, and scores come from one matrix product per batch, which keeps the per-hypothesis cost low on every device. The refinements are a few dozen small operations per iteration and run on the CPU in float64 for every device, where launch latency would dominate. Early stopping follows the support of the best minimal model.

On PhotoTourism pairs this raises the pose mAA of fundamental matrices by 0.04 to 0.1 over the subset refits below at the same sample budget. SIFT matches take a third of the time on CPU and half on CUDA; ALIKED matches with LightGlue, on which both stop early, take about 0.5 ms longer on CPU. On the HEB homographies it adds about 0.04 mAA at a quarter to a half of the time.

Essential matrices are estimated in the caller’s calibrated coordinates, without the normalization, which would take them off the essential manifold, and are returned with unit Frobenius norm and their largest entry positive. They are refined as E = U diag(1, 1, 0) V^T with five parameters, as many as the rotation and translation direction of PoseLib’s refine_relpose: rotations of U about three axes and of V about its first two, which leaves out the rotation of both about their third axes that does not change E. The five-point samples are solved with Nister’s method in float64: a Householder null space, the degree-ten polynomial from polynomial products, and its real roots from the eigenvalues of its companion matrix on the host. MPS, which has no float64, solves its samples on the host. On PhotoTourism pairs this raises the pose mAA of essential matrices by 0.03 to 0.07 over the subset refits below, in about half the time on CPU, and an eighth of it with 8000 SIFT features per image.

local_optimization="dlt", the only choice for line-segment homographies, refits the incumbent on its inliers. For the homography models the refit is the iteratively re-weighted least squares of find_homography_dlt_iterated(), whose Gaussian weights use inl_th as their standard deviation, so a correspondence at the threshold keeps weight exp(-1/2). A refit replaces the model when it raises the score, or when it ties it: a least-squares fit on the same support is more precise than the minimal-sample model. A refit that does not raise the score ends the refinement. This is iterative refitting, not the full LO-RANSAC algorithm with inner resampling and a threshold schedule. Set max_lo_iters=0 to disable it. By default, lo_sample_size=32 caps each randomized non-minimal refit: max_lo_iters independent 32-inlier subsets are fit in one solver batch, then the best accepted consensus gets a full-inlier refit. Consensus sets no larger than the cap use ordinary iterative full-inlier refitting, which lo_sample_size=None selects for every consensus set. This bounded variant is inspired by Lebeda et al. (BMVC 2012); it does not implement the complete LO+ algorithm. The cap must be at least the non-minimal solver’s sample size (eight for fundamental/essential matrices). On PhotoTourism fundamental matrices the subset refits match or beat full-inlier refitting at a lower cost on both CPU and CUDA.

Compiled estimation#

compile=True (torch 2.14 or later) runs the whole local_optimization="lm" estimation as one torch.compile graph on CPU and CUDA. The sampling loop is a torch.while_loop that carries the pool of the best minimal models, the early-stopping bound is computed inside the graph, and the batch sizes and the number of models that survive the degeneracy tests are dynamic. The graph does not depend on the number of correspondences, the threshold, the confidence or the sample budget: it is traced once per model type, score type, iteration counts, dtype and device. The first call compiles it, in 30 s for homographies to 60 s for essential matrices on an Apple M1, and saves the compiled function next to inductor’s cache (TORCHINDUCTOR_CACHE_DIR); later processes load it in about a second. KORNIA_RANSAC_AOT=0 disables that file, and each process then compiles again, in 17 to 35 s once inductor’s kernels are cached. Ordinary calls and calls under torch.inference_mode() share the same artifact. Compiled arithmetic also runs with autocast disabled, using the input’s working precision and float64 host refinement. The artifact cache distinguishes CPU thread counts, default dtypes and default devices; changing these settings may require another first-call compilation. On CPU, large explicit sampling batches are split so scoring holds at most 2**22 residuals per batch (or one hypothesis when N alone exceeds that limit). This keeps the total sample budget and can check confidence stopping earlier than the requested batch size would. The sampling keys are a counter-based hash of the call’s seed and each sample’s index, computed inside the graph, and Floyd’s algorithm turns them into subsets on every device, where compilation fuses its steps into one kernel. A sample’s keys therefore depend neither on batch boundaries nor on the device, calls with different seeds do not share samples, and seeded calls never touch a global generator, including during concurrent calls. An unseeded call draws one seed from the input device’s global generator and then hashes it as a seeded call does. On CUDA the loop counters and stopping bound stay on the host, with a transfer of the leading score and inlier count after each batch; compilation therefore does not remove every host-device synchronization.

For the initial implementation at revision ad91a621, on that CPU (4 threads, synthetic scenes of 500 to 5000 correspondences with 20% or 50% inliers, averaged over seeds), compiled calls were 1.2 to 1.7 times faster for homographies, 1.6 to 2.4 times for fundamental matrices and 1.1 to 1.6 times for essential matrices, whose five-point solver gained nothing: its time goes to LAPACK factorizations and the eigenvalues of the companion matrix. The compiled call runs the same algorithm with its own random stream, so a seeded call is reproducible but differs from the eager one. On synthetic scenes of 500 and 2000 correspondences with 20% to 50% inliers its mean inlier recall was within 0.015 of eager’s for homographies and essential matrices and within 0.03 for fundamental matrices, where eager runs with other seeds differ from each other as much (60 seeds per case at 20% and 30% inliers). PROSAC sampling, degensac=True and local_optimization="dlt" are not supported. The current sampler draws a different seeded stream; use benchmarks/geometry/ransac_compile_synthetic.py to measure current latency and recovery, including --confidence 1 for equal sample budgets.

Dominant planes#

A seven-point sample with five correspondences on one plane yields a fundamental matrix consistent with the whole plane, whatever its two other correspondences are (Chum, Werner and Matas, CVPR 2005). When most inliers lie on one plane, such a wrong model can have the most support and stop the sampling, and no refinement recovers the off-plane inliers it never counted. "fundamental" and "fundamental_7pt" support opt-in DEGENSAC (degensac=True): record-setting samples are tested for this degeneracy, and a degenerate one hands its plane homography to a plane-and-parallax search over the correspondences off the plane, whose best model, refined, competes with the minimal ones. Each plane is searched once per call. sampson_homography_distance() measures the plane. The test’s tolerance, taken from Chum’s implementation, also flags samples in scenes without a dominant plane, so the estimate for a given seed can change there too. degensac=False (the default) and degensac=None leave it off, retaining the plain seven-point behavior.

class kornia.geometry.ransac.RANSAC(model_type='homography', inl_th=2.0, batch_size='auto', max_iter=10, confidence=0.99, max_lo_iters=5, score_type='msac', prosac_sampling=False, seed=None, lo_sample_size=32, max_samples=None, local_optimization=None, refine_iters=3, degensac=False, compile=False)[source]#

Module for robust geometry estimation with RANSAC. https://en.wikipedia.org/wiki/Random_sample_consensus.

Convention:

kp1 and kp2 are passed as the first- and second-image arguments of the estimator selected by model_type, so a homography maps kp1 to kp2. forward returns a (3, 3) model and an (N,) bool inlier mask, or an all-zero model and no inliers when no candidate has more inliers than its minimal sample (three correspondences for affine transforms, four for homographies, five for "essential", seven for "fundamental" and "fundamental_7pt", eight for "fundamental_8pt").
"fundamental" draws seven-point samples, like "fundamental_7pt"; "fundamental_8pt" draws eight-point ones.
inl_th is in the keypoints’ own units (pixels, or calibrated units for "essential"): a correspondence is an inlier when its one-way transfer error for "homography" or "affine", its Sampson distance for the fundamental models and "essential", or the mean distance of its transferred endpoints from the image-2 segment’s line for "homography_from_linesegments" is at most inl_th. Two-view geometry compares this with OpenCV.
score_type="msac" (the default) ranks candidates by sum(1 - min(e / inl_th**2, 1)) over the squared errors e; acceptance and early stopping count inliers for either score.
local_optimization="lm", the default for homographies, fundamental and essential matrices, keeps the eight best-scoring minimal models of the whole run. After sampling, it refines them together with max_lo_iters Levenberg-Marquardt iterations on all correspondences with the squared error truncated at inl_th, keeps the best-scoring refit or minimal model, and refines that one on its inliers with refine_iters iterations of a Cauchy loss of scale inl_th / 3. The returned mask holds the inliers of the returned model after conversion to the input dtype; if that conversion loses sufficient support, the call returns the all-zero failure result. Early stopping follows the support of the best minimal model. The refinements run on the CPU in float64 whatever the device of the correspondences: they are a few dozen small operations per iteration, which launch latency makes slower on an accelerator. Homographies and fundamental matrices are estimated on Hartley-normalized correspondences; essential matrices in the caller’s calibrated coordinates, which a normalization would take off the essential manifold, refined on that manifold and returned with unit Frobenius norm.
local_optimization="dlt", the only choice for "affine" and line segments, refits each new best model from its inliers: with the default lo_sample_size=32, max_lo_iters randomized refits on 32-inlier subsets followed by one full-inlier refit; lo_sample_size=None refits all inliers iteratively. Affine models use a linear least-squares refit with free translation and return a homogeneous matrix with bottom row [0, 0, 1]; their minimal samples need three non-collinear source points.
degensac=True, an opt-in for "fundamental" and "fundamental_7pt" with local_optimization="lm", runs DEGENSAC (Chum, Werner and Matas, CVPR 2005). Every seven-point model that sets a new record among the raw scores is tested for an H-degenerate sample: five or more of its seven correspondences related by one homography. Such a model fits the dominant plane and whatever happens to agree with it off the plane, so the homography is refined, and fundamental matrices are drawn from it and pairs of correspondences off the plane (plane and parallax). The best of them, refined, joins the eight-model pool and, when it outscores the incumbent, sets the stopping bound. Thresholds and iteration counts follow Chum’s implementation in pydegensac; the draws come from a private host generator. A plane already searched in the call is not searched again: a sample whose homography’s inliers lie 95% or more inside it is skipped before refinement, and a refined plane whose inliers match it (Jaccard index 0.95 or more) before the search. The default, degensac=False, keeps the plain seven-point loop. Explicit recovery reproduces that result exactly when no record-setting sample is degenerate. Chum’s tolerance, three times the squared threshold for five of the seven correspondences, also flags samples in many scenes without a dominant plane; there the recovered models only join the competition.
prosac_sampling=True expects correspondences sorted best-first and stops with PROSAC’s termination-length test; confidence=1 runs the whole batch_size * max_iter budget.
A seeded call uses a private generator and leaves torch’s global RNG state unchanged; seed=None draws the minimal samples from the global generator. DEGENSAC’s recovery draws always come from a private host generator, which an unseeded call seeds from the sample that triggers the recovery.
compile=True runs the whole local_optimization="lm" estimation as one torch.compile graph (torch 2.14 or later, CPU and CUDA). The graph is traced once per configuration and does not depend on the number of correspondences, the threshold, the confidence or the budget; it is also saved next to inductor’s cache (TORCHINDUCTOR_CACHE_DIR), so a later process loads it instead of compiling. It runs the same algorithm with its own random stream: a seeded call is reproducible and leaves every global generator unchanged, while an unseeded call consumes one global random seed. Its result is not the eager one. PROSAC sampling and DEGENSAC are not supported.

Parameters:

model_type (str, optional) – “affine”, “homography”, “fundamental”, “fundamental_7pt”, “fundamental_8pt”, “essential”, or “homography_from_linesegments”. Default: "homography"
inl_th (float, optional) – positive inlier threshold, in the units given above. Default: 2.0
batch_size (Union[int, str], optional) – hypotheses generated and verified at once, or "auto" to pick the batches per call from the device and the number of correspondences (see resolve_batch_size()). Default: "auto"
max_iter (int, optional) – with an integer batch_size, the maximum number of batches, for a budget of batch_size * max_iter minimal samples; with "auto", the budget is 2048 * max_iter. The seven- and five-point solvers can return multiple models per sample. Default: 10
confidence (float, optional) – stopping confidence in (0, 1]; 1 disables early stopping. Default: 0.99
max_lo_iters (int, optional) – local optimization iterations: Levenberg-Marquardt iterations with local_optimization="lm", refits with "dlt"; zero disables local optimization. Default: 5
score_type (str, optional) – “msac” (default) for truncated squared residuals, or “ransac” for support count. Default: "msac"
prosac_sampling (bool, optional) – use PROSAC sampling on best-first ordered correspondences. The growth schedule advances per sampled set within each batch; stopping tests the incumbent’s support within ranked prefixes (Chum and Matas, 2005, section 2.2) as well as within the whole set. It pays off when the ranking tracks inlier-ness and the inlier ratio is low (ratio-tested SIFT). On learned matches with 85% or more inliers it can certify a model fitted to a spatially clustered top-ranked prefix after one batch and score below uniform sampling; use uniform sampling or confidence=1 there. Default: False
seed (Optional[int], optional) – optional seed, reset on each call for reproducible estimation on the same device. Default: None
lo_sample_size (Optional[int], optional) – with local_optimization="dlt", the inlier-subset size for a batch of max_lo_iters randomized local refits followed by a full-inlier refit (default 32). None uses iterative full-inlier refitting. Unused with "lm". Default: 32
max_samples (Optional[int], optional) – optional budget of minimal samples that overrides the one implied by batch_size and max_iter; the last batch is truncated to it. Default: None
local_optimization (Optional[str], optional) – "lm" or "dlt", as described above; None picks "lm" for homographies, fundamental and essential matrices and "dlt" for affine transforms and line segments. Default: None
refine_iters (int, optional) – Levenberg-Marquardt iterations of the final refinement with local_optimization="lm"; zero disables it. Default: 3
compile (bool, optional) – run local_optimization="lm" as one compiled graph, as described above. Default: False
degensac (Optional[bool], optional) – run DEGENSAC’s dominant-plane recovery, as described above. Defaults to False; None also leaves it off. True enables it for "fundamental" and "fundamental_7pt" with local_optimization="lm", the only supported combinations; True with any other raises ValueError. Default: False

forward(kp1, kp2, weights=None)[source]#

Call main forward method to execute the RANSAC algorithm.

Parameters:

kp1 (Tensor) – source image keypoints \((N, 2)\) (or line segments \((N, 2, 2)\)).
kp2 (Tensor) – target image keypoints with the same shape. For PROSAC, both inputs must be sorted best-first using the same correspondence-quality ordering. Essential estimation uses camera-normalized coordinates and a threshold in those units.
weights (Optional[Tensor], optional) – optional correspondences weights. Not used now. Default: None

Return type:

Tuple[Tensor, Tensor]

Returns:

Estimated model, shape of \((3, 3)\), or zeros if no valid model is found. A model needs more inliers than its minimal sample, since a model fitted to a sample fits that sample.
Boolean inlier mask, shape of \((N,)\), in the supplied correspondence order.

resolve_batch_size(num_tc, device)[source]#

Return the batch size of a local_optimization="dlt" call: the configured one, or the "auto" choice.

On CUDA and MPS a homography or affine batch costs about the same from a few hundred up to 8192 hypotheses (the four- and three-point solves are launch-bound), so the whole sample_budget is drawn in batches of up to 8192; the epipolar solvers are compute-bound past 2048 hypotheses, so their batches stop there and early stopping is checked in between. The verification holds a batch x N residual matrix and a few temporaries of that size, so past 2**27 entries (about 1 GiB at peak in float32) the batch shrinks with N, down to the 2048 of the historical fixed batch. On CPU the cost is linear in batch * N residuals, and the eight-point solver is about ten times a DLT, so the batch aims at a millisecond or so of work, 256 to 2048 hypotheses for homographies and 128 to 512 for affine and the epipolar models: a three-point affine sample needs far fewer draws than a four-point one, and the smaller batches let early stopping act sooner (1.1-4.3x faster at equal recall on 500-8000 synthetic matches with 10-50% inliers). Other devices retain the historical 2048-sample batch.

With local_optimization="lm" an "auto" batch instead starts at 256 samples on CPU (512 for homographies) and doubles after every batch, up to 2048 (4096 for homographies), so that inputs with many inliers stop after a small first batch while the rest pay the per-batch overhead a few times only. On CUDA and MPS it is the whole budget up to 8192 samples. Essential matrices start at 64 samples on CPU, up to 1024, and at 256 on CUDA and MPS, up to 8192: a five-point sample needs few draws at high inlier ratios, and its host eigenvalue solve costs the same on every device. All shrink, down to 64 samples, when a batch would score more than 2**22 (CPU) or 2**25 (accelerators) residuals, counting the three models of a seven-point sample and the ten candidate slots of a five-point one. On accelerators scoring holds two or three times that many entries at its peak, about 0.5 GiB in float32; on CPU it scores tiles of about a million residuals, independently of the batch. An integer batch_size is kept for every batch, except that compiled CPU RANSAC splits a batch whose candidate-by-correspondence score matrix would exceed the CPU memory bound. This preserves the sample budget, though the extra batch boundaries can change when confidence-based stopping is evaluated.

Return type:: int