Stereo#

kornia.metrics.mean_absolute_disparity_error(input, target, valid_mask=None, reduction='mean')[source]#

Compute the mean absolute error (MAE) between two disparity maps, over the valid pixels.

mae = mean_absolute_disparity_error(input, target, valid_mask) takes two floating disparity maps in pixels with shape (*) and returns the mean of their absolute difference over the valid pixels \(\mathcal{V}\) as a 0-d tensor:

\[\text{MAE}(D, D^{gt}) = \frac{1}{|\mathcal{V}|}\sum_{p \in \mathcal{V}} |D_{p} - D^{gt}_{p}|\]

Convention:

'mean' and 'sum' pool every valid pixel of every image into one 0-d value, so for images with different numbers of valid pixels the result is not the mean of the per-image errors. The other disparity metrics pool the same way. If valid_mask selects no pixels, 'mean' returns nan and 'sum' returns 0.
valid_mask is broadcast to the maps from the right: an (H, W) mask applies to every image of a (B, H, W) batch and a (B, 1, 1) mask selects whole images, while a (B,) mask is matched against the last axis, not the batch. Nothing is masked automatically: a pixel with an invalid ground truth, such as a zero disparity, counts unless valid_mask excludes it.
A nan at a valid pixel of either map propagates into this error and the RMSE, while mean_bad_pixel_error() and kitti_d1_error() count it as a good pixel.
Half-precision sums are accumulated in float32 and the result is cast back to the input dtype, so 'mean' stays accurate on image-sized maps, while a float16 'sum' above 65504 saturates to inf: reduce with 'none' and sum in a wider dtype for totals that large.

Parameters:

input (Tensor) – predicted disparity map in pixels, with shape (*).
target (Tensor) – ground-truth disparity map with the shape of input.
valid_mask (Optional[Tensor], optional) – optional mask broadcastable to input; nonzero (True) values mark the pixels to evaluate. None evaluates all pixels. Default: None
reduction (str, optional) – 'mean' (default) averages over the valid pixels, 'sum' adds them and 'none' returns the per-pixel error map with masked-out positions set to zero. Default: "mean"

Return type:

Tensor

Returns:

the error as a 0-d tensor, or the per-pixel error map for reduction='none'.

See also

root_mean_squared_disparity_error(): the root of the mean squared error, which weights large errors more.
mean_bad_pixel_error(): the fraction of pixels above an error threshold.
kitti_d1_error(): the KITTI outlier fraction, with a relative criterion.

Examples

>>> input = torch.tensor([[0.0, 1.0], [2.0, 3.0]])
>>> target = torch.tensor([[0.0, 1.0], [2.0, 4.0]])
>>> mean_absolute_disparity_error(input, target)
tensor(0.2500)
>>> valid_mask = torch.tensor([[True, True], [True, False]])
>>> mean_absolute_disparity_error(input, target, valid_mask)
tensor(0.)

Reference:: D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 2002. https://vision.middlebury.edu/stereo/taxonomy-IJCV.pdf

kornia.metrics.root_mean_squared_disparity_error(input, target, valid_mask=None, reduction='mean')[source]#

Compute the root mean squared error (RMSE) between two disparity maps, over the valid pixels.

rmse = root_mean_squared_disparity_error(input, target, valid_mask) takes two floating disparity maps in pixels with shape (*) and returns the root of the mean squared difference over the valid pixels \(\mathcal{V}\) as a 0-d tensor:

\[\text{RMSE}(D, D^{gt}) = \sqrt{\frac{1}{|\mathcal{V}|}\sum_{p \in \mathcal{V}} (D_{p} - D^{gt}_{p})^{2}}\]

Convention:

See the Convention block of mean_absolute_disparity_error() for the pooling, the mask and the empty-mask result.
'sum' gives \(\sqrt{\sum e^2}\), not a sum of errors. The squared error is computed in float32 for half-precision inputs and only the final value is cast back, so an error above 256 px does not saturate float16.

Parameters:

input (Tensor) – predicted disparity map in pixels, with shape (*).
target (Tensor) – ground-truth disparity map with the shape of input.
valid_mask (Optional[Tensor], optional) – optional mask broadcastable to input; nonzero (True) values mark the pixels to evaluate. None evaluates all pixels. Default: None
reduction (str, optional) – 'mean' (default) averages the squared error over the valid pixels before the root, 'sum' adds it, and 'none' returns the per-pixel absolute error with masked-out positions set to zero. Default: "mean"

Return type:

Tensor

Returns:

the error as a 0-d tensor, or the per-pixel absolute error map for reduction='none'.

See also

mean_absolute_disparity_error(): the mean of the absolute errors, which weights large errors less.

Examples

>>> input = torch.zeros(2, 2)
>>> target = torch.tensor([[0.0, 0.0], [0.0, 1.0]])
>>> root_mean_squared_disparity_error(input, target)
tensor(0.5000)

Reference:: D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 2002. https://vision.middlebury.edu/stereo/taxonomy-IJCV.pdf

kornia.metrics.mean_bad_pixel_error(input, target, threshold=3.0, valid_mask=None, reduction='mean')[source]#

Compute the fraction of bad pixels between two disparity maps, over the valid pixels.

bad = mean_bad_pixel_error(input, target, threshold, valid_mask) takes two floating disparity maps in pixels with shape (*) and returns the fraction of valid pixels \(\mathcal{V}\) whose absolute error is strictly greater than threshold \(\tau\), a 0-d tensor in \([0, 1]\):

\[\text{Bad}_{\tau}(D, D^{gt}) = \frac{1}{|\mathcal{V}|}\sum_{p \in \mathcal{V}} [|D_{p} - D^{gt}_{p}| > \tau]\]

This is the bad-pixel percentage of the Middlebury stereo benchmark as a fraction. It is not the number KITTI reports: use kitti_d1_error().

Convention:

See the Convention block of mean_absolute_disparity_error() for the pooling, the mask, the empty-mask result and half-precision sums. 'sum' gives the number of bad pixels.
The output is built from comparisons, so it does not require grad and cannot be used as a loss.

Parameters:

input (Tensor) – predicted disparity map in pixels, with shape (*).
target (Tensor) – ground-truth disparity map with the shape of input.
threshold (float, optional) – the absolute error above which a pixel is bad. Default: 3.0
valid_mask (Optional[Tensor], optional) – optional mask broadcastable to input; nonzero (True) values mark the pixels to evaluate. None evaluates all pixels. Default: None
reduction (str, optional) – 'mean' (default) gives the fraction of bad pixels among the valid pixels, 'sum' their number, and 'none' the per-pixel bad-pixel map with masked-out positions set to zero. Default: "mean"

Return type:

Tensor

Returns:

the fraction as a 0-d tensor, or the per-pixel bad-pixel map for reduction='none'.

See also

kitti_d1_error(): adds a relative criterion.
mean_absolute_disparity_error(): the mean error in pixels, with no threshold.

Examples

>>> input = torch.zeros(2, 2)
>>> target = torch.tensor([[0.0, 1.0], [2.0, 4.0]])
>>> mean_bad_pixel_error(input, target, threshold=1.5)
tensor(0.5000)

Reference:: D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 2002. https://vision.middlebury.edu/stereo/taxonomy-IJCV.pdf

kornia.metrics.kitti_d1_error(input, target, abs_threshold=3.0, rel_threshold=0.05, valid_mask=None, reduction='mean')[source]#

Compute the KITTI D1 outlier fraction between two disparity maps, over the valid pixels.

d1 = kitti_d1_error(input, target, abs_threshold, rel_threshold, valid_mask) takes two floating disparity maps in pixels with shape (*) and returns the fraction of valid pixels \(\mathcal{V}\) that are outliers, a 0-d tensor in \([0, 1]\). A pixel is an outlier when its absolute error exceeds abs_threshold and its error relative to the ground truth exceeds rel_threshold:

\[\text{D1}(D, D^{gt}) = \frac{1}{|\mathcal{V}|}\sum_{p \in \mathcal{V}} \left[|D_{p} - D^{gt}_{p}| > \tau_{abs} \;\wedge\; \frac{|D_{p} - D^{gt}_{p}|}{|D^{gt}_{p}|} > \tau_{rel}\right]\]

Convention:

See the Convention block of mean_absolute_disparity_error() for the pooling, the mask, the empty-mask result and half-precision sums. 'sum' gives the number of outliers. Losses and metrics relates it to the KITTI devkit.
The relative error divides by target, so the two maps are not interchangeable: swapping them can change the result.
A zero ground-truth disparity gives a non-finite relative error, so that pixel is classified by abs_threshold alone and the output stays finite. KITTI marks these pixels invalid: pass valid_mask.
A nan in either map counts as an inlier, because every comparison against nan is false and both criteria must hold. Mask non-finite pixels out with valid_mask.
The output is built from comparisons, so it does not require grad and cannot be used as a loss.

Parameters:

input (Tensor) – predicted disparity map in pixels, with shape (*).
target (Tensor) – ground-truth disparity map with the shape of input.
abs_threshold (float, optional) – the absolute error above which a pixel may be an outlier. Default: 3.0
rel_threshold (float, optional) – the relative error above which a pixel may be an outlier. Default: 0.05
valid_mask (Optional[Tensor], optional) – optional mask broadcastable to input; nonzero (True) values mark the pixels to evaluate. None evaluates all pixels. Default: None
reduction (str, optional) – 'mean' (default) gives the fraction of outliers among the valid pixels, 'sum' their number, and 'none' the per-pixel outlier map with masked-out positions set to zero. Default: "mean"

Return type:

Tensor

Returns:

the fraction as a 0-d tensor, or the per-pixel outlier map for reduction='none'.

See also

mean_bad_pixel_error(): the absolute criterion alone, as Middlebury reports.
mean_absolute_disparity_error(): the mean absolute error, whose rules this one shares.

Examples

>>> input = torch.tensor([1.0, 5.0, 104.0, 20.0])
>>> target = torch.tensor([1.0, 1.0, 100.0, 10.0])
>>> kitti_d1_error(input, target)
tensor(0.5000)

Reference:: M. Menze and A. Geiger. Object Scene Flow for Autonomous Vehicles. CVPR 2015. https://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=stereo