kornia.geometry.bbox#

Functional API for 2D and 3D bounding boxes represented as corner tensors of shape \((B, 4, 2)\) and \((B, 8, 3)\). For an object-oriented interface with format conversions, see kornia.geometry.boxes.

kornia.geometry.bbox.bbox_generator(x_start, y_start, width, height)[source]#

Generate 2D bounding boxes according to the provided start coords, width and height.

Convention:: The far corner is at start + size - 1 (inclusive, see infer_bbox_shape()), so infer_bbox_shape() reads back width and height; a zero size puts the far corner one before the start. A scalar input gives a batch of one. The four tensors must share dtype and device, otherwise AssertionError; the output keeps a gradient path.

Warning

bbox_generator3d() places its far corner at start + size instead: #4018.

Parameters:

x_start (Tensor) – a tensor containing the x coordinates of the bounding boxes to be extracted. Shape must be a scalar tensor or \((B,)\).
y_start (Tensor) – a tensor containing the y coordinates of the bounding boxes to be extracted. Shape must be a scalar tensor or \((B,)\).
width (Tensor) – widths of the bounding boxes. Shape must be a scalar tensor or \((B,)\).
height (Tensor) – heights of the bounding boxes. Shape must be a scalar tensor or \((B,)\).

Return type:

Tensor

Returns:

the bounding box tensor, shape of \((B, 4, 2)\).

Examples

>>> x_start = torch.tensor([0, 1])
>>> y_start = torch.tensor([1, 0])
>>> width = torch.tensor([5, 3])
>>> height = torch.tensor([7, 4])
>>> bbox_generator(x_start, y_start, width, height)
tensor([[[0, 1],
         [4, 1],
         [4, 7],
         [0, 7]],

        [[1, 0],
         [3, 0],
         [3, 3],
         [1, 3]]])

kornia.geometry.bbox.bbox_generator3d(x_start, y_start, z_start, width, height, depth)[source]#

Generate 3D bounding boxes according to the provided start coords, width, height and depth.

Convention:: The four front vertices precede the four back vertices. A scalar input gives a batch of one. The six tensors must share dtype and device, otherwise AssertionError; the output keeps a gradient path.

Warning

The far corner is at start + size, one further than bbox_generator(), so infer_bbox_shape3d() reads back size + 1 on every axis: #4018.

Parameters:

x_start (Tensor) – a tensor containing the x coordinates of the bounding boxes to be extracted. Shape must be a scalar tensor or \((B,)\).
y_start (Tensor) – a tensor containing the y coordinates of the bounding boxes to be extracted. Shape must be a scalar tensor or \((B,)\).
z_start (Tensor) – a tensor containing the z coordinates of the bounding boxes to be extracted. Shape must be a scalar tensor or \((B,)\).
width (Tensor) – widths of the bounding boxes. Shape must be a scalar tensor or \((B,)\).
height (Tensor) – heights of the bounding boxes. Shape must be a scalar tensor or \((B,)\).
depth (Tensor) – depths of the bounding boxes. Shape must be a scalar tensor or \((B,)\).

Return type:

Tensor

Returns:

the 3d bounding box tensor \((B, 8, 3)\).

Examples

>>> x_start = torch.tensor([0, 3])
>>> y_start = torch.tensor([1, 4])
>>> z_start = torch.tensor([2, 5])
>>> width = torch.tensor([10, 40])
>>> height = torch.tensor([20, 50])
>>> depth = torch.tensor([30, 60])
>>> bbox_generator3d(x_start, y_start, z_start, width, height, depth)
tensor([[[ 0,  1,  2],
         [10,  1,  2],
         [10, 21,  2],
         [ 0, 21,  2],
         [ 0,  1, 32],
         [10,  1, 32],
         [10, 21, 32],
         [ 0, 21, 32]],

        [[ 3,  4,  5],
         [43,  4,  5],
         [43, 54,  5],
         [ 3, 54,  5],
         [ 3,  4, 65],
         [43,  4, 65],
         [43, 54, 65],
         [ 3, 54, 65]]])

kornia.geometry.bbox.bbox_to_mask(boxes, width, height)[source]#

Convert 2D bounding boxes to masks. Covered area is 1. and the remaining is 0.

Convention:: The size is (width, height) and the mask \((B, height, width)\); kornia.geometry.boxes.Boxes.to_mask() takes (height, width). Only vertices 0 (top-left) and 2 (bottom-right) are read. A pixel is covered when xmin <= x <= xmax and ymin <= y <= ymax on the unrounded values, i.e. inclusively: pass the 'vertices_plus' export of Boxes. The mask has the input dtype and no gradient path. \((B, N, 4, 2)\) raises ShapeError.

Warning

The (width, height) order is #4014. For fractional coordinates the raw comparison differs from the rounding of to_mask() and the truncation of bbox_to_mask3d(): #4015. The exclusive-export trap is #4009.

Parameters:

boxes (Tensor) – a tensor containing the coordinates of the bounding boxes to be extracted. The tensor must have the shape of \((B, 4, 2)\), where each box is defined in the following clockwise order: top-left, top-right, bottom-right and bottom-left. The coordinates must be in the x, y order.
width (int) – width of the masked image.
height (int) – height of the masked image.

Return type:

Tensor

Returns:

the output mask tensor, shape of \((B, height, width)\) and dtype of boxes.

Raises:

ShapeError – if boxes does not have shape \((B, 4, 2)\).

Note

It is currently non-differentiable.

Note

The pixel-position grid is built in float32 regardless of boxes.dtype, so float16/bfloat16 boxes are masked exactly even on images wider or taller than the integer-exact range of those dtypes (2048 px for float16, 256 px for bfloat16). The result is still returned in boxes.dtype.

Examples

>>> boxes = torch.tensor([[
...        [1., 1.],
...        [3., 1.],
...        [3., 2.],
...        [1., 2.],
...   ]])  # 1x4x2
>>> bbox_to_mask(boxes, 5, 5)
tensor([[[0., 0., 0., 0., 0.],
         [0., 1., 1., 1., 0.],
         [0., 1., 1., 1., 0.],
         [0., 0., 0., 0., 0.],
         [0., 0., 0., 0., 0.]]])

kornia.geometry.bbox.bbox_to_mask3d(boxes, size)[source]#

Convert 3D bounding boxes to masks. Covered area is 1. and the remaining is 0.

Convention:: size is (depth, height, width) and the mask \((B, 1, depth, height, width)\) in the input dtype, with a channel axis that bbox_to_mask() does not have (kornia.geometry.boxes.Boxes3D.to_mask() returns \((N, depth, height, width)\)). The bounds are truncated toward zero and compared inclusively: pass the 'vertices_plus' export. There is no gradient path. \((B, N, 8, 3)\) input raises ShapeError.

Warning

For fractional coordinates the truncation differs from bbox_to_mask() and to_mask(): #4015. An invalid box, including a non-finite coordinate, raises AssertionError: #4013.

Parameters:

boxes (Tensor) – a tensor containing the coordinates of the bounding boxes to be extracted. The tensor must have the shape of \((B, 8, 3)\), where each box is defined in the following clockwise order: front-top-left, front-top-right, front-bottom-right, front-bottom-left, back-top-left, back-top-right, back-bottom-right, back-bottom-left. The coordinates must be in the x, y, z order.
size (tuple[int, int, int]) – depth, height and width of the masked image.

Return type:

Tensor

Returns:

the output mask tensor, shape of \((B, 1, depth, height, width)\) and dtype of boxes.

Examples

>>> boxes = torch.tensor([[
...     [1., 1., 1.],
...     [2., 1., 1.],
...     [2., 2., 1.],
...     [1., 2., 1.],
...     [1., 1., 2.],
...     [2., 1., 2.],
...     [2., 2., 2.],
...     [1., 2., 2.],
... ]])  # 1x8x3
>>> bbox_to_mask3d(boxes, (4, 5, 5))
tensor([[[[[0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.]],

          [[0., 0., 0., 0., 0.],
           [0., 1., 1., 0., 0.],
           [0., 1., 1., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.]],

          [[0., 0., 0., 0., 0.],
           [0., 1., 1., 0., 0.],
           [0., 1., 1., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.]],

          [[0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.],
           [0., 0., 0., 0., 0.]]]]])

kornia.geometry.bbox.infer_bbox_shape(boxes)[source]#

Auto-infer the output sizes for the given 2D bounding boxes.

Convention:: Vertices are inclusive: a box covering pixels 0..9 has corners at 0 and 9 and width 10. The vertex-based helpers of this module share it, except bbox_generator3d() (#4018); nms() takes exclusive xyxy and transform_bbox() converts 'xywh' exclusively (xmax = xmin + width). The order is clockwise top-left, top-right, bottom-right, bottom-left. The result is (heights, widths), read from fixed vertex indices, width = boxes[:, 1, 0] - boxes[:, 0, 0] + 1 and height = boxes[:, 2, 1] - boxes[:, 0, 1] + 1, not from a max - min reduction, so a box in another vertex order (such as a rotated polygon from transform_bbox()) can give a negative extent. kornia.geometry.boxes.Boxes.get_boxes_shape() is reduction based.

Warning

The inclusive +1 arithmetic differs from torchvision, COCO and albumentations, which are exclusive: #3934.

Parameters:

boxes (Tensor) – a tensor containing the coordinates of the bounding boxes to be extracted. The tensor must have shape \((N, 4, 2)\), where each box is defined in the following clockwise order: top-left, top-right, bottom-right, bottom-left. The coordinates must be in the x, y order.

Raises:

ShapeError – if boxes does not have shape \((N, 4, 2)\).

Return type:

tuple[Tensor, Tensor]

Returns:

Bounding box heights, shape of \((N,)\).
Bounding box widths, shape of \((N,)\).

Example

>>> boxes = torch.tensor([[
...     [1., 1.],
...     [2., 1.],
...     [2., 2.],
...     [1., 2.],
... ], [
...     [1., 1.],
...     [3., 1.],
...     [3., 2.],
...     [1., 2.],
... ]])  # 2x4x2
>>> infer_bbox_shape(boxes)
(tensor([2., 2.]), tensor([2., 3.]))

kornia.geometry.bbox.infer_bbox_shape3d(boxes)[source]#

Auto-infer the output sizes for the given 3D bounding boxes.

Convention:: Vertices are inclusive (see infer_bbox_shape()), in validate_bbox3d() order. The result is (depths, heights, widths), each read from fixed vertex indices like infer_bbox_shape() (for example width = boxes[:, 1, 0] - boxes[:, 0, 0] + 1), so another vertex order can give a negative extent. Pass the 'vertices_plus' export of Boxes3D; 'vertices' reads one larger per axis. \((B, N, 8, 3)\) input raises ShapeError; flatten it first.

Warning

The +1 is #3934 and the exclusive-export trap #4009. An invalid box, including a non-finite coordinate, raises AssertionError rather than being reported: #4013.

Parameters:

boxes (Tensor) – a tensor containing the coordinates of the bounding boxes to be extracted. The tensor must have the shape of \((B, 8, 3)\), where each box is defined in the following clockwise order: front-top-left, front-top-right, front-bottom-right, front-bottom-left, back-top-left, back-top-right, back-bottom-right, back-bottom-left. The coordinates must be in the x, y, z order.

Return type:

tuple[Tensor, Tensor, Tensor]

Returns:

Bounding box depths, shape of \((B,)\).
Bounding box heights, shape of \((B,)\).
Bounding box widths, shape of \((B,)\).

Example

>>> boxes = torch.tensor([[[ 0,  1,  2],
...         [10,  1,  2],
...         [10, 21,  2],
...         [ 0, 21,  2],
...         [ 0,  1, 32],
...         [10,  1, 32],
...         [10, 21, 32],
...         [ 0, 21, 32]],
...        [[ 3,  4,  5],
...         [43,  4,  5],
...         [43, 54,  5],
...         [ 3, 54,  5],
...         [ 3,  4, 65],
...         [43,  4, 65],
...         [43, 54, 65],
...         [ 3, 54, 65]]]) # 2x8x3
>>> infer_bbox_shape3d(boxes)
(tensor([31, 61]), tensor([21, 51]), tensor([11, 41]))

kornia.geometry.bbox.nms(boxes, scores, iou_threshold)[source]#

Perform non-maxima suppression (NMS) on tensor of bounding boxes according to the intersection-over-union (IoU).

Convention:: Boxes use exclusive xyxy coordinates. IoU area is (x_2 - x_1) * (y_2 - y_1) and the result contains kept indices rather than a boolean mask.

Parameters:

boxes (Tensor) – tensor containing exclusive xyxy bounding boxes with shape \((N, 4)\), ordered as (x_1, y_1, x_2, y_2).
scores (Tensor) – tensor containing the scores associated to each bounding box with shape \((N,)\).
iou_threshold (float) – the threshold to discard the overlapping boxes.

Return type:

Tensor

Returns:

Indices of the boxes kept from the input set, ordered by descending score.

Warning

This differs from the inclusive coordinate arithmetic used by several other kornia.geometry.bbox operations and is documented in #4008.

Example

>>> boxes = torch.tensor([
...     [10., 10., 20., 20.],
...     [15., 5., 15., 25.],
...     [100., 100., 200., 200.],
...     [100., 100., 200., 200.]])
>>> scores = torch.tensor([0.9, 0.8, 0.7, 0.9])
>>> nms(boxes, scores, iou_threshold=0.8)
tensor([0, 3, 1])

kornia.geometry.bbox.transform_bbox(trans_mat, boxes, mode='xyxy', restore_coordinates=None)[source]#

Apply a transformation matrix to a box or batch of boxes.

Convention:: xyxy and xywh inputs are transformed through endpoint pairs. Coordinate restoration sorts their endpoints after a flip; polygon vertices retain their transformed cyclic order.

Parameters:

trans_mat (Tensor) – The transformation matrix to be applied, with supported shape \((B, 3, 3)\). For boxes shaped \((N, 4)\), B is one or N. For boxes shaped \((B, N, 4)\) or \((B, N, 4, 2)\), it is one or the boxes’ B.
boxes (Tensor) – The boxes to be transformed with a common shape of \((N, 4)\) or batched as \((B, N, 4)\), the polygon shape of \((B, N, 4, 2)\) is also supported.
mode (str, optional) – The format in which the boxes are provided. If set to ‘xyxy’ the boxes are assumed to be in the format xmin, ymin, xmax, ymax. If set to ‘xywh’ the boxes are assumed to be in the format xmin, ymin, width, height Default: "xyxy"
restore_coordinates (Optional[bool], optional) – Reorder endpoints after a flipped xyxy or xywh transform. Enabled by default (None behaves as True); pass False to preserve raw transformed endpoints. Polygon vertices retain their transformed order. Default: None

Return type:

Tensor

Returns:

The transformed boxes in the specified mode.

kornia.geometry.bbox.validate_bbox(boxes)[source]#

Validate whether a 2D box has matching top and bottom edge vectors.

Convention:: Vertices are inclusive, in clockwise top-left, top-right, bottom-right, bottom-left order (see infer_bbox_shape()). Accepts \((B, 4, 2)\) and \((B, N, 4, 2)\). Returns False, without raising, for an invalid shape or when the top and bottom edge vectors differ by more than 1e-4. Right angles, positive area and direction are not checked, so any parallelogram in cyclic vertex order passes.

Warning

validate_bbox3d() raises AssertionError where this function returns False: #4013.

Parameters:: boxes (Tensor) – a tensor containing the coordinates of the bounding boxes to be extracted. The tensor must have the shape of \((B, 4, 2)\) or \((B, N, 4, 2)\), where each box is defined in the following clockwise order: top-left, top-right, bottom-right, bottom-left. The coordinates must be in the x, y order.
Return type:: bool

kornia.geometry.bbox.validate_bbox3d(boxes)[source]#

Validate that a 3D box has equal inclusive edge extents along each axis, raising when it does not.

Convention:: Vertices are inclusive, front-top-left, front-top-right, front-bottom-right, front-bottom-left, then the same four back vertices. Accepts \((B, 8, 3)\) and \((B, N, 8, 3)\); another shape, or unequal extents of the four edges parallel to an axis, raises AssertionError. Right angles and positive extent are not checked. A non-finite coordinate returns False. Under graph capture only the shape is checked.

Warning

validate_bbox() returns False where this function raises: #4013.

Parameters:: boxes (Tensor) – a tensor containing the coordinates of the bounding boxes to be extracted. The tensor must have the shape of \((B, 8, 3)\) or \((B, N, 8, 3)\), where each box is defined in the following clockwise order: front-top-left, front-top-right, front-bottom-right, front-bottom-left, back-top-left, back-top-right, back-bottom-right, back-bottom-left. The coordinates must be in the x, y, z order.
Return type:: bool
Returns:: True, or False when any coordinate is non-finite. Invalid input otherwise raises instead of returning False.