kornia.geometry.boxes#

Box containers: 2D, 3D and video boxes stored as inclusive corners, with conversions.

Boxes stores 2D boxes as four (x, y) corners each, (N, 4, 2) for one image or (B, N, 4, 2) for a batch; Boxes3D stores 3D boxes as eight (x, y, z) corners each; VideoBoxes stores the 2D boxes of (B, T) video frames. from_tensor() and from_tensor() import boxes in one of several formats, 'xyxy' and 'xyzxyz' by default, and to_tensor exports them in any format, always as the axis-aligned bounds of the stored corners. The stored corners are inclusive pixel centres, x_max - x_min + 1 wide, as in kornia.geometry.bbox (#3934): the formats ending in _plus are inclusive, the others exclusive. The Convention block of each class gives its formats, padded batches and validation.

Which class to use:

2D boxes of images, several per image, with padding, clamping, area filtering and masks: Boxes.
2D boxes of video frames, exported with the temporal axis, (B, T, N, 4) in 'xyxy': VideoBoxes.
3D boxes in volumes: Boxes3D.
One box per image as a plain (B, 4, 2) corner tensor, without a container: the functions of kornia.geometry.bbox, such as bbox_to_mask().
Points instead of boxes: Keypoints.

class kornia.geometry.boxes.Boxes(boxes, raise_if_not_floating_point=True, mode='vertices_plus')[source]#

Store 2D boxes as inclusive corners, converting to and from xyxy, xywh and vertices.

boxes = Boxes.from_tensor(xyxy, mode='xyxy') takes the (N, 4) boxes of one image or the (B, N, 4) boxes of a batch ((N, 4, 2) or (B, N, 4, 2) corners in the vertex modes, or a list of per-image tensors) and stores them as inclusive corners; boxes.to_tensor('xywh') exports them in any mode, with the same batch layout. In between, transform_boxes() applies 3 x 3 matrices to all four corners, to_mask() draws (N, H, W) or (B, N, H, W) masks, pad() shifts the corners by image padding, clamp() limits them to per-image bounds, filter_boxes_by_area() removes boxes and merge() appends others. Boxes(corners) wraps inclusive 'vertices_plus' corners as they are. AugmentationSequential accepts a Boxes and transforms it with the images. For one box per image as a plain (B, 4, 2) tensor, use the kornia.geometry.bbox functions; for 3D boxes, Boxes3D; for the boxes of video frames, VideoBoxes.

Parameters:

boxes (Tensor | list[Tensor]) – 2D boxes, shape of (N, 4, 2), (B, N, 4, 2) or a list of (N, 4, 2). See below for more details.
raise_if_not_floating_point (bool, optional) – flag to control floating point casting behaviour when boxes is not a floating point tensor. True to raise an error when boxes isn’t a floating point tensor, False to cast to float32, or float64 when that is the default dtype. Floating-point input keeps its dtype. Default: True
mode (str, optional) – Representation label reused as the default output mode by to_tensor(). The constructor does not convert boxes; use from_tensor() to import another representation. Default: "vertices_plus"

Convention:

A box is a quadrilateral of four floating-point (x, y) vertices, (N, 4, 2) or (B, N, 4, 2), built in clockwise top-left, top-right, bottom-right, bottom-left order; transform_boxes() can make it an arbitrary (rotated) quadrilateral.
The stored form is inclusive ('vertices_plus'): width = xmax - xmin + 1. The exclusive 'xyxy', 'xywh' and 'vertices' modes are converted by from_tensor() and to_tensor(); the constructor converts nothing and only records mode.
to_tensor() reduces the vertices with amin/amax, so every export is an axis-aligned bounding box and to_tensor('vertices_plus') is not the identity for a rotated box.
get_boxes_shape() returns (heights, widths); padding entries of a list-backed object report as 1-by-1 boxes (the inclusive +1, #3934).
infer_bbox_shape() and bbox_to_mask() read their input as inclusive: pass them the 'vertices_plus' export, unbatched. nms() takes exclusive xyxy.
The modes of from_tensor() and to_tensor(), case-insensitive: 'xyxy' is (xmin, ymin, xmax, ymax) with xmax and ymax one past the last pixel, 'xyxy_plus' the same on the last pixel, 'xywh' is (xmin, ymin, width, height), 'vertices' the corners with xmax and ymax one past the last pixel, and 'vertices_plus' the stored corners. from_tensor(torch.tensor([[[0., 0., 10., 20.]]])) stores [[0, 0], [9, 0], [9, 19], [0, 19]], which to_tensor exports as [0, 0, 10, 20] in 'xyxy' and 'xywh' and [0, 0, 9, 19] in 'xyxy_plus'.
With validate_boxes=True, every mode rejects a non-finite coordinate. The 'xy*' modes raise ValueError and also reject non-positive extents. The vertex modes raise RuntimeError through torch._assert_async (by the next synchronization on CUDA, and on MPS from torch 2.13), so they stay fullgraph-compilable, and take no extent or shape check, so the arbitrary quadrilaterals of transform_boxes() import unchanged.
The constructor rejects an integer tensor unless raise_if_not_floating_point=False (for list inputs, every element is checked); from_tensor() casts integer input to float32, or float64 when that is the default dtype.
A list of per-image tensors, (N_i, 4) or (N_i, 4, 2) in from_tensor() and (N_i, 4, 2) in the constructor, is zero-padded to one (B, max N_i, 4, 2) batch; to_tensor() then returns a list trimmed back to each image’s boxes, or the padded tensor with as_padded_sequence=True.
Which rows are padding is recorded per box and follows the coordinates through indexing, item assignment, index_put() and merge(): a padding row written elsewhere stays padding, and a Boxes that does not come from a list writes real boxes. A raw tensor passed to index_put() changes only coordinates. Writing padding to a Boxes that does not come from a list marks those rows invalid, since that object has no trimmed export.
A selection by integer or slice shares its coordinates, validity and padding with the object it came from, as tensor indexing does, and any other key copies all three. to() and type() to another device or dtype, and an in-place transform_boxes(), translate() or merge(), end the sharing of all three, for the object they are called on, whether it is the selection or the original.
filter_boxes_by_area() removes boxes without changing the shape: it zeroes their corners and marks them invalid, and every export of an invalid box is exact zeros, after later transforms too.
transform_boxes(), translate(), clamp(), filter_boxes_by_area(), merge() and index_put() are non-mutating by default and return a new object; with inplace=True they update and return self. pad() and unpad() always shift self. The constructor wraps a floating-point tensor without copying it, so item assignment and the in-place updates of pad(), unpad(), clamp(), filter_boxes_by_area() and index_put() reach the caller’s tensor.
Integer indexing selects one image’s box set for batched input, or one box with shape (1, 4, 2) for unbatched input. Scalar integer tensors follow the same rule.

Warning

The inclusive +1 differs from torchvision, COCO and albumentations: #3934. A sub-unit extent (normalized [0, 1] boxes) is corrupted by the - 1 of the converting modes: #4061. Passing the 'vertices' export to the bbox helpers reads one pixel larger: #4009. compute_area() is the shoelace area of the stored inclusive vertices, (width - 1) * (height - 1): #4010. nms() is exclusive: #4008. to_mask() and bbox_to_mask() agree on (height, width) once that function is called with size; its deprecated width, height form takes the opposite order: #4014. The integer-input split is #4012. 'vertices' subtracts the inclusive offset at fixed vertex slots, so it deforms a quadrilateral that is not an axis-aligned rectangle in clockwise order: #3934. trim was never implemented and is deprecated: #4017.

See also

Boxes3D: 3D boxes as eight corners, from (N, 6) boxes, with no list input, padding or filtering.
VideoBoxes: the 2D boxes of (B, T) video frames, exported with the temporal axis.
bbox_to_mask(): the mask of one box per image of a plain (B, 4, 2) tensor, with unrounded inclusive bounds.
transform_bbox(): transforms plain xyxy or xywh tensors through their two endpoints only.
nms(): suppresses overlapping boxes of one image, given the 'xyxy' export.
Keypoints: (x, y) points in the same pixel frame, with matching pad, unpad and transform methods.

clamp(topleft=None, botright=None, inplace=False)[source]#

Clamp every box vertex inside per-image coordinate limits.

See the Convention block on Boxes.

Convention:: Bounds are tensors with one (x, y) pair per batch element, or a single row that broadcasts (an unbatched container needs exactly one row). Every vertex is clamped independently, so a box wholly outside the bounds collapses onto the boundary instead of being removed.

Parameters:

topleft (Optional[Tensor], optional) – Required tensor of shape (B, 2) containing the minimum x and y coordinate allowed for each batch item. A single row broadcasts across the batch. Default: None
botright (Optional[Tensor], optional) – Required tensor of shape (B, 2) containing the maximum x and y coordinate allowed for each batch item. A single row broadcasts across the batch. Default: None
inplace (bool, optional) – If True, clamp this object in place. Otherwise, return a new Boxes object with clamped data. Default: False

Return type:

Boxes

Returns:

Boxes whose vertex coordinates are restricted to the provided bounds.

clone()[source]#

Create an independent copy of the box container.

Return type:: Boxes
Returns:: New Boxes object with cloned tensor storage. Metadata such as the current mode, list padding, and batched flag is preserved.

compute_area()[source]#

Compute polygon area with the shoelace formula.

See the Convention block on Boxes.

Convention:: Vertices are sorted by angle about their centroid, so a convex quadrilateral is assumed.

Warning

On the stored inclusive vertices an axis-aligned box gives (width - 1) * (height - 1), not the width * height of get_boxes_shape(): #4010.

Return type:: Tensor
Returns:: Area for each box, shaped (N,) or (B, N).

property data: Tensor#

Return the raw quadrilateral coordinate tensor.

Returns:: Tensor storing four vertices per box in (x, y) order. The common shapes are (N, 4, 2) for unbatched boxes and (B, N, 4, 2) for batched boxes, where \(B\) is batch size and \(N\) is the number of boxes.

property device: device#: Returns boxes device.

property dtype: dtype#: Returns boxes dtype.

filter_boxes_by_area(min_area=None, max_area=None, inplace=False)[source]#

Zero boxes whose polygon area is outside the requested range.

See the Convention block on Boxes.

The box area is computed from its four vertices. Boxes smaller than min_area or larger than max_area are not dropped from the tensor; their coordinates are replaced with zeros so the original batch and box dimensions stay unchanged. See compute_area() for the area convention used by the thresholds. Filtered boxes export as exact zeros in every mode and draw no mask pixels, including after subsequent coordinate transforms.

Parameters:

min_area (Optional[float], optional) – Optional lower inclusive area threshold. Boxes with area below this value are zeroed. Default: None
max_area (Optional[float], optional) – Optional upper inclusive area threshold. Boxes with area above this value are zeroed. Default: None
inplace (bool, optional) – If True, update this object in place. Otherwise, return a filtered clone. Default: False

Return type:

Boxes

Returns:

Boxes with the same shape as the input container and out-of-range boxes replaced by zero coordinates.

classmethod from_tensor(boxes, mode='xyxy', validate_boxes=True)[source]#

Create Boxes from boxes stored in another format.

See the Convention block on Boxes.

Parameters:

boxes (Tensor | list) – 2D boxes, shape of (N, 4), (B, N, 4), (N, 4, 2) or (B, N, 4, 2), a nested numeric list with one of those shapes, or a list of (N, 4) or (N, 4, 2) tensors matching mode. Integer input is cast to float32, or float64 when that is the default dtype. Floating-point input keeps its dtype.
mode (str, optional) –
The format in which the boxes are provided: Default: "xyxy"
- ’xyxy’: xmin, ymin, xmax, ymax with exclusive extent. With shape (N, 4), (B, N, 4).
- ’xyxy_plus’: xmin, ymin, xmax, ymax with inclusive extent. With shape (N, 4), (B, N, 4).
- ’xywh’: xmin, ymin, width, height with exclusive extent. With shape (N, 4), (B, N, 4).
- ’vertices’: boxes are defined by their vertices points in the following clockwise order: top-left, top-right, bottom-right, bottom-left. Vertices coordinates are in (x,y) order. This is the exclusive input form. With shape (N, 4, 2), (B, N, 4, 2).
- ’vertices_plus’: the inclusive stored vertex form. With shape (N, 4, 2), (B, N, 4, 2).
validate_boxes (bool, optional) – Reject a non-finite coordinate in every mode: ValueError for the 'xy*' modes, which also reject non-positive extents in each mode’s convention, and RuntimeError from torch._assert_async for the vertex modes, which take no extent or shape check. The 'xy*' checks read the data, so graph capture (torch.compile, torch.export) skips them. Default: True

Return type:

Boxes

Returns:

Boxes containing the converted inclusive vertex representation.

Examples

>>> boxes_xyxy = torch.as_tensor([[0, 3, 1, 4], [5, 1, 8, 4]])
>>> boxes = Boxes.from_tensor(boxes_xyxy, mode='xyxy')
>>> boxes.data  # (2, 4, 2)
tensor([[[0., 3.],
         [0., 3.],
         [0., 3.],
         [0., 3.]],

        [[5., 1.],
         [7., 1.],
         [7., 3.],
         [5., 3.]]])

get_boxes_shape()[source]#

Compute boxes heights and widths.

See the Convention block on Boxes.

Return type:

tuple[Tensor, Tensor]

Returns:

Boxes heights, shape of (N,) or (B,N).
Boxes widths, shape of (N,) or (B,N).

Example

>>> boxes_xyxy = torch.tensor([[[1,1,2,2],[1,1,3,2]]])
>>> boxes = Boxes.from_tensor(boxes_xyxy)
>>> boxes.get_boxes_shape()
(tensor([[1., 1.]]), tensor([[1., 2.]]))

index_put(indices, values, inplace=False)[source]#

Write box coordinates at selected tensor indices.

See the Convention block on Boxes.

This mirrors torch.Tensor.index_put_() for the internal quadrilateral tensor. It is useful when a subset of boxes in a batch must be replaced while keeping the Boxes wrapper and metadata.

Parameters:

indices (tuple[Tensor, ...] | list[Tensor]) – Index tuple or list accepted by Tensor.index_put_. The indices address entries in the stored tensor, commonly shaped (B, N, 4, 2) or (N, 4, 2).
values (Tensor | Boxes) – Replacement coordinates. If a Boxes object is passed, its coordinates, filtering validity and list padding are copied. Raw tensor writes change only coordinates and preserve the existing filtering validity and list padding.
inplace (bool, optional) – If True, update this object and return self. If False, clone the current data first and return a new Boxes instance. Default: False

Return type:

Boxes

Returns:

Boxes containing the updated coordinates.

merge(boxes, inplace=False)[source]#

Merge boxes.

See the Convention block on Boxes.

For batched boxes, if the current instance holds (B, N, 4, 2) and the incoming boxes holds (B, M, 4, 2), the merge results in (B, N + M, 4, 2).

For unbatched boxes, if the current instance holds (N, 4, 2) and the incoming boxes holds (M, 4, 2), the merge results in (N + M, 4, 2).

Parameters:

boxes (Boxes) – 2D boxes.
inplace (bool, optional) – do transform in-place and return self. Default: False

Return type:

Boxes

Note

When either input was created from a list, each batch row is repacked so that its real boxes precede all of its padding, and the merged object exports a list trimmed back to them.

property mode: str#

Return the box format remembered by this container.

Returns:: Mode string used as the default by to_tensor(), such as "xyxy", "xyxy_plus", "xywh", "vertices" or "vertices_plus".

pad(padding_size)[source]#

Pad every box in place.

See the Convention block on Boxes.

padding_size is ordered as (left, right, top, bottom). Only left and top change the coordinate origin; this method returns self after adding those two values to every vertex. Both the batched (B, N, 4, 2) and the unbatched (N, 4, 2) container are supported; the unbatched form carries a single image, so padding_size must have exactly one row.

Parameters:: padding_size (Tensor) – Per-batch padding in (left, right, top, bottom) order, shaped (B, 4). A single row broadcasts across the batch.
Return type:: Boxes

property shape: tuple[int, ...] | Size#

Return the tensor shape used to store the boxes.

Returns:: Shape of data. For unbatched boxes this is usually (N, 4, 2), where \(N\) is the number of boxes, 4 is the number of corner vertices, and 2 stores (x, y). For batched boxes the shape is usually (B, N, 4, 2), where \(B\) is the batch size.

to(device=None, dtype=None)[source]#

Move or cast the stored tensor and return self, as torch.nn.Module.to() does.

The tensor is rebound, not converted in place: one obtained from data before the call keeps its device and dtype. A dtype that is not floating point raises ValueError.

Return type:: Boxes

to_mask(height, width)[source]#

Convert 2D boxes to masks. Covered area is 1 and the remaining is 0.

See the Convention block on Boxes.

Convention:: The size is (height, width) and the mask (N, height, width) or (B, N, height, width) in the box dtype. The exclusive 'xyxy' export is clamped to the image, rounded to the nearest integer and filled over [xmin, xmax), [ymin, ymax). Padding entries of a list-backed object give empty masks. A box tensor that requires grad raises RuntimeError.

Warning

bbox_to_mask() takes (height, width) through its size argument, like this method (#4014); its deprecated width, height form takes the opposite order. It also does not round, so a fractional box can fill a different area (#4015).

Parameters:

height (int) – height of the masked image/images.
width (int) – width of the masked image/images.

Return type:

Tensor

Returns:

the output mask tensor, shape of (N, height, width) or (B, N, height, width) and dtype of Boxes.dtype() (it can be any floating point dtype).

Note

It is currently non-differentiable.

Examples

>>> boxes = Boxes(torch.tensor([[  # Equivalent to Boxes.from_tensor([[1, 1, 4, 3]], mode='xyxy_plus')
...        [1., 1.],
...        [4., 1.],
...        [4., 3.],
...        [1., 3.],
...   ]]))  # 1x4x2
>>> boxes.to_mask(5, 5)
tensor([[[0., 0., 0., 0., 0.],
         [0., 1., 1., 1., 1.],
         [0., 1., 1., 1., 1.],
         [0., 1., 1., 1., 1.],
         [0., 0., 0., 0., 0.]]])

to_tensor(mode=None, as_padded_sequence=False)[source]#

Cast Boxes to a tensor.

See the Convention block on Boxes.

mode selects the output representation. 'xyxy', 'xywh', and 'vertices' are exclusive exports; the '_plus' variants are inclusive. Every mode exports the axis-aligned bounds of the stored vertices, reduced with amin/amax, so the export is lossy for rotated boxes: to_tensor('vertices_plus') does not return data unchanged after a transform_boxes() call that rotates, shears, or otherwise reorders the vertices. A quarter turn keeps the box axis-aligned and still yields an export that differs from data, because the reduction re-canonicalizes the vertex order.

Warning

Before torch 2.10, torch.compile(..., fullgraph=True) around the trimmed export of boxes built from a list raises, because that export reads its rows with nonzero, whose output size depends on the data. Setting torch._dynamo.config.capture_dynamic_output_shape_ops, as the error suggests, still raises. Setting capture_scalar_outputs as well makes it compile, and on torch 2.5 and 2.6 it then returns wrong rows with no error, from a miscompile of nonzero in torch. as_padded_sequence=True and default torch.compile return the right rows on every one of these versions.

Parameters:

mode (Optional[str], optional) –
the output box format, or None to reuse mode. That attribute depends on the construction path: the constructor defaults to 'vertices_plus', while from_tensor() records the mode its input was given in. It could be: Default: None
- ’xyxy’: xmin, ymin, xmax, ymax with exclusive extent.
- ’xyxy_plus’: xmin, ymin, xmax, ymax with inclusive extent.
- ’xywh’: xmin, ymin, width, height with exclusive extent.
- ’vertices’: boxes are defined by their vertices points in the following clockwise order: top-left, top-right, bottom-right, bottom-left. Vertices coordinates are in (x,y) order. This is the exclusive export form.
- ’vertices_plus’: the inclusive stored vertex form.
as_padded_sequence (bool, optional) – If this object was created from a list, return its padded tensor rather than a list of tensors trimmed to their original lengths. The padded values follow the selected output mode. Batch slicing, index tensors, and boolean masks keep the list export. A selection that returns an unbatched object, such as an integer or scalar tensor index, exports all its rows, padding included, as one tensor. Default: False

Return type:

Tensor | list[Tensor]

Returns:

Boxes tensor in the mode format, or a list of tensors when the object was created from a list and as_padded_sequence=False. The tensor shape depends on the mode value:

’vertices’ or ‘vertices_plus’: (N, 4, 2) or (B, N, 4, 2).

Any other value: (N, 4) or (B, N, 4).

Examples

>>> boxes_xyxy = torch.as_tensor([[0, 3, 1, 4], [5, 1, 8, 4]])
>>> boxes = Boxes.from_tensor(boxes_xyxy)
>>> assert (boxes_xyxy == boxes.to_tensor(mode='xyxy')).all()

transform_boxes(M, inplace=False)[source]#

Apply a transformation matrix to the 2D boxes.

See the Convention block on Boxes.

Parameters:

M (Tensor) – The transformation matrix to be applied, shape of (3, 3) or (B, 3, 3).
inplace (bool, optional) – do transform in-place and return self. Default: False

Return type:

Boxes

Returns:

The transformed boxes.

transform_boxes_(M)[source]#

Apply transform_boxes() in place and return self.

See the Convention block on Boxes.

Convention:: The internal tensor is rebound, not written: a tensor obtained from data before the call keeps the old coordinates (an empty container keeps its tensor).

Return type:: Boxes
Returns:: This Boxes object after the transformation.

translate(size, method='warp', inplace=False)[source]#

Translate boxes by the provided size.

See the Convention block on Boxes.

size supplies one (x, y) translation per batch item. Only the "warp" method is supported; any other method raises NotImplementedError.

Parameters:

size (Tensor) – translate size for x, y direction, shape of (B, 2).
method (str, optional) – only “warp” is supported. Default: "warp"
inplace (bool, optional) – do transform in-place and return self. Default: False

Return type:

Boxes

Returns:

The transformed boxes.

trim(correspondence_preserve=False, inplace=False)[source]#

Deprecated since 0.9.1 and never implemented: raises NotImplementedError.

See the Convention block on Boxes.

Parameters:

correspondence_preserve (bool, optional) – Reserved for a future implementation. Default: False
inplace (bool, optional) – Reserved for a future implementation. Default: False

Raises:

NotImplementedError – Always.

Return type:

Boxes

type(dtype)[source]#

Cast the stored box coordinates to a new dtype.

Parameters:: dtype (dtype) – Target dtype for the coordinate tensor. Unlike to(), an integer dtype is not rejected.
Return type:: Boxes
Returns:: self, rebound to the converted tensor; the tensor it wrapped before is not modified.

unpad(padding_size)[source]#

Undo pad() in place.

See the Convention block on Boxes.

padding_size is ordered as (left, right, top, bottom). Only left and top change the coordinate origin; this method returns self after subtracting those two values from every vertex. Both the batched (B, N, 4, 2) and the unbatched (N, 4, 2) container are supported; the unbatched form carries a single image, so padding_size must have exactly one row.

Parameters:: padding_size (Tensor) – Per-batch padding in (left, right, top, bottom) order, shaped (B, 4). A single row broadcasts across the batch.
Return type:: Boxes

class kornia.geometry.boxes.Boxes3D(boxes, raise_if_not_floating_point=True, mode='xyzxyz_plus')[source]#

Store 3D boxes as eight inclusive corners, converting to and from xyzxyz and xyzwhd.

boxes = Boxes3D.from_tensor(xyzxyz) takes (N, 6) or (B, N, 6) boxes in 'xyzxyz' (the default), 'xyzxyz_plus' or 'xyzwhd' mode and stores them as (N, 8, 3) or (B, N, 8, 3) inclusive corners; boxes.to_tensor('xyzwhd') exports them in any of these modes or as corners. In between, transform_boxes() applies 4 x 4 matrices to all eight corners and to_mask() draws (N, D, H, W) or (B, N, D, H, W) masks. Unlike Boxes it takes no list input and has no padding, clamping, merging or area filtering. For one box per volume as a plain (B, 8, 3) tensor, use bbox_to_mask3d() and infer_bbox_shape3d().

Parameters:

boxes (Tensor) – 3D boxes, shape of (N,8,3) or (B,N,8,3). See below for more details.
raise_if_not_floating_point (bool, optional) – flag to control floating point casting behaviour when boxes is not a floating point tensor. True to raise an error when boxes isn’t a floating point tensor, False to cast to float32, or float64 when that is the default dtype. Floating-point input keeps its dtype. Default: True
mode (str, optional) – Representation label reported by mode. The constructor does not convert boxes and to_tensor() does not consult the label; use from_tensor() to import another representation. Default: "xyzxyz_plus"

Convention:

A box is a hexahedron of eight floating-point (x, y, z) vertices, (N, 8, 3) or (B, N, 8, 3): front-top-left, front-top-right, front-bottom-right, front-bottom-left, then the same four back vertices.
The stored form is inclusive (depth = zmax - zmin + 1), as for Boxes. from_tensor() takes (N, 6) or (B, N, 6) in 'xyzxyz', 'xyzwhd' (exclusive) or 'xyzxyz_plus'; the vertex modes exist for to_tensor() only. Modes are case-insensitive.
to_tensor() exports the amin/amax axis-aligned bounding box and defaults to 'xyzxyz' whatever the stored label.
get_boxes_shape() returns (depths, heights, widths), inclusive.
infer_bbox_shape3d() and bbox_to_mask3d() read their input as inclusive: pass them the 'vertices_plus' export, unbatched.
With validate_boxes=True, from_tensor() rejects a non-finite coordinate and extents that are not positive in the given mode (xmax == xmin fails in 'xyzxyz', passes in 'xyzxyz_plus').
The constructor rejects an integer tensor unless raise_if_not_floating_point=False; from_tensor() casts integer input to float32, or float64 when that is the default dtype.
transform_boxes() returns a new object labelled 'xyzxyz_plus'; transform_boxes_() rebinds the internal tensor of self and keeps the label.
Integer indexing selects one image’s box set for batched input, or one box with shape (1, 8, 3) for unbatched input. Scalar integer tensors follow the same rule.

Warning

The inclusive +1 is #3934 and the 'vertices' export trap #4009. The integer-input split is #4012 and the validator split with validate_bbox() is #4013. The to_tensor() default differs from Boxes.to_tensor(), which defaults to the stored mode: #4251.

See also

Boxes: 2D boxes as four corners, with list input, padding, clamping and area filtering.
VideoBoxes: the 2D boxes of (B, T) video frames, rather than boxes in a volume.
bbox_to_mask3d(): the (B, 1, D, H, W) mask of one box per volume of a plain (B, 8, 3) tensor, with truncated bounds.
infer_bbox_shape3d(): the inclusive sizes of a plain (B, 8, 3) tensor.
Keypoints3D: (x, y, z) points, transformed by the same 4 x 4 matrices.

property data: Tensor#

Return the raw 3D corner-coordinate tensor.

Returns:: Tensor containing eight 3D corner coordinates per box, usually shaped (N, 8, 3) or (B, N, 8, 3).

property device: device#: Returns boxes device.

property dtype: dtype#: Returns boxes dtype.

classmethod from_tensor(boxes, mode='xyzxyz', validate_boxes=True)[source]#

Create Boxes3D from 3D boxes stored in another format.

See the Convention block on Boxes3D.

Parameters:

boxes (Tensor) – 3D boxes, shape of (N,6) or (B,N,6). Integer input is cast to float32, or float64 when that is the default dtype. Floating-point input keeps its dtype.
mode (str, optional) –
The format in which the 3D boxes are provided, matched case-insensitively. Default: "xyzxyz"
- ’xyzxyz’: boxes are assumed to be in the format xmin, ymin, zmin, xmax, ymax, zmax where width = xmax - xmin, height = ymax - ymin and depth = zmax - zmin.
- ’xyzxyz_plus’: similar to ‘xyzxyz’ mode but where box width, length and depth are defined as width = xmax - xmin + 1, height = ymax - ymin + 1 and depth = zmax - zmin + 1.
- ’xyzwhd’: boxes are assumed to be in the format xmin, ymin, zmin, width, height, depth where width = xmax - xmin, height = ymax - ymin and depth = zmax - zmin.
validate_boxes (bool, optional) – reject boxes with a non-finite coordinate, and boxes whose width, height or depth is not positive when measured in the given mode’s convention, so xmax == xmin is rejected in 'xyzxyz' and accepted in 'xyzxyz_plus'. The checks read the data, so graph capture (torch.compile, torch.export) skips them. Default: True

Return type:

Boxes3D

Returns:

Boxes3D containing the converted inclusive vertex representation, labelled with mode.

Examples

>>> boxes_xyzxyz = torch.as_tensor([[0, 3, 6, 1, 4, 8], [5, 1, 3, 8, 4, 9]])
>>> boxes = Boxes3D.from_tensor(boxes_xyzxyz, mode='xyzxyz')
>>> boxes.data  # (2, 8, 3)
tensor([[[0., 3., 6.],
         [0., 3., 6.],
         [0., 3., 6.],
         [0., 3., 6.],
         [0., 3., 7.],
         [0., 3., 7.],
         [0., 3., 7.],
         [0., 3., 7.]],

        [[5., 1., 3.],
         [7., 1., 3.],
         [7., 3., 3.],
         [5., 3., 3.],
         [5., 1., 8.],
         [7., 1., 8.],
         [7., 3., 8.],
         [5., 3., 8.]]])

get_boxes_shape()[source]#

Compute boxes depths, heights and widths.

See the Convention block on Boxes3D.

Return type:

tuple[Tensor, Tensor, Tensor]

Returns:

Boxes depths, shape of (N,) or (B,N).
Boxes heights, shape of (N,) or (B,N).
Boxes widths, shape of (N,) or (B,N).

Example

>>> boxes_xyzxyz = torch.tensor([[ 0,  1,  2, 10, 21, 32], [3, 4, 5, 43, 54, 65]])
>>> boxes3d = Boxes3D.from_tensor(boxes_xyzxyz)
>>> boxes3d.get_boxes_shape()
(tensor([30., 60.]), tensor([20., 50.]), tensor([10., 40.]))

property mode: str#

Return the 3D box format remembered by this container.

Returns:

The mode label given to from_tensor() or the constructor; the new object that transform_boxes() returns is labelled 'xyzxyz_plus'. to_tensor() does not read it (#4251).

property shape: tuple[int, ...] | Size#

Return the tensor shape used to store 3D boxes.

Returns:: Shape of data. For unbatched boxes this is usually (N, 8, 3), where \(N\) is the number of boxes, 8 is the number of cuboid corners, and 3 stores (x, y, z). For batched boxes the shape is usually (B, N, 8, 3).

to(device=None, dtype=None)[source]#

Move or cast the stored tensor and return self, as torch.nn.Module.to() does.

The tensor is rebound, not converted in place: one obtained from data before the call keeps its device and dtype. A dtype that is not floating point raises ValueError.

Return type:: Boxes3D

to_mask(depth, height, width)[source]#

Convert 3D boxes to masks. Covered area is 1 and the remaining is 0.

See the Convention block on Boxes3D.

Convention:: The size is (depth, height, width) and the mask (N, depth, height, width) or (B, N, depth, height, width) in the box dtype, with no channel axis. The exclusive 'xyzxyz' export is clamped to the volume, rounded to the nearest integer and filled over half-open ranges. A box tensor that requires grad raises RuntimeError.

Warning

bbox_to_mask3d() truncates instead of rounding, so a fractional box can fill a different volume: #4015.

Parameters:

depth (int) – depth of the masked image/images.
height (int) – height of the masked image/images.
width (int) – width of the masked image/images.

Return type:

Tensor

Returns:

the output mask tensor, shape of (N, depth, height, width) or (B, N, depth, height, width) and dtype of Boxes3D.dtype() (it can be any floating point dtype).

Note

It is currently non-differentiable.

Examples

>>> boxes = Boxes3D(torch.tensor([[  # Same as Boxes3D.from_tensor([[1, 1, 1, 3, 3, 2]], 'xyzxyz_plus')
...     [1., 1., 1.],
...     [3., 1., 1.],
...     [3., 3., 1.],
...     [1., 3., 1.],
...     [1., 1., 2.],
...     [3., 1., 2.],
...     [3., 3., 2.],
...     [1., 3., 2.],
... ]]))  # 1x8x3
>>> boxes.to_mask(4, 5, 5)
tensor([[[[0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]],

         [[0., 0., 0., 0., 0.],
          [0., 1., 1., 1., 0.],
          [0., 1., 1., 1., 0.],
          [0., 1., 1., 1., 0.],
          [0., 0., 0., 0., 0.]],

         [[0., 0., 0., 0., 0.],
          [0., 1., 1., 1., 0.],
          [0., 1., 1., 1., 0.],
          [0., 1., 1., 1., 0.],
          [0., 0., 0., 0., 0.]],

         [[0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]]]])

to_tensor(mode='xyzxyz')[source]#

Cast Boxes3D to a tensor.

See the Convention block on Boxes3D.

Convention:: mode defaults to 'xyzxyz' regardless of the stored label (#4251). Every export is the amin/amax axis-aligned bounding box of the stored vertices.

Parameters:

mode (str, optional) –

The format in which the boxes are provided, matched case-insensitively. Default: "xyzxyz"

’xyzxyz’: boxes are assumed to be in the format xmin, ymin, zmin, xmax, ymax, zmax where width = xmax - xmin, height = ymax - ymin and depth = zmax - zmin.
’xyzxyz_plus’: similar to ‘xyzxyz’ mode but where box width, length and depth are defined as width = xmax - xmin + 1, height = ymax - ymin + 1 and depth = zmax - zmin + 1.
’xyzwhd’: boxes are assumed to be in the format xmin, ymin, zmin, width, height, depth where width = xmax - xmin, height = ymax - ymin and depth = zmax - zmin.
’vertices’: boxes are defined by their vertices points in the following clockwise order: front-top-left, front-top-right, front-bottom-right, front-bottom-left, back-top-left, back-top-right, back-bottom-right, back-bottom-left. Vertices coordinates are in (x,y, z) order. Finally, box width, height and depth are defined as width = xmax - xmin, height = ymax - ymin and depth = zmax - zmin.
’vertices_plus’: similar to ‘vertices’ mode but where box width, height and depth are defined as width = xmax - xmin + 1, height = ymax - ymin + 1 and depth = zmax - zmin + 1; this is the stored form.

Returns:

‘vertices’ or ‘vertices_plus’: (N, 8, 3) or (B, N, 8, 3).
Any other value: (N, 6) or (B, N, 6).

Return type:

3D Boxes tensor in the mode format. The shape depends with the mode value

Note

The amin/amax reduction ties on every face of an axis-aligned box (4 vertices share each extremum), so the gradient there is a subgradient shared among the tied vertices; it sums to one over them but need not match a finite-difference estimate such as gradcheck().

Examples

>>> boxes_xyzxyz = torch.as_tensor([[0, 3, 6, 1, 4, 8], [5, 1, 3, 8, 4, 9]])
>>> boxes = Boxes3D.from_tensor(boxes_xyzxyz, mode='xyzxyz')
>>> assert (boxes.to_tensor(mode='xyzxyz') == boxes_xyzxyz).all()

transform_boxes(M, inplace=False)[source]#

Apply a transformation matrix to the 3D boxes.

See the Convention block on Boxes3D.

Parameters:

M (Tensor) – The transformation matrix to be applied, shape of (4, 4) or (B, 4, 4), where B must equal the box batch size and an unbatched box counts as one.
inplace (bool, optional) – do transform in-place and return self. Default: False

Returns:

a new Boxes3D labelled 'xyzxyz_plus' when inplace is false, otherwise self.

Return type:

The transformed boxes

transform_boxes_(M)[source]#

Apply transform_boxes() in place and return self.

See the Convention block on Boxes3D.

Convention:: The internal tensor is rebound, not written: a tensor obtained from data before the call keeps the old coordinates. The stored mode label is kept.

Return type:: Boxes3D
Returns:: This Boxes3D object after the transformation.

class kornia.geometry.boxes.VideoBoxes(boxes, raise_if_not_floating_point=True, mode='vertices_plus')[source]#

Store the 2D boxes of (B, T) video frames, restoring the temporal axis on export.

boxes = VideoBoxes.from_tensor(corners) takes inclusive 'vertices_plus' corners (B, T, N, 4, 2), with no mode or conversion, and stores them flattened as (B * T, N, 4, 2), so the methods it inherits from Boxes act on every frame; boxes.to_tensor('xyxy') exports (B, T, N, 4), with the temporal axis restored, and boxes.to_tensor() the (B, T, N, 4, 2) corners. AugmentationSequential stores the boxes of a pipeline that contains a VideoSequential in this class. Convert boxes in another format to corners first: Boxes.from_tensor(xyxy.flatten(0, 1)).data.unflatten(0, (B, T)) converts (B, T, N, 4) xyxy boxes. For the boxes of single images, use Boxes.

See the Convention block on Boxes.

Convention:

from_tensor() stores (B, T, N, 4, 2) input unchanged as (B * T, N, 4, 2) 'vertices_plus' data, with no mode or conversion; integer input is cast to float32, or float64 when that is the default dtype. Another shape or a list raises ValueError. validate_boxes=True rejects a non-finite coordinate as Boxes.from_tensor() does for 'vertices_plus'.
to_tensor() accepts every Boxes mode and restores the temporal axis (to_tensor('xyxy') is (B, T, N, 4)).
compute_area() returns (B, T, N) areas, and to_mask() returns (B, T, N, height, width) masks. shape and data stay the flattened storage (B * T, N, 4, 2).
Indexing selects the video batch axis while preserving the temporal axis. An integer index returns a one-video VideoBoxes so to_tensor() keeps the shape (1, T, ...). Index assignment, video_boxes[key] = other, writes the same whole videos.
A transformation matrix is (B * T, 3, 3); a (3, 3) matrix raises ValueError unless \(B \cdot T = 1\).
Boxes.transform_boxes(), Boxes.translate(), Boxes.clamp(), Boxes.filter_boxes_by_area() and Boxes.merge() return a new VideoBoxes; Boxes.pad(), Boxes.unpad(), Boxes.to() and Boxes.type() update and return self.
Every frame holds the same number of rows, with no trimmed export. The padding of list-built Boxes merged or written into a VideoBoxes is marked invalid, so it exports as exact zeros in every mode and draws no mask.

Variables:: temporal_channel_size – Number of frames \(T\) stored with the boxes.