Loss functions#

Segmentation Losses#

DiceLoss#

class monai.losses.DiceLoss(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, squared_pred=False, jaccard=False, reduction=mean, smooth_nr=1e-05, smooth_dr=1e-05, batch=False, weight=None, soft_label=False)[source]#

Compute average Dice loss between two tensors. It can support both multi-classes and multi-labels tasks. The data input (BNHW[D] where N is number of classes) is compared with ground truth target (BNHW[D]).

Note that axis N of input is expected to be logits or probabilities for each class, if passing logits as input, must set sigmoid=True or softmax=True, or specifying other_act. And the same axis of target can be 1 or N (one-hot format).

The smooth_nr and smooth_dr parameters are values added to the intersection and union components of the inter-over-union calculation to smooth results respectively, these values should be small.

The original papers:

Milletari, F. et. al. (2016) V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. 3DV 2016.

Wang, Z. et. al. (2023) Jaccard Metric Losses: Optimizing the Jaccard Index with Soft Labels. NeurIPS 2023.

Wang, Z. et. al. (2023) Dice Semimetric Losses: Optimizing the Dice Score with Soft Labels. MICCAI 2023.

__init__(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, squared_pred=False, jaccard=False, reduction=mean, smooth_nr=1e-05, smooth_dr=1e-05, batch=False, weight=None, soft_label=False)[source]#
Parameters:

  • include_background – if False, channel index 0 (background category) is excluded from the calculation. if the non-background segmentations are small compared to the total image size they can get overwhelmed by the signal from the background so excluding it in such cases helps convergence.

  • to_onehot_y – whether to convert the target into the one-hot format, using the number of classes inferred from input (input.shape[1]). Defaults to False.

  • sigmoid – if True, apply a sigmoid function to the prediction.

  • softmax – if True, apply a softmax function to the prediction.

  • other_act – callable function to execute other activation layers, Defaults to None. for example: other_act = torch.tanh.

  • squared_pred – use squared versions of targets and predictions in the denominator or not.

  • jaccard – compute Jaccard Index (soft IoU) instead of dice or not.

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • smooth_nr – a small constant added to the numerator to avoid zero.

  • smooth_dr – a small constant added to the denominator to avoid nan.

  • batch – whether to sum the intersection and union areas over the batch dimension before the dividing. Defaults to False, a Dice loss value is computed independently from each item in the batch before any reduction.

  • weight – weights to apply to the voxels of each class. If None no weights are applied. The input can be a single value (same weight for all classes), a sequence of values (the length of the sequence should be the same as the number of classes. If not include_background, the number of classes should not include the background category class 0). The value/values should be no less than 0. Defaults to None.

  • soft_label – whether the target contains non-binary values (soft labels) or not. If True a soft label formulation of the loss will be used.

Raises:

  • TypeError – When other_act is not an Optional[Callable].

  • ValueError – When more than 1 of [sigmoid=True, softmax=True, other_act is not None]. Incompatible values.

forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be BNH[WD], where N is the number of classes.

  • target (Tensor) – the shape should be BNH[WD] or B1H[WD], where N is the number of classes.

Raises:

  • AssertionError – When input and target (after one hot transform if set) have different shapes.

  • ValueError – When self.reduction is not one of [“mean”, “sum”, “none”].

Example

>>> from monai.losses.dice import *  # NOQA
>>> import torch
>>> from monai.losses.dice import DiceLoss
>>> B, C, H, W = 7, 5, 3, 2
>>> input = torch.rand(B, C, H, W)
>>> target_idx = torch.randint(low=0, high=C - 1, size=(B, H, W)).long()
>>> target = one_hot(target_idx[:, None, ...], num_classes=C)
>>> self = DiceLoss(reduction='none')
>>> loss = self(input, target)
>>> assert np.broadcast_shapes(loss.shape, input.shape) == input.shape
Return type:

Tensor

monai.losses.Dice[source]#

alias of DiceLoss

monai.losses.dice[source]#

alias of <module ‘monai.losses.dice’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/monai/checkouts/latest/monai/losses/dice.py’>

MaskedDiceLoss#

class monai.losses.MaskedDiceLoss(*args, **kwargs)[source]#

Add an additional masking process before DiceLoss, accept a binary mask ([0, 1]) indicating a region, input and target will be masked by the region: region with mask 1 will keep the original value, region with 0 mask will be converted to 0. Then feed input and target to normal DiceLoss computation. This has the effect of ensuring only the masked region contributes to the loss computation and hence gradient calculation.

__init__(*args, **kwargs)[source]#

Args follow monai.losses.DiceLoss.

forward(input, target, mask=None)[source]#
Parameters:

  • input – the shape should be BNH[WD].

  • target – the shape should be BNH[WD].

  • mask – the shape should B1H[WD] or 11H[WD].

GeneralizedDiceLoss#

class monai.losses.GeneralizedDiceLoss(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, w_type=square, reduction=mean, smooth_nr=1e-05, smooth_dr=1e-05, batch=False, soft_label=False)[source]#

Compute the generalised Dice loss defined in:

Sudre, C. et. al. (2017) Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations. DLMIA 2017.

Adapted from:

NifTK/NiftyNet

__init__(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, w_type=square, reduction=mean, smooth_nr=1e-05, smooth_dr=1e-05, batch=False, soft_label=False)[source]#
Parameters:

  • include_background – If False channel index 0 (background category) is excluded from the calculation.

  • to_onehot_y – whether to convert the target into the one-hot format, using the number of classes inferred from input (input.shape[1]). Defaults to False.

  • sigmoid – If True, apply a sigmoid function to the prediction.

  • softmax – If True, apply a softmax function to the prediction.

  • other_act – callable function to execute other activation layers, Defaults to None. for example: other_act = torch.tanh.

  • w_type – {"square", "simple", "uniform"} Type of function to transform ground truth volume to a weight factor. Defaults to "square".

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • smooth_nr – a small constant added to the numerator to avoid zero.

  • smooth_dr – a small constant added to the denominator to avoid nan.

  • batch – whether to sum the intersection and union areas over the batch dimension before the dividing. Defaults to False, intersection over union is computed from each item in the batch. If True, the class-weighted intersection and union areas are first summed across the batches.

  • soft_label – whether the target contains non-binary values (soft labels) or not. If True a soft label formulation of the loss will be used.

Raises:

  • TypeError – When other_act is not an Optional[Callable].

  • ValueError – When more than 1 of [sigmoid=True, softmax=True, other_act is not None]. Incompatible values.

forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be BNH[WD].

  • target (Tensor) – the shape should be BNH[WD].

Raises:

ValueError – When self.reduction is not one of [“mean”, “sum”, “none”].

Return type:

Tensor

monai.losses.generalized_dice[source]#

alias of GeneralizedDiceLoss

GeneralizedWassersteinDiceLoss#

class monai.losses.GeneralizedWassersteinDiceLoss(dist_matrix, weighting_mode='default', reduction=mean, smooth_nr=1e-05, smooth_dr=1e-05)[source]#

Compute the generalized Wasserstein Dice Loss defined in:

Fidon L. et al. (2017) Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks. BrainLes 2017.

Or its variant (use the option weighting_mode=”GDL”) defined in the Appendix of:

Tilborghs, S. et al. (2020) Comparative study of deep learning methods for the automatic segmentation of lung, lesion and lesion type in CT scans of COVID-19 patients. arXiv preprint arXiv:2007.15546

Adapted from:

LucasFidon/GeneralizedWassersteinDiceLoss

__init__(dist_matrix, weighting_mode='default', reduction=mean, smooth_nr=1e-05, smooth_dr=1e-05)[source]#
Parameters:

  • dist_matrix – 2d tensor or 2d numpy array; matrix of distances between the classes.

  • classes. (It must have dimension C x C where C is the number of)

  • weighting_mode –

    {"default", "GDL"} Specifies how to weight the class-specific sum of errors. Default to "default".

    • "default": (recommended) use the original weighting method as in:

      Fidon L. et al. (2017) Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks. BrainLes 2017.

    • "GDL": use a GDL-like weighting method as in the Appendix of:

      Tilborghs, S. et al. (2020) Comparative study of deep learning methods for the automatic segmentation of lung, lesion and lesion type in CT scans of COVID-19 patients. arXiv preprint arXiv:2007.15546

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • smooth_nr – a small constant added to the numerator to avoid zero.

  • smooth_dr – a small constant added to the denominator to avoid nan.

Raises:

ValueError – When dist_matrix is not a square matrix.

Example

import torch
import numpy as np
from monai.losses import GeneralizedWassersteinDiceLoss

# Example with 3 classes (including the background: label 0).
# The distance between the background class (label 0) and the other classes is the maximum, equal to 1.
# The distance between class 1 and class 2 is 0.5.
dist_mat = np.array([[0.0, 1.0, 1.0], [1.0, 0.0, 0.5], [1.0, 0.5, 0.0]], dtype=np.float32)
wass_loss = GeneralizedWassersteinDiceLoss(dist_matrix=dist_mat)

pred_score = torch.tensor([[1000, 0, 0], [0, 1000, 0], [0, 0, 1000]], dtype=torch.float32)
grnd = torch.tensor([0, 1, 2], dtype=torch.int64)
wass_loss(pred_score, grnd)  # 0
forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be BNH[WD].

  • target (Tensor) – the shape should be BNH[WD].

Return type:

Tensor

wasserstein_distance_map(flat_proba, flat_target)[source]#

Compute the voxel-wise Wasserstein distance between the flattened prediction and the flattened labels (ground_truth) with respect to the distance matrix on the label space M. This corresponds to eq. 6 in:

Fidon L. et al. (2017) Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks. BrainLes 2017.

Parameters:

  • flat_proba (Tensor) – the probabilities of input(predicted) tensor.

  • flat_target (Tensor) – the target tensor.

Return type:

Tensor

monai.losses.generalized_wasserstein_dice[source]#

alias of GeneralizedWassersteinDiceLoss

DiceCELoss#

class monai.losses.DiceCELoss(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, squared_pred=False, jaccard=False, reduction='mean', smooth_nr=1e-05, smooth_dr=1e-05, batch=False, weight=None, lambda_dice=1.0, lambda_ce=1.0, label_smoothing=0.0)[source]#

Compute both Dice loss and Cross Entropy Loss, and return the weighted sum of these two losses. The details of Dice loss is shown in monai.losses.DiceLoss. The details of Cross Entropy Loss is shown in torch.nn.CrossEntropyLoss and torch.nn.BCEWithLogitsLoss(). In this implementation, two deprecated parameters size_average and reduce, and the parameter ignore_index are not supported.

__init__(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, squared_pred=False, jaccard=False, reduction='mean', smooth_nr=1e-05, smooth_dr=1e-05, batch=False, weight=None, lambda_dice=1.0, lambda_ce=1.0, label_smoothing=0.0)[source]#
Parameters:

  • loss. (reduction and weight is used for both losses and other parameters are only used for dice)

  • loss.

  • include_background – if False channel index 0 (background category) is excluded from the calculation.

  • to_onehot_y – whether to convert the target into the one-hot format, using the number of classes inferred from input (input.shape[1]). Defaults to False.

  • sigmoid – if True, apply a sigmoid function to the prediction, only used by the DiceLoss, don’t need to specify activation function for CrossEntropyLoss and BCEWithLogitsLoss.

  • softmax – if True, apply a softmax function to the prediction, only used by the DiceLoss, don’t need to specify activation function for CrossEntropyLoss and BCEWithLogitsLoss.

  • other_act – callable function to execute other activation layers, Defaults to None. for example: other_act = torch.tanh. only used by the DiceLoss, not for the CrossEntropyLoss and BCEWithLogitsLoss.

  • squared_pred – use squared versions of targets and predictions in the denominator or not.

  • jaccard – compute Jaccard Index (soft IoU) instead of dice or not.

  • reduction –

    {"mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean". The dice loss should as least reduce the spatial dimensions, which is different from cross entropy loss, thus here the none option cannot be used.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • smooth_nr – a small constant added to the numerator to avoid zero.

  • smooth_dr – a small constant added to the denominator to avoid nan.

  • batch – whether to sum the intersection and union areas over the batch dimension before the dividing. Defaults to False, a Dice loss value is computed independently from each item in the batch before any reduction.

  • weight – a rescaling weight given to each class for cross entropy loss for CrossEntropyLoss. or a weight of positive examples to be broadcasted with target used as pos_weight for BCEWithLogitsLoss. See torch.nn.CrossEntropyLoss() or torch.nn.BCEWithLogitsLoss() for more information. The weight is also used in DiceLoss.

  • lambda_dice – the trade-off weight value for dice loss. The value should be no less than 0.0. Defaults to 1.0.

  • lambda_ce – the trade-off weight value for cross entropy loss. The value should be no less than 0.0. Defaults to 1.0.

  • label_smoothing – a value in [0, 1] range. If > 0, the labels are smoothed by the given factor to reduce overfitting. Defaults to 0.0.

bce(input, target)[source]#

Compute Binary CrossEntropy loss for the input logits and target in one single class.

Return type:

Tensor

ce(input, target)[source]#

Compute CrossEntropy loss for the input logits and target. Will remove the channel dim according to PyTorch CrossEntropyLoss: https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html?#torch.nn.CrossEntropyLoss.

Return type:

Tensor

forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be BNH[WD].

  • target (Tensor) – the shape should be BNH[WD] or B1H[WD].

Raises:

  • ValueError – When number of dimensions for input and target are different.

  • ValueError – When number of channels for target is neither 1 (without one-hot encoding) nor the same as input.

Returns:

value of the loss.

Return type:

torch.Tensor

DiceFocalLoss#

class monai.losses.DiceFocalLoss(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, squared_pred=False, jaccard=False, reduction='mean', smooth_nr=1e-05, smooth_dr=1e-05, batch=False, gamma=2.0, weight=None, lambda_dice=1.0, lambda_focal=1.0, alpha=None)[source]#

Compute both Dice loss and Focal Loss, and return the weighted sum of these two losses. The details of Dice loss is shown in monai.losses.DiceLoss. The details of Focal Loss is shown in monai.losses.FocalLoss.

gamma and lambda_focal are only used for the focal loss. include_background, weight, reduction, and alpha are used for both losses, and other parameters are only used for dice loss.

__init__(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, squared_pred=False, jaccard=False, reduction='mean', smooth_nr=1e-05, smooth_dr=1e-05, batch=False, gamma=2.0, weight=None, lambda_dice=1.0, lambda_focal=1.0, alpha=None)[source]#
Parameters:

  • include_background – if False channel index 0 (background category) is excluded from the calculation.

  • to_onehot_y – whether to convert the target into the one-hot format, using the number of classes inferred from input (input.shape[1]). Defaults to False.

  • sigmoid – if True, apply a sigmoid function to the prediction, only used by the DiceLoss, don’t need to specify activation function for FocalLoss.

  • softmax – if True, apply a softmax function to the prediction, only used by the DiceLoss, don’t need to specify activation function for FocalLoss.

  • other_act – callable function to execute other activation layers, Defaults to None. for example: other_act = torch.tanh. only used by the DiceLoss, not for FocalLoss.

  • squared_pred – use squared versions of targets and predictions in the denominator or not.

  • jaccard – compute Jaccard Index (soft IoU) instead of dice or not.

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • smooth_nr – a small constant added to the numerator to avoid zero.

  • smooth_dr – a small constant added to the denominator to avoid nan.

  • batch – whether to sum the intersection and union areas over the batch dimension before the dividing. Defaults to False, a Dice loss value is computed independently from each item in the batch before any reduction.

  • gamma – value of the exponent gamma in the definition of the Focal loss.

  • weight – weights to apply to the voxels of each class. If None no weights are applied. The input can be a single value (same weight for all classes), a sequence of values (the length of the sequence should be the same as the number of classes).

  • lambda_dice – the trade-off weight value for dice loss. The value should be no less than 0.0. Defaults to 1.0.

  • lambda_focal – the trade-off weight value for focal loss. The value should be no less than 0.0. Defaults to 1.0.

  • alpha – value of the alpha in the definition of the alpha-balanced Focal loss. The value should be in [0, 1]. Defaults to None.

forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be BNH[WD]. The input should be the original logits due to the restriction of monai.losses.FocalLoss.

  • target (Tensor) – the shape should be BNH[WD] or B1H[WD].

Raises:

  • ValueError – When number of dimensions for input and target are different.

  • ValueError – When number of channels for target is neither 1 (without one-hot encoding) nor the same as input.

Returns:

value of the loss.

Return type:

torch.Tensor

GeneralizedDiceFocalLoss#

class monai.losses.GeneralizedDiceFocalLoss(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, w_type=square, reduction=mean, smooth_nr=1e-05, smooth_dr=1e-05, batch=False, gamma=2.0, weight=None, lambda_gdl=1.0, lambda_focal=1.0)[source]#

Compute both Generalized Dice Loss and Focal Loss, and return their weighted average. The details of Generalized Dice Loss and Focal Loss are available at monai.losses.GeneralizedDiceLoss and monai.losses.FocalLoss.

Parameters:

  • include_background (bool, optional) – if False channel index 0 (background category) is excluded from the calculation. Defaults to True.

  • to_onehot_y – whether to convert the target into the one-hot format, using the number of classes inferred from input (input.shape[1]). Defaults to False.

  • sigmoid (bool, optional) – if True, apply a sigmoid function to the prediction. Defaults to False.

  • softmax (bool, optional) – if True, apply a softmax function to the prediction. Defaults to False.

  • other_act (Optional[Callable], optional) – callable function to execute other activation layers, Defaults to None. for example: other_act = torch.tanh. only used by the GeneralizedDiceLoss, not for the FocalLoss.

  • w_type (Union[Weight, str], optional) – {"square", "simple", "uniform"}. Type of function to transform ground-truth volume to a weight factor. Defaults to "square".

  • reduction (Union[LossReduction, str], optional) – {"none", "mean", "sum"}. Specified the reduction to apply to the output. Defaults to "mean". - "none": no reduction will be applied. - "mean": the sum of the output will be divided by the number of elements in the output. - "sum": the output will be summed.

  • smooth_nr (float, optional) – a small constant added to the numerator to avoid zero. Defaults to 1e-5.

  • smooth_dr (float, optional) – a small constant added to the denominator to avoid nan. Defaults to 1e-5.

  • batch (bool, optional) – whether to sum the intersection and union areas over the batch dimension before the dividing. Defaults to False, i.e., the areas are computed for each item in the batch.

  • gamma (float, optional) – value of the exponent gamma in the definition of the Focal loss. Defaults to 2.0.

  • weight (Optional[Union[Sequence[float], float, int, torch.Tensor]], optional) – weights to apply to the voxels of each class. If None no weights are applied. The input can be a single value (same weight for all classes), a sequence of values (the length of the sequence hould be the same as the number of classes). Defaults to None.

  • lambda_gdl (float, optional) – the trade-off weight value for Generalized Dice Loss. The value should be no less than 0.0. Defaults to 1.0.

  • lambda_focal (float, optional) – the trade-off weight value for Focal Loss. The value should be no less than 0.0. Defaults to 1.0.

Raises:

ValueError – if either lambda_gdl or lambda_focal is less than 0.

forward(input, target)[source]#
Parameters:

  • input (torch.Tensor) – the shape should be BNH[WD]. The input should be the original logits due to the restriction of monai.losses.FocalLoss.

  • target (torch.Tensor) – the shape should be BNH[WD] or B1H[WD].

Raises:

  • ValueError – When number of dimensions for input and target are different.

  • ValueError – When number of channels for target is neither 1 (without one-hot encoding) nor the same as input.

Returns:

value of the loss.

Return type:

torch.Tensor

FocalLoss#

class monai.losses.FocalLoss(include_background=True, to_onehot_y=False, gamma=2.0, alpha=None, weight=None, reduction=mean, use_softmax=False)[source]#

FocalLoss is an extension of BCEWithLogitsLoss that down-weights loss from high confidence correct predictions.

Reimplementation of the Focal Loss described in:

  • [“Focal Loss for Dense Object Detection”](https://arxiv.org/abs/1708.02002), T. Lin et al., ICCV 2017

  • “AnatomyNet: Deep learning for fast and fully automated whole-volume segmentation of head and neck anatomy”, Zhu et al., Medical Physics 2018

Example

>>> import torch
>>> from monai.losses import FocalLoss
>>> from torch.nn import BCEWithLogitsLoss
>>> shape = B, N, *DIMS = 2, 3, 5, 7, 11
>>> input = torch.rand(*shape)
>>> target = torch.rand(*shape)
>>> # Demonstrate equivalence to BCE when gamma=0
>>> fl_g0_criterion = FocalLoss(reduction='none', gamma=0)
>>> fl_g0_loss = fl_g0_criterion(input, target)
>>> bce_criterion = BCEWithLogitsLoss(reduction='none')
>>> bce_loss = bce_criterion(input, target)
>>> assert torch.allclose(fl_g0_loss, bce_loss)
>>> # Demonstrate "focus" by setting gamma > 0.
>>> fl_g2_criterion = FocalLoss(reduction='none', gamma=2)
>>> fl_g2_loss = fl_g2_criterion(input, target)
>>> # Mark easy and hard cases
>>> is_easy = (target > 0.7) & (input > 0.7)
>>> is_hard = (target > 0.7) & (input < 0.3)
>>> easy_loss_g0 = fl_g0_loss[is_easy].mean()
>>> hard_loss_g0 = fl_g0_loss[is_hard].mean()
>>> easy_loss_g2 = fl_g2_loss[is_easy].mean()
>>> hard_loss_g2 = fl_g2_loss[is_hard].mean()
>>> # Gamma > 0 causes the loss function to "focus" on the hard
>>> # cases.  IE, easy cases are downweighted, so hard cases
>>> # receive a higher proportion of the loss.
>>> hard_to_easy_ratio_g2 = hard_loss_g2 / easy_loss_g2
>>> hard_to_easy_ratio_g0 = hard_loss_g0 / easy_loss_g0
>>> assert hard_to_easy_ratio_g2 > hard_to_easy_ratio_g0
__init__(include_background=True, to_onehot_y=False, gamma=2.0, alpha=None, weight=None, reduction=mean, use_softmax=False)[source]#
Parameters:

  • include_background – if False, channel index 0 (background category) is excluded from the loss calculation. If False, alpha is invalid when using softmax.

  • to_onehot_y – whether to convert the label y into the one-hot format. Defaults to False.

  • gamma – value of the exponent gamma in the definition of the Focal loss. Defaults to 2.

  • alpha – value of the alpha in the definition of the alpha-balanced Focal loss. The value should be in [0, 1]. Defaults to None.

  • weight – weights to apply to the voxels of each class. If None no weights are applied. The input can be a single value (same weight for all classes), a sequence of values (the length of the sequence should be the same as the number of classes. If not include_background, the number of classes should not include the background category class 0). The value/values should be no less than 0. Defaults to None.

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • use_softmax – whether to use softmax to transform the original logits into probabilities. If True, softmax is used. If False, sigmoid is used. Defaults to False.

Example

>>> import torch
>>> from monai.losses import FocalLoss
>>> pred = torch.tensor([[1, 0], [0, 1], [1, 0]], dtype=torch.float32)
>>> grnd = torch.tensor([[0], [1], [0]], dtype=torch.int64)
>>> fl = FocalLoss(to_onehot_y=True)
>>> fl(pred, grnd)
forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be BNH[WD], where N is the number of classes. The input should be the original logits since it will be transformed by a sigmoid/softmax in the forward function.

  • target (Tensor) – the shape should be BNH[WD] or B1H[WD], where N is the number of classes.

Raises:

  • ValueError – When input and target (after one hot transform if set) have different shapes.

  • ValueError – When self.reduction is not one of [“mean”, “sum”, “none”].

  • ValueError – When self.weight is a sequence and the length is not equal to the number of classes.

  • ValueError – When self.weight is/contains a value that is less than 0.

Return type:

Tensor

TverskyLoss#

class monai.losses.TverskyLoss(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, alpha=0.5, beta=0.5, reduction=mean, smooth_nr=1e-05, smooth_dr=1e-05, batch=False, soft_label=False)[source]#

Compute the Tversky loss defined in:

Sadegh et al. (2017) Tversky loss function for image segmentation using 3D fully convolutional deep networks. (https://arxiv.org/abs/1706.05721)

Wang, Z. et. al. (2023) Dice Semimetric Losses: Optimizing the Dice Score with Soft Labels. MICCAI 2023.

Adapted from:

NifTK/NiftyNet

__init__(include_background=True, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, alpha=0.5, beta=0.5, reduction=mean, smooth_nr=1e-05, smooth_dr=1e-05, batch=False, soft_label=False)[source]#
Parameters:

  • include_background – If False channel index 0 (background category) is excluded from the calculation.

  • to_onehot_y – whether to convert y into the one-hot format. Defaults to False.

  • sigmoid – If True, apply a sigmoid function to the prediction.

  • softmax – If True, apply a softmax function to the prediction.

  • other_act – if don’t want to use sigmoid or softmax, use other callable function to execute other activation layers, Defaults to None. for example: other_act = torch.tanh.

  • alpha – weight of false positives

  • beta – weight of false negatives

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • smooth_nr – a small constant added to the numerator to avoid zero.

  • smooth_dr – a small constant added to the denominator to avoid nan.

  • batch – whether to sum the intersection and union areas over the batch dimension before the dividing. Defaults to False, a Dice loss value is computed independently from each item in the batch before any reduction.

  • soft_label – whether the target contains non-binary values (soft labels) or not. If True a soft label formulation of the loss will be used.

Raises:

  • TypeError – When other_act is not an Optional[Callable].

  • ValueError – When more than 1 of [sigmoid=True, softmax=True, other_act is not None]. Incompatible values.

forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be BNH[WD].

  • target (Tensor) – the shape should be BNH[WD].

Raises:

ValueError – When self.reduction is not one of [“mean”, “sum”, “none”].

Return type:

Tensor

ContrastiveLoss#

class monai.losses.ContrastiveLoss(temperature=0.5, batch_size=-1)[source]#

Compute the Contrastive loss defined in:

Chen, Ting, et al. “A simple framework for contrastive learning of visual representations.” International conference on machine learning. PMLR, 2020. (http://proceedings.mlr.press/v119/chen20j.html)

Adapted from:

Sara-Ahmed/SiT

__init__(temperature=0.5, batch_size=-1)[source]#
Parameters:

temperature (float) – Can be scaled between 0 and 1 for learning from negative samples, ideally set to 0.5.

Raises:

  • ValueError – When an input of dimension length > 2 is passed

  • ValueError – When input and target are of different shapes

forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be B[F].

  • target (Tensor) – the shape should be B[F].

Return type:

Tensor

BarlowTwinsLoss#

class monai.losses.BarlowTwinsLoss(lambd=0.005)[source]#

The Barlow Twins cost function takes the representations extracted by a neural network from two distorted views and seeks to make the cross-correlation matrix of the two representations tend towards identity. This encourages the neural network to learn similar representations with the least amount of redundancy. This cost function can be used in particular in multimodal learning to work on representations from two modalities. The most common use case is for unsupervised learning, where data augmentations are used to generate 2 distorted views of the same sample to force the encoder to extract useful features for downstream tasks.

Zbontar, Jure, et al. “Barlow Twins: Self-Supervised Learning via Redundancy Reduction” International conference on machine learning. PMLR, 2020. (http://proceedings.mlr.press/v139/zbontar21a/zbontar21a.pdf)

Adapted from:

facebookresearch/barlowtwins

__init__(lambd=0.005)[source]#
Parameters:

lamb – Can be any float to handle the informativeness and invariance trade-off. Ideally set to 5e-3.

Raises:

  • ValueError – When an input of dimension length > 2 is passed

  • ValueError – When input and target are of different shapes

  • ValueError – When batch size is less than or equal to 1

forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be B[F].

  • target (Tensor) – the shape should be B[F].

Return type:

Tensor

HausdorffDTLoss#

class monai.losses.HausdorffDTLoss(alpha=2.0, include_background=False, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, reduction=mean, batch=False)[source]#

Compute channel-wise binary Hausdorff loss based on distance transform. It can support both multi-classes and multi-labels tasks. The data input (BNHW[D] where N is number of classes) is compared with ground truth target (BNHW[D]).

Note that axis N of input is expected to be logits or probabilities for each class, if passing logits as input, must set sigmoid=True or softmax=True, or specifying other_act. And the same axis of target can be 1 or N (one-hot format).

The original paper: Karimi, D. et. al. (2019) Reducing the Hausdorff Distance in Medical Image Segmentation with Convolutional Neural Networks, IEEE Transactions on medical imaging, 39(2), 499-513

__init__(alpha=2.0, include_background=False, to_onehot_y=False, sigmoid=False, softmax=False, other_act=None, reduction=mean, batch=False)[source]#
Parameters:

  • include_background – if False, channel index 0 (background category) is excluded from the calculation. if the non-background segmentations are small compared to the total image size they can get overwhelmed by the signal from the background so excluding it in such cases helps convergence.

  • to_onehot_y – whether to convert the target into the one-hot format, using the number of classes inferred from input (input.shape[1]). Defaults to False.

  • sigmoid – if True, apply a sigmoid function to the prediction.

  • softmax – if True, apply a softmax function to the prediction.

  • other_act – callable function to execute other activation layers, Defaults to None. for example: other_act = torch.tanh.

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • batch – whether to sum the intersection and union areas over the batch dimension before the dividing. Defaults to False, a loss value is computed independently from each item in the batch before any reduction.

Raises:

  • TypeError – When other_act is not an Optional[Callable].

  • ValueError – When more than 1 of [sigmoid=True, softmax=True, other_act is not None]. Incompatible values.

distance_field(img)#

Generate distance transform.

Parameters:

img (np.ndarray) – input mask as NCHWD or NCHW.

Returns:

Distance field.

Return type:

np.ndarray

forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be BNHW[D], where N is the number of classes.

  • target (Tensor) – the shape should be BNHW[D] or B1HW[D], where N is the number of classes.

Raises:

  • ValueError – If the input is not 2D (NCHW) or 3D (NCHWD).

  • AssertionError – When input and target (after one hot transform if set) have different shapes.

  • ValueError – When self.reduction is not one of [“mean”, “sum”, “none”].

Example

>>> import torch
>>> from monai.losses.hausdorff_loss import HausdorffDTLoss
>>> from monai.networks.utils import one_hot
>>> B, C, H, W = 7, 5, 3, 2
>>> input = torch.rand(B, C, H, W)
>>> target_idx = torch.randint(low=0, high=C - 1, size=(B, H, W)).long()
>>> target = one_hot(target_idx[:, None, ...], num_classes=C)
>>> self = HausdorffDTLoss(reduction='none')
>>> loss = self(input, target)
>>> assert np.broadcast_shapes(loss.shape, input.shape) == input.shape
Return type:

Tensor

SoftclDiceLoss#

class monai.losses.SoftclDiceLoss(iter_=3, smooth=1.0)[source]#

Compute the Soft clDice loss defined in:

Shit et al. (2021) clDice – A Novel Topology-Preserving Loss Function for Tubular Structure Segmentation. (https://arxiv.org/abs/2003.07311)

Adapted from:

jocpae/clDice

__init__(iter_=3, smooth=1.0)[source]#
Parameters:

  • iter – Number of iterations for skeletonization

  • smooth (float) – Smoothing parameter

forward(y_true, y_pred)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Return type:

Tensor

SoftDiceclDiceLoss#

class monai.losses.SoftDiceclDiceLoss(iter_=3, alpha=0.5, smooth=1.0)[source]#

Compute the Soft clDice loss defined in:

Shit et al. (2021) clDice – A Novel Topology-Preserving Loss Function for Tubular Structure Segmentation. (https://arxiv.org/abs/2003.07311)

Adapted from:

jocpae/clDice

__init__(iter_=3, alpha=0.5, smooth=1.0)[source]#
Parameters:

  • iter – Number of iterations for skeletonization

  • smooth (float) – Smoothing parameter

  • alpha (float) – Weighing factor for cldice

forward(y_true, y_pred)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Return type:

Tensor

NACLLoss#

class monai.losses.NACLLoss(classes, dim, kernel_size=3, kernel_ops='mean', distance_type='l1', alpha=0.1, sigma=1.0)[source]#

Neighbor-Aware Calibration Loss (NACL) is primarily developed for developing calibrated models in image segmentation. NACL computes standard cross-entropy loss with a linear penalty that enforces the logit distributions to match a soft class proportion of surrounding pixel.

Murugesan, Balamurali, et al. “Trust your neighbours: Penalty-based constraints for model calibration.” International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023. https://arxiv.org/abs/2303.06268

Murugesan, Balamurali, et al. “Neighbor-Aware Calibration of Segmentation Networks with Penalty-Based Constraints.” https://arxiv.org/abs/2401.14487

__init__(classes, dim, kernel_size=3, kernel_ops='mean', distance_type='l1', alpha=0.1, sigma=1.0)[source]#
Parameters:

  • classes (int) – number of classes

  • dim (int) – dimension of data (supports 2d and 3d)

  • kernel_size (int) – size of the spatial kernel

  • distance_type (str) – l1/l2 distance between spatial kernel and predicted logits

  • alpha (float) – weightage between cross entropy and logit constraint

  • sigma (float) – sigma of gaussian

forward(inputs, targets)[source]#

Computes standard cross-entropy loss and constraints it neighbor aware logit penalty.

Parameters:

  • inputs (Tensor) – the shape should be BNH[WD], where N is the number of classes.

  • targets (Tensor) – the shape should be BH[WD].

Returns:

value of the loss.

Return type:

torch.Tensor

Example

>>> import torch
>>> from monai.losses import NACLLoss
>>> B, N, H, W = 8, 3, 64, 64
>>> input = torch.rand(B, N, H, W)
>>> target = torch.randint(0, N, (B, H, W))
>>> criterion = NACLLoss(classes = N, dim = 2)
>>> loss = criterion(input, target)
get_constr_target(mask)[source]#

Converts the mask to one hot represenation and is smoothened with the selected spatial filter.

Parameters:

mask (Tensor) – the shape should be BH[WD].

Returns:

the shape would be BNH[WD], N being number of classes.

Return type:

torch.Tensor

Registration Losses#

BendingEnergyLoss#

class monai.losses.BendingEnergyLoss(normalize=False, reduction=mean)[source]#

Calculate the bending energy based on second-order differentiation of pred using central finite difference.

For more information, see Project-MONAI/tutorials.

Adapted from:

DeepReg (DeepRegNet/DeepReg)

__init__(normalize=False, reduction=mean)[source]#
Parameters:

  • normalize – Whether to divide out spatial sizes in order to make the computation roughly invariant to image scale (i.e. vector field sampling resolution). Defaults to False.

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

forward(pred)[source]#
Parameters:

pred (Tensor) – the shape should be BCH(WD)

Raises:

  • ValueError – When self.reduction is not one of [“mean”, “sum”, “none”].

  • ValueError – When pred is not 3-d, 4-d or 5-d.

  • ValueError – When any spatial dimension of pred has size less than or equal to 4.

  • ValueError – When the number of channels of pred does not match the number of spatial dimensions.

Return type:

Tensor

DiffusionLoss#

class monai.losses.DiffusionLoss(normalize=False, reduction=mean)[source]#

Calculate the diffusion based on first-order differentiation of pred using central finite difference. For the original paper, please refer to VoxelMorph: A Learning Framework for Deformable Medical Image Registration, Guha Balakrishnan, Amy Zhao, Mert R. Sabuncu, John Guttag, Adrian V. Dalca IEEE TMI: Transactions on Medical Imaging. 2019. eprint arXiv:1809.05231.

For more information, see Project-MONAI/tutorials.

Adapted from:

VoxelMorph (voxelmorph/voxelmorph)

__init__(normalize=False, reduction=mean)[source]#
Parameters:

  • normalize – Whether to divide out spatial sizes in order to make the computation roughly invariant to image scale (i.e. vector field sampling resolution). Defaults to False.

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

forward(pred)[source]#
Parameters:

pred (Tensor) – Predicted dense displacement field (DDF) with shape BCH[WD], where C is the number of spatial dimensions. Note that diffusion loss can only be calculated when the sizes of the DDF along all spatial dimensions are greater than 2.

Raises:

  • ValueError – When self.reduction is not one of [“mean”, “sum”, “none”].

  • ValueError – When pred is not 3-d, 4-d or 5-d.

  • ValueError – When any spatial dimension of pred has size less than or equal to 2.

  • ValueError – When the number of channels of pred does not match the number of spatial dimensions.

Return type:

Tensor

LocalNormalizedCrossCorrelationLoss#

class monai.losses.LocalNormalizedCrossCorrelationLoss(spatial_dims=3, kernel_size=3, kernel_type='rectangular', reduction=mean, smooth_nr=0.0, smooth_dr=1e-05)[source]#

Local squared zero-normalized cross-correlation. The loss is based on a moving kernel/window over the y_true/y_pred, within the window the square of zncc is calculated. The kernel can be a rectangular / triangular / gaussian window. The final loss is the averaged loss over all windows.

Adapted from:

voxelmorph/voxelmorph DeepReg (DeepRegNet/DeepReg)

__init__(spatial_dims=3, kernel_size=3, kernel_type='rectangular', reduction=mean, smooth_nr=0.0, smooth_dr=1e-05)[source]#
Parameters:

  • spatial_dims – number of spatial dimensions, {1, 2, 3}. Defaults to 3.

  • kernel_size – kernel spatial size, must be odd.

  • kernel_type – {"rectangular", "triangular", "gaussian"}. Defaults to "rectangular".

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • smooth_nr – a small constant added to the numerator to avoid nan.

  • smooth_dr – a small constant added to the denominator to avoid nan.

forward(pred, target)[source]#
Parameters:

  • pred (Tensor) – the shape should be BNH[WD].

  • target (Tensor) – the shape should be BNH[WD].

Raises:

ValueError – When self.reduction is not one of [“mean”, “sum”, “none”].

Return type:

Tensor

GlobalMutualInformationLoss#

class monai.losses.GlobalMutualInformationLoss(kernel_type='gaussian', num_bins=23, sigma_ratio=0.5, reduction=mean, smooth_nr=1e-07, smooth_dr=1e-07)[source]#

Differentiable global mutual information loss via Parzen windowing method.

Reference:

https://dspace.mit.edu/handle/1721.1/123142, Section 3.1, equation 3.1-3.5, Algorithm 1

__init__(kernel_type='gaussian', num_bins=23, sigma_ratio=0.5, reduction=mean, smooth_nr=1e-07, smooth_dr=1e-07)[source]#
Parameters:

  • kernel_type –

    {"gaussian", "b-spline"} "gaussian": adapted from DeepReg Reference: https://dspace.mit.edu/handle/1721.1/123142, Section 3.1, equation 3.1-3.5, Algorithm 1. "b-spline": based on the method of Mattes et al [1,2] and adapted from ITK .. rubric:: References

    [1] “Nonrigid multimodality image registration”

    D. Mattes, D. R. Haynor, H. Vesselle, T. Lewellen and W. Eubank Medical Imaging 2001: Image Processing, 2001, pp. 1609-1620.

    [2] “PET-CT Image Registration in the Chest Using Free-form Deformations”

    D. Mattes, D. R. Haynor, H. Vesselle, T. Lewellen and W. Eubank IEEE Transactions in Medical Imaging. Vol.22, No.1, January 2003. pp.120-128.

  • num_bins – number of bins for intensity

  • sigma_ratio – a hyper param for gaussian function

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • smooth_nr – a small constant added to the numerator to avoid nan.

  • smooth_dr – a small constant added to the denominator to avoid nan.

forward(pred, target)[source]#
Parameters:

  • pred (Tensor) – the shape should be B[NDHW].

  • target (Tensor) – the shape should be same as the pred shape.

Raises:

ValueError – When self.reduction is not one of [“mean”, “sum”, “none”].

Return type:

Tensor

parzen_windowing_b_spline(img, order)[source]#

Parzen windowing with b-spline kernel (adapted from ITK)

Parameters:

  • img (Tensor) – the shape should be B[NDHW].

  • order (int) – int.

Return type:

tuple[Tensor, Tensor]

parzen_windowing_gaussian(img)[source]#

Parzen windowing with gaussian kernel (adapted from DeepReg implementation) Note: the input is expected to range between 0 and 1 :type img: Tensor :param img: the shape should be B[NDHW].

Return type:

tuple[Tensor, Tensor]

Reconstruction Losses#

SSIMLoss#

class monai.losses.ssim_loss.SSIMLoss(spatial_dims, data_range=1.0, kernel_type=gaussian, win_size=11, kernel_sigma=1.5, k1=0.01, k2=0.03, reduction=mean)[source]#

Compute the loss function based on the Structural Similarity Index Measure (SSIM) Metric.

For more info, visit

https://vicuesoft.com/glossary/term/ssim-ms-ssim/

SSIM reference paper:

Wang, Zhou, et al. “Image quality assessment: from error visibility to structural similarity.” IEEE transactions on image processing 13.4 (2004): 600-612.

__init__(spatial_dims, data_range=1.0, kernel_type=gaussian, win_size=11, kernel_sigma=1.5, k1=0.01, k2=0.03, reduction=mean)[source]#
Parameters:

  • spatial_dims – number of spatial dimensions of the input images.

  • data_range – value range of input images. (usually 1.0 or 255)

  • kernel_type – type of kernel, can be “gaussian” or “uniform”.

  • win_size – window size of kernel

  • kernel_sigma – standard deviation for Gaussian kernel.

  • k1 – stability constant used in the luminance denominator

  • k2 – stability constant used in the contrast denominator

  • reduction – {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean". - "none": no reduction will be applied. - "mean": the sum of the output will be divided by the number of elements in the output. - "sum": the output will be summed.

forward(input, target)[source]#
Parameters:

  • input (Tensor) – batch of predicted images with shape (batch_size, channels, spatial_dim1, spatial_dim2[, spatial_dim3])

  • target (Tensor) – batch of target images with shape (batch_size, channels, spatial_dim1, spatial_dim2[, spatial_dim3])

Return type:

Tensor

Returns:

1 minus the ssim index (recall this is meant to be a loss function)

Example

import torch

# 2D data
x = torch.ones([1,1,10,10])/2
y = torch.ones([1,1,10,10])/2
print(1-SSIMLoss(spatial_dims=2)(x,y))

# pseudo-3D data
x = torch.ones([1,5,10,10])/2  # 5 could represent number of slices
y = torch.ones([1,5,10,10])/2
print(1-SSIMLoss(spatial_dims=2)(x,y))

# 3D data
x = torch.ones([1,1,10,10,10])/2
y = torch.ones([1,1,10,10,10])/2
print(1-SSIMLoss(spatial_dims=3)(x,y))

PatchAdversarialLoss#

class monai.losses.PatchAdversarialLoss(reduction=mean, criterion=least_squares, no_activation_leastsq=False)[source]#

Calculates an adversarial loss on a Patch Discriminator or a Multi-scale Patch Discriminator. Warning: due to the possibility of using different criterions, the output of the discrimination mustn’t be passed to a final activation layer. That is taken care of internally within the loss.

Parameters:

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

  • criterion – which criterion (hinge, least_squares or bce) you want to use on the discriminators outputs. Depending on the criterion, a different activation layer will be used. Make sure you don’t run the outputs through an activation layer prior to calling the loss.

  • no_activation_leastsq – if True, the activation layer in the case of least-squares is removed.

forward(input, target_is_real, for_discriminator)[source]#
Parameters:

  • input – output of Multi-Scale Patch Discriminator or Patch Discriminator; being a list of tensors or a tensor; they shouldn’t have gone through an activation layer.

  • target_is_real – whereas the input corresponds to discriminator output for real or fake images

  • for_discriminator – whereas this is being calculated for discriminator or generator loss. In the last case, target_is_real is set to True, as the generator wants the input to be dimmed as real.

Returns: if reduction is None, returns a list with the loss tensors of each discriminator if multi-scale

discriminator is active, or the loss tensor if there is just one discriminator. Otherwise, it returns the summed or mean loss over the tensor and discriminator/s.

get_target_tensor(input, target_is_real)[source]#

Gets the ground truth tensor for the discriminator depending on whether the input is real or fake.

Parameters:

  • input (Tensor) – input tensor from the discriminator (output of discriminator, or output of one of the multi-scale

  • shape. (discriminator). This is used to match the)

  • target_is_real (bool) – whether the input is real or wannabe-real (1s) or fake (0s).

Returns:

Return type:

Tensor

get_zero_tensor(input)[source]#

Gets a zero tensor.

Parameters:

input (Tensor) – tensor which shape you want the zeros tensor to correspond to.

Returns:

Return type:

Tensor

PerceptualLoss#

class monai.losses.PerceptualLoss(spatial_dims, network_type=alex, is_fake_3d=True, fake_3d_ratio=0.5, cache_dir=None, pretrained=True, pretrained_path=None, pretrained_state_dict_key=None, channel_wise=False)[source]#

Perceptual loss using features from pretrained deep neural networks trained. The function supports networks pretrained on: ImageNet that use the LPIPS approach from Zhang, et al. “The unreasonable effectiveness of deep features as a perceptual metric.” https://arxiv.org/abs/1801.03924 ; RadImagenet from Mei, et al. “RadImageNet: An Open Radiologic Deep Learning Research Dataset for Effective Transfer Learning” https://pubs.rsna.org/doi/full/10.1148/ryai.210315 ; MedicalNet from Chen et al. “Med3D: Transfer Learning for 3D Medical Image Analysis” https://arxiv.org/abs/1904.00625 ; and ResNet50 from Torchvision: https://pytorch.org/vision/main/models/generated/torchvision.models.resnet50.html .

The fake 3D implementation is based on a 2.5D approach where we calculate the 2D perceptual loss on slices from all three axes and average. The full 3D approach uses a 3D network to calculate the perceptual loss. MedicalNet networks are only compatible with 3D inputs and support channel-wise loss.

Parameters:

  • spatial_dims – number of spatial dimensions.

  • network_type – {"alex", "vgg", "squeeze", "radimagenet_resnet50",

  • "medicalnet_resnet10_23datasets" – Specifies the network architecture to use. Defaults to "alex".

  • "medicalnet_resnet50_23datasets" – Specifies the network architecture to use. Defaults to "alex".

  • "resnet50"} – Specifies the network architecture to use. Defaults to "alex".

  • is_fake_3d – if True use 2.5D approach for a 3D perceptual loss.

  • fake_3d_ratio – ratio of how many slices per axis are used in the 2.5D approach.

  • cache_dir – path to cache directory to save the pretrained network weights.

  • pretrained – whether to load pretrained weights. This argument only works when using networks from LIPIS or Torchvision. Defaults to "True".

  • pretrained_path – if pretrained is True, users can specify a weights file to be loaded via using this argument. This argument only works when "network_type" is “resnet50”. Defaults to None.

  • pretrained_state_dict_key – if pretrained_path is not None, this argument is used to extract the expected state dict. This argument only works when "network_type" is “resnet50”. Defaults to None.

  • channel_wise – if True, the loss is returned per channel. Otherwise the loss is averaged over the channels. Defaults to False.

forward(input, target)[source]#
Parameters:

  • input (Tensor) – the shape should be BNHW[D].

  • target (Tensor) – the shape should be BNHW[D].

Return type:

Tensor

JukeboxLoss#

class monai.losses.JukeboxLoss(spatial_dims, fft_signal_size=None, fft_norm='ortho', reduction=mean)[source]#

Calculate spectral component based on the magnitude of Fast Fourier Transform (FFT).

Based on:

Dhariwal, et al. ‘Jukebox: A generative model for music.’ https://arxiv.org/abs/2005.00341

Parameters:

  • spatial_dims – number of spatial dimensions.

  • fft_signal_size – signal size in the transformed dimensions. See torch.fft.fftn() for more information.

  • fft_norm – {"forward", "backward", "ortho"} Specifies the normalization mode in the fft. See torch.fft.fftn() for more information.

  • reduction –

    {"none", "mean", "sum"} Specifies the reduction to apply to the output. Defaults to "mean".

    • "none": no reduction will be applied.

    • "mean": the sum of the output will be divided by the number of elements in the output.

    • "sum": the output will be summed.

forward(input, target)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Return type:

Tensor

SURELoss#

class monai.losses.SURELoss(perturb_noise=None, eps=None)[source]#

Calculate the Stein’s Unbiased Risk Estimator (SURE) loss for a given operator.

This is a differentiable loss function that can be used to train/guide an operator (e.g. neural network), where the pseudo ground truth is available but the reference ground truth is not. For example, in the MRI reconstruction, the pseudo ground truth is the zero-filled reconstruction and the reference ground truth is the fully sampled reconstruction. Often, the reference ground truth is not available due to the lack of fully sampled data.

The original SURE loss is proposed in [1]. The SURE loss used for guiding the diffusion model based MRI reconstruction is proposed in [2].

Reference

[1] Stein, C.M.: Estimation of the mean of a multivariate normal distribution. Annals of Statistics

[2] B. Ozturkler et al. SMRD: SURE-based Robust MRI Reconstruction with Diffusion Models. (https://arxiv.org/pdf/2310.01799.pdf)

__init__(perturb_noise=None, eps=None)[source]#
Parameters:

  • perturb_noise (torch.Tensor, optional) – The noise vector of shape

  • (B (B, 2, H, W)

  • C (B, 2, H, W)

  • H (B, 2, H, W)

  • input (For real)

  • is (the shape)

  • input

  • is

  • eps (float, optional) – The perturbation scalar. Defaults to None.

forward(operator, x, y_pseudo_gt, y_ref=None, complex_input=False)[source]#
Parameters:

  • operator (function) – The operator function that takes in an input

  • compute (tensor x and returns an output tensor y. We will use this to)

  • specifically (the divergence. More)

  • a (we will perturb the input x by)

  • output (and the reference)

  • output

  • x (torch.Tensor) – The input tensor of shape (B, C, H, W) to the

  • 2 (W) used to compute the L2 loss. C=1 or) – For complex input, the shape is (B, 2, H, W) aka

  • input (B, 2, H, W)

  • is (shape)

  • y_pseudo_gt (same shape as) – The pseudo ground truth tensor of shape

  • (B – For complex

  • C – For complex

  • H – For complex

  • 2 – For complex

  • input

  • is

  • is

  • y_ref (torch.Tensor, optional) – The reference output tensor of the

  • y_pseudo_gt

Returns:

The SURE loss scalar.

Return type:

sure_loss (torch.Tensor)

Loss Wrappers#

MultiScaleLoss#

class monai.losses.MultiScaleLoss(loss, scales=None, kernel='gaussian', reduction=mean)[source]#

This is a wrapper class. It smooths the input and target at different scales before passing them into the wrapped loss function.

Adapted from:

DeepReg (DeepRegNet/DeepReg)

__init__(loss, scales=None, kernel='gaussian', reduction=mean)[source]#
Parameters:

  • loss – loss function to be wrapped

  • scales – list of scalars or None, if None, do not apply any scaling.

  • kernel – gaussian or cauchy.

forward(y_true, y_pred)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Return type:

Tensor

MaskedLoss#

class monai.losses.MaskedLoss(loss, *loss_args, **loss_kwargs)[source]#

This is a wrapper class for the loss functions. It allows for additional weighting masks to be applied to both input and target.

See also

__init__(loss, *loss_args, **loss_kwargs)[source]#
Parameters:

  • loss – loss function to be wrapped, this could be a loss class or an instance of a loss class.

  • loss_args – arguments to the loss function’s constructor if loss is a class.

  • loss_kwargs – keyword arguments to the loss function’s constructor if loss is a class.

forward(input, target, mask=None)[source]#
Parameters:

  • input (Tensor) – the shape should be BNH[WD].

  • target (Tensor) – the shape should be BNH[WD].

  • mask (Optional[Tensor]) – the shape should be B1H[WD] or 11H[WD].

Return type:

Tensor

DeepSupervisionLoss#

class monai.losses.DeepSupervisionLoss(loss, weight_mode='exp', weights=None)[source]#

Wrapper class around the main loss function to accept a list of tensors returned from a deeply supervised networks. The final loss is computed as the sum of weighted losses for each of deep supervision levels.

__init__(loss, weight_mode='exp', weights=None)[source]#
Parameters:

  • loss – main loss instance, e.g DiceLoss().

  • weight_mode – {"same", "exp", "two"} Specifies the weights calculation for each image level. Defaults to "exp". - "same": all weights are equal to 1. - "exp": exponentially decreasing weights by a power of 2: 1, 0.5, 0.25, 0.125, etc . - "two": equal smaller weights for lower levels: 1, 0.5, 0.5, 0.5, 0.5, etc

  • weights – a list of weights to apply to each deeply supervised sub-loss, if provided, this will be used regardless of the weight_mode

forward(input, target)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Return type:

Tensor

get_loss(input, target)[source]#

Calculates a loss output accounting for differences in shapes, and downsizing targets if necessary (using nearest neighbor interpolation) Generally downsizing occurs for all level, except for the first (level==0)

Return type:

Tensor

get_weights(levels=1)[source]#

Calculates weights for a given number of scale levels

Return type:

list[float]

On this page

This Page