Visualizations

Tensorboard visuals

monai.visualize.img2tensorboard.add_animated_gif(writer, tag, image_tensor, max_out, scale_factor, global_step=None)[source]

Creates an animated gif out of an image tensor in ‘CHWD’ format and writes it with SummaryWriter.

Parameters
  • writer (SummaryWriter) – Tensorboard SummaryWriter to write to

  • tag (str) – Data identifier

  • image_tensor (Union[ndarray, Tensor]) – tensor for the image to add, expected to be in CHWD format

  • max_out (int) – maximum number of slices to animate through

  • scale_factor (float) – amount to multiply values by. If the image data is between 0 and 1, using 255 for this value will scale it to displayable range

  • global_step (Optional[int]) – Global step value to record

Return type

None

monai.visualize.img2tensorboard.add_animated_gif_no_channels(writer, tag, image_tensor, max_out, scale_factor, global_step=None)[source]

Creates an animated gif out of an image tensor in ‘HWD’ format that does not have a channel dimension and writes it with SummaryWriter. This is similar to the “add_animated_gif” after inserting a channel dimension of 1.

Parameters
  • writer (SummaryWriter) – Tensorboard SummaryWriter to write to

  • tag (str) – Data identifier

  • image_tensor (Union[ndarray, Tensor]) – tensor for the image to add, expected to be in CHWD format

  • max_out (int) – maximum number of slices to animate through

  • scale_factor (float) – amount to multiply values by. If the image data is between 0 and 1, using 255 for this value will scale it to displayable range

  • global_step (Optional[int]) – Global step value to record

Return type

None

monai.visualize.img2tensorboard.make_animated_gif_summary(tag, image, max_out=3, animation_axes=(3), image_axes=(1, 2), other_indices=None, scale_factor=1.0)[source]

Creates an animated gif out of an image tensor in ‘CHWD’ format and returns Summary.

Parameters
  • tag (str) – Data identifier

  • image (Union[ndarray, Tensor]) – The image, expected to be in CHWD format

  • max_out (int) – maximum number of slices to animate through

  • animation_axes (Sequence[int]) – axis to animate on (not currently used)

  • image_axes (Sequence[int]) – axes of image (not currently used)

  • other_indices (Optional[Dict]) – (not currently used)

  • scale_factor (float) – amount to multiply values by. if the image data is between 0 and 1, using 255 for this value will scale it to displayable range

Return type

Summary

monai.visualize.img2tensorboard.plot_2d_or_3d_image(data, step, writer, index=0, max_channels=1, max_frames=64, tag='output')[source]

Plot 2D or 3D image on the TensorBoard, 3D image will be converted to GIF image.

Note

Plot 3D or 2D image(with more than 3 channels) as separate images.

Parameters
  • data (Union[Tensor, ndarray]) – target data to be plotted as image on the TensorBoard. The data is expected to have ‘NCHW[D]’ dimensions, and only plot the first in the batch.

  • step (int) – current step to plot in a chart.

  • writer (SummaryWriter) – specify TensorBoard SummaryWriter to plot the image.

  • index (int) – plot which element in the input data batch, default is the first element.

  • max_channels (int) – number of channels to plot.

  • max_frames (int) – number of frames for 2D-t plot.

  • tag (str) – tag of the plotted image on TensorBoard.

Return type

None

Class activation map

class monai.visualize.class_activation_maps.CAM(nn_module, target_layers, fc_layers='fc', upsampler=<function default_upsampler>, postprocessing=<function default_normalizer>)[source]

Compute class activation map from the last fully-connected layers before the spatial pooling.

Examples

# densenet 2d
from monai.networks.nets import densenet121
from monai.visualize import CAM

model_2d = densenet121(spatial_dims=2, in_channels=1, out_channels=3)
cam = CAM(nn_module=model_2d, target_layers="class_layers.relu", fc_layers="class_layers.out")
result = cam(x=torch.rand((1, 1, 48, 64)))

# resnet 2d
from monai.networks.nets import se_resnet50
from monai.visualize import CAM

model_2d = se_resnet50(spatial_dims=2, in_channels=3, num_classes=4)
cam = CAM(nn_module=model_2d, target_layers="layer4", fc_layers="last_linear")
result = cam(x=torch.rand((2, 3, 48, 64)))
Parameters
  • nn_module – the model to be visualised

  • target_layers (str) – name of the model layer to generate the feature map.

  • fc_layers (Union[str, Callable]) – a string or a callable used to get fully-connected weights to compute activation map from the target_layers (without pooling). and evaluate it at every spatial location.

  • upsampler – an upsampling method to upsample the feature map.

  • postprocessing (Callable) – a callable that applies on the upsampled feature map.

compute_map(x, class_idx=None, layer_idx=- 1)[source]

Compute the actual feature map with input tensor x.

feature_map_size(input_size, device='cpu', layer_idx=- 1)[source]

Computes the actual feature map size given nn_module and the target_layer name.

Parameters
  • input_size – shape of the input tensor

  • device – the device used to initialise the input tensor

  • layer_idx – index of the target layer if there are multiple target layers. Defaults to -1.

Returns

shape of the actual feature map.

class monai.visualize.class_activation_maps.GradCAM(nn_module, target_layers, upsampler=<function default_upsampler>, postprocessing=<function default_normalizer>)[source]

Computes Gradient-weighted Class Activation Mapping (Grad-CAM). This implementation is based on:

Selvaraju et al., Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization, https://arxiv.org/abs/1610.02391

Examples

# densenet 2d
from monai.networks.nets import densenet121
from monai.visualize import GradCAM

model_2d = densenet121(spatial_dims=2, in_channels=1, out_channels=3)
cam = GradCAM(nn_module=model_2d, target_layers="class_layers.relu")
result = cam(x=torch.rand((1, 1, 48, 64)))

# resnet 2d
from monai.networks.nets import se_resnet50
from monai.visualize import GradCAM

model_2d = se_resnet50(spatial_dims=2, in_channels=3, num_classes=4)
cam = GradCAM(nn_module=model_2d, target_layers="layer4")
result = cam(x=torch.rand((2, 3, 48, 64)))
Parameters
  • nn_module – the model to be used to generate the visualisations.

  • target_layers (str) – name of the model layer to generate the feature map.

  • upsampler – an upsampling method to upsample the feature map.

  • postprocessing – a callable that applies on the upsampled feature map.

compute_map(x, class_idx=None, retain_graph=False, layer_idx=- 1)[source]

Compute the actual feature map with input tensor x.

feature_map_size(input_size, device='cpu', layer_idx=- 1)[source]

Computes the actual feature map size given nn_module and the target_layer name.

Parameters
  • input_size – shape of the input tensor

  • device – the device used to initialise the input tensor

  • layer_idx – index of the target layer if there are multiple target layers. Defaults to -1.

Returns

shape of the actual feature map.

class monai.visualize.class_activation_maps.GradCAMpp(nn_module, target_layers, upsampler=<function default_upsampler>, postprocessing=<function default_normalizer>)[source]

Computes Gradient-weighted Class Activation Mapping (Grad-CAM++). This implementation is based on:

Chattopadhyay et al., Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks, https://arxiv.org/abs/1710.11063

Parameters
  • nn_module – the model to be used to generate the visualisations.

  • target_layers (str) – name of the model layer to generate the feature map.

  • upsampler – an upsampling method to upsample the feature map.

  • postprocessing – a callable that applies on the upsampled feature map.

compute_map(x, class_idx=None, retain_graph=False, layer_idx=- 1)[source]

Compute the actual feature map with input tensor x.

class monai.visualize.class_activation_maps.ModelWithHooks(nn_module, target_layer_names, register_forward=False, register_backward=False)[source]

A model wrapper to run model forward/backward steps and storing some intermediate feature/gradient information.

Parameters
  • nn_module – the model to be wrapped.

  • target_layer_names (Union[str, Sequence[str]]) – the names of the layer to cache.

  • register_forward (bool) – whether to cache the forward pass output corresponding to target_layer_names.

  • register_backward (bool) – whether to cache the backward pass output corresponding to target_layer_names.

get_layer(layer_id)[source]
Parameters

layer_id (Union[str, Callable]) – a layer name string or a callable. If it is a callable such as lambda m: m.fc, this method will return the module self.model.fc.

Returns

a submodule from self.model.

monai.visualize.class_activation_maps.default_normalizer(acti_map)[source]

A linear intensity scaling by mapping the (min, max) to (1, 0).

Return type

ndarray

monai.visualize.class_activation_maps.default_upsampler(spatial_size)[source]

A linear interpolation method for upsampling the feature map. The output of this function is a callable func, such that func(activation_map) returns an upsampled tensor.

Return type

Callable[[Tensor], Tensor]