Event handlers

Checkpoint loader

class CheckpointLoader(load_path, load_dict)

CheckpointLoader acts as an ignite handler to load checkpoint data from file. It can load variables for network, optimizer, lr_scheduler. And also can restore training if load the state_dict of ignite engine.

Parameters

• load_path (string) – the file path of checkpoint, it should be a PyTorch pth file.

• load_dict (dict) –

  target objects that load checkpoint to. examples:

  {'network': net, 'optimizer': optimizer, 'engine', engine}
  

CSV saver

class ClassificationSaver(output_dir='./', filename='predictions.csv', overwrite=True, batch_transform=<function ClassificationSaver.<lambda>>, output_transform=<function ClassificationSaver.<lambda>>, name=None)

Event handler triggered on completing every iteration to save the classification predictions as CSV file.

Parameters

• output_dir (str) – output CSV file directory.

• filename (str) – name of the saved CSV file name.

• overwrite (bool) – whether to overwriting existing CSV file content. If we are not overwriting, then we check if the results have been previously saved, and load them to the prediction_dict.

• batch_transform (Callable) – a callable that is used to transform the ignite.engine.batch into expected format to extract the meta_data dictionary.

• output_transform (Callable) – a callable that is used to transform the ignite.engine.output into the form expected model prediction data. The first dimension of this transform’s output will be treated as the batch dimension. Each item in the batch will be saved individually.

• name (str) – identifier of logging.logger to use, defaulting to engine.logger.

Mean Dice metrics handler

class MeanDice(include_background=True, to_onehot_y=False, mutually_exclusive=False, add_sigmoid=False, logit_thresh=0.5, output_transform: Callable = <function MeanDice.<lambda>>, device: Union[str, torch.device, None] = None)

Computes dice score metric from full size Tensor and collects average over batch, class-channels, iterations.

Parameters

• include_background (Bool) – whether to include dice computation on the first channel of the predicted output. Defaults to True.

• to_onehot_y (Bool) – whether to convert the output prediction into the one-hot format. Defaults to False.

• mutually_exclusive (Bool) – if True, the output prediction will be converted into a binary matrix using a combination of argmax and to_onehot. Defaults to False.

• add_sigmoid (Bool) – whether to add sigmoid function to the output prediction before computing Dice. Defaults to False.

• logit_thresh (Float) – the threshold value to round value to 0.0 and 1.0. Defaults to None (no thresholding).

• output_transform (Callable) – transform the ignite.engine.state.output into [y_pred, y] pair.

• device (torch.device) – device specification in case of distributed computation usage.

See also

monai.metrics.meandice.compute_meandice()

compute()

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest.

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed.

reset()

Resets the metric to it’s initial state.

This is called at the start of each epoch.

update(output: Sequence[Union[torch.Tensor, dict]])

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function.

ROC AUC metrics handler

class ROCAUC(to_onehot_y=False, add_softmax=False, average='macro', output_transform=<function ROCAUC.<lambda>>)

Computes Area Under the Receiver Operating Characteristic Curve (ROC AUC). accumulating predictions and the ground-truth during an epoch and applying compute_roc_auc.

Parameters

• to_onehot_y (bool) – whether to convert y into the one-hot format. Defaults to False.

• add_softmax (bool) – whether to add softmax function to y_pred before computation. Defaults to False.

• average (macro|weighted|micro|None) –

  type of averaging performed if not binary classification. default is ‘macro’.

  • ’macro’: calculate metrics for each label, and find their unweighted mean. this does not take label imbalance into account.

  • ’weighted’: calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label).

  • ’micro’: calculate metrics globally by considering each element of the label indicator matrix as a label.

  • None: the scores for each class are returned.

• output_transform (callable, optional) – a callable that is used to transform the Engine process_function output into the form expected by the metric. This can be useful if, for example, you have a multi-output model and you want to compute the metric with respect to one of the outputs.

Note

ROCAUC expects y to be comprised of 0’s and 1’s. y_pred must either be probability estimates or confidence values.

compute()

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest.

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed.

reset()

Resets the metric to it’s initial state.

This is called at the start of each epoch.

update(output: Sequence[torch.Tensor])

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function.

Metric logger

Segmentation saver

class SegmentationSaver(output_dir='./', output_postfix='seg', output_ext='.nii.gz', dtype=None, batch_transform=<function SegmentationSaver.<lambda>>, output_transform=<function SegmentationSaver.<lambda>>, name=None)

Event handler triggered on completing every iteration to save the segmentation predictions as nifti files.

Parameters

• output_dir (str) – output image directory.

• output_postfix (str) – a string appended to all output file names.

• output_ext (str) – output file extension name.

• dtype (np.dtype, optional) – convert the image data to save to this data type. If None, keep the original type of data.

• batch_transform (Callable) – a callable that is used to transform the ignite.engine.batch into expected format to extract the meta_data dictionary.

• output_transform (Callable) – a callable that is used to transform the ignite.engine.output into the form expected nifti image data. The first dimension of this transform’s output will be treated as the batch dimension. Each item in the batch will be saved individually.

• name (str) – identifier of logging.logger to use, defaulting to engine.logger.

Training stats handler

class StatsHandler(epoch_print_logger=None, iteration_print_logger=None, output_transform=<function StatsHandler.<lambda>>, global_epoch_transform=<function StatsHandler.<lambda>>, name=None, tag_name='Loss', key_var_format='{}: {:.4f} ')

StatsHandler defines a set of Ignite Event-handlers for all the log printing logics. It’s can be used for any Ignite Engine(trainer, validator and evaluator). And it can support logging for epoch level and iteration level with pre-defined loggers.

Default behaviors:

• When EPOCH_COMPLETED, logs engine.state.metrics using self.logger.

• When ITERATION_COMPLETED, logs self.output_transform(engine.state.output) using self.logger.

Parameters

• epoch_print_logger (Callable) – customized callable printer for epoch level logging. must accept parameter “engine”, use default printer if None.

• iteration_print_logger (Callable) – customized callable printer for iteration level logging. must accept parameter “engine”, use default printer if None.

• output_transform (Callable) – a callable that is used to transform the ignite.engine.output into a scalar to print, or a dictionary of {key: scalar}. in the latter case, the output string will be formatted as key: value. by default this value logging happens when every iteration completed.

• global_epoch_transform (Callable) – a callable that is used to customize global epoch number. For example, in evaluation, the evaluator engine might want to print synced epoch number with the trainer engine.

• name (str) – identifier of logging.logger to use, defaulting to engine.logger.

• tag_name (string) – when iteration output is a scalar, tag_name is used to print tag_name: scalar_value to logger. Defaults to 'Loss'.

• key_var_format (string) – a formatting string to control the output string format of key: value.

attach(engine: ignite.engine.engine.Engine)

Register a set of Ignite Event-Handlers to a specified Ignite engine.

Parameters

engine (ignite.engine) – Ignite Engine, it can be a trainer, validator or evaluator.

epoch_completed(engine: ignite.engine.engine.Engine)

handler for train or validation/evaluation epoch completed Event. Print epoch level log, default values are from ignite state.metrics dict.

Parameters

engine (ignite.engine) – Ignite Engine, it can be a trainer, validator or evaluator.

exception_raised(engine: ignite.engine.engine.Engine, e)

handler for train or validation/evaluation exception raised Event. Print the exception information and traceback.

Parameters

• engine (ignite.engine) – Ignite Engine, it can be a trainer, validator or evaluator.

• e (Exception) – the exception caught in Ignite during engine.run().

iteration_completed(engine: ignite.engine.engine.Engine)

handler for train or validation/evaluation iteration completed Event. Print iteration level log, default values are from ignite state.logs dict.

Parameters

engine (ignite.engine) – Ignite Engine, it can be a trainer, validator or evaluator.

Tensorboard handler

class TensorBoardImageHandler(summary_writer=None, batch_transform=<function TensorBoardImageHandler.<lambda>>, output_transform=<function TensorBoardImageHandler.<lambda>>, global_iter_transform=<function TensorBoardImageHandler.<lambda>>, index=0, max_channels=1, max_frames=64)

TensorBoardImageHandler is an ignite Event handler that can visualise images, labels and outputs as 2D/3D images. 2D output (shape in Batch, channel, H, W) will be shown as simple image using the first element in the batch, for 3D to ND output (shape in Batch, channel, H, W, D) input, each of self.max_channels number of images’ last three dimensions will be shown as animated GIF along the last axis (typically Depth).

It’s can be used for any Ignite Engine (trainer, validator and evaluator). User can easily added it to engine for any expected Event, for example: EPOCH_COMPLETED, ITERATION_COMPLETED. The expected data source is ignite’s engine.state.batch and engine.state.output.

Default behavior:

• Show y_pred as images (GIF for 3D) on TensorBoard when Event triggered,

• need to use batch_transform and output_transform to specify how many images to show and show which channel.

• Expects batch_transform(engine.state.batch) to return data format: (image[N, channel, …], label[N, channel, …]).

• Expects output_transform(engine.state.output) to return a torch tensor in format (y_pred[N, channel, …], loss).

Parameters

• summary_writer (SummaryWriter) – user can specify TensorBoard SummaryWriter, default to create a new writer.

• batch_transform (Callable) – a callable that is used to transform the ignite.engine.batch into expected format to extract several label data.

• output_transform (Callable) – a callable that is used to transform the ignite.engine.output into expected format to extract several output data.

• global_iter_transform (Callable) – a callable that is used to customize global step number for TensorBoard. For example, in evaluation, the evaluator engine needs to know current epoch from trainer.

• index (int) – plot which element in a data batch, default is the first element.

• max_channels (int) – number of channels to plot.

• max_frames (int) – number of frames for 2D-t plot.

class TensorBoardStatsHandler(summary_writer=None, epoch_event_writer=None, iteration_event_writer=None, output_transform=<function TensorBoardStatsHandler.<lambda>>, global_epoch_transform=<function TensorBoardStatsHandler.<lambda>>, tag_name='Loss')

TensorBoardStatsHandler defines a set of Ignite Event-handlers for all the TensorBoard logics. It’s can be used for any Ignite Engine(trainer, validator and evaluator). And it can support both epoch level and iteration level with pre-defined TensorBoard event writer. The expected data source is ignite engine.state.output and engine.state.metrics.

Default behaviors:

• When EPOCH_COMPLETED, write each dictionary item in engine.state.metrics to TensorBoard.

• When ITERATION_COMPLETED, write each dictionary item in self.output_transform(engine.state.output) to TensorBoard.

Parameters

• summary_writer (SummaryWriter) – user can specify TensorBoard SummaryWriter, default to create a new writer.

• epoch_event_writer (Callable) – customized callable TensorBoard writer for epoch level. must accept parameter “engine” and “summary_writer”, use default event writer if None.

• iteration_event_writer (Callable) – customized callable TensorBoard writer for iteration level. must accept parameter “engine” and “summary_writer”, use default event writer if None.

• output_transform (Callable) – a callable that is used to transform the ignite.engine.output into a scalar to plot, or a dictionary of {key: scalar}. in the latter case, the output string will be formatted as key: value. by default this value plotting happens when every iteration completed.

• global_epoch_transform (Callable) – a callable that is used to customize global epoch number. For example, in evaluation, the evaluator engine might want to use trainer engines epoch number when plotting epoch vs metric curves.

• tag_name (string) – when iteration output is a scalar, tag_name is used to plot, defaults to 'Loss'.

attach(engine: ignite.engine.engine.Engine)

Register a set of Ignite Event-Handlers to a specified Ignite engine.

Parameters

engine (ignite.engine) – Ignite Engine, it can be a trainer, validator or evaluator.

epoch_completed(engine: ignite.engine.engine.Engine)

handler for train or validation/evaluation epoch completed Event. Write epoch level events, default values are from ignite state.metrics dict.

Parameters

engine (ignite.engine) – Ignite Engine, it can be a trainer, validator or evaluator.

iteration_completed(engine: ignite.engine.engine.Engine)

handler for train or validation/evaluation iteration completed Event. Write iteration level events, default values are from ignite state.logs dict.

Parameters

engine (ignite.engine) – Ignite Engine, it can be a trainer, validator or evaluator.