Modules Overview

MONAI Label aims to allow researchers to build labeling applications in a serverless way. This means that MONAI Label applications are always ready-to-deploy via MONAL Label server.

To develop a new MONAI labeling app, developers must inherit the MONAILabelApp interface and implement the methods in the interface that are relevant to their application. Typically a labeling applications will consist of

  • inferencing tasks to allow end-users to invoke select pre-trained models,

  • training tasks used to allow end-users to train a set of models,

  • strategies that allow the users to select the next image to label based on some criteria.

Figure 1 shows the base interfaces that a developer may use to implement their app and the various tasks their app may perform. For example, in the figure the user app MyApp employs

  • two inferencing tasks, namely MyInfer which is a custom implementation of InferTask,
    and InferDeepGrow2D which is a ready-to-use utility included with MONAI Label,

  • one training task, TrainDeepGrow which is an extension of the BasicTrainTask utility,

  • and two next image selection strategies, Random included with MONAL Label which allow
    the user to select the next image at random, and MyStrategy which implements the interface
    Strategy which the end user may select as a custom alternative for next image selection
MONAI Label Interfaces and Utilities

Figure 1: MONAI Label provides interfaces which can be implemented by the label app developer for custom functionality as well as utilities which are readily usable in the labeling app.

Quickstart with Template App

MONAI Label currently provides three template applications which developers may start using out of the box, or with few modifications to achieve the desired behavior. Template applications currently available are

For a quickstart the developer may use

monailabel apps --name <desired_app> --download --output myapp

where desired_app may be any of segmentation, deepgrow, or deepedit.

To better understand template apps, the next few sections we will go into the details of implementing

and putting these to work together in a Developing a MONAI Label App.

Inference Task

Inference tasks must implement the InferTask interface where one must specify a list of pre- and post-transforms and an inferer model. The code snippet below is an example implementation of InferTask where the image is pre-processed to a Numpy array, input into SimpleInferer, and the result is post-processed by applying sigmoid activation with binary discretization.

from monai.inferers import SimpleInferer
from monai.transforms import (LoadImaged, ToNumpyd, Activationsd
                              AsDiscreted, ToNumpyd)

from monailabel.interfaces.tasks import InferTask

class MyInfer(InferTask):

  def pre_transforms(self):
      return [
          LoadImaged(keys="image"),
          ToNumpyd(keys="image"),
      ]

  def inferer(self):
      return SimpleInferer()

  def post_transforms(self):
      return [
          Activationsd(keys="pred", sigmoid=True),
          AsDiscreted(keys="pred", threshold_values=True, logit_thresh=0.5),
          ToNumpyd(keys="pred"),
      ]

Training Task

Training tasks may extend the base class BasicTrainTask which is an abstraction over supervised trainers and evaluators. Here, the developer may override the functionality of the base training class with the desired behavior.

The code block below shows a sample implementation specifying the loss function, training pre- and post-transforms, and validation pre-transforms and inference. There are many more aspects of BasicTrainTask that the developer may choose to override, but in this example they follow the default behavior in the base class.

from monai.inferers import SlidingWindowInferer
from monai.transforms import *

from monailabel.utils.train.basic_train import BasicTrainTask

class MyTrainTask(BasicTrainTask):

  def loss_function(self):
      return DiceLoss(sigmoid=True, squared_pred=True)

  def train_pre_transforms(self):
      return Compose([
          LoadImaged(keys=("image", "label")),
          AsChannelFirstd(keys=("image", "label")),
          SpatialCropForegroundd(keys=("image", "label"), source_key="label", spatial_size=(128, 128, 128)),
          NormalizeIntensityd(keys="image"),
      ])

  def train_post_transforms(self):
      return Compose([
          Activationsd(keys="pred", sigmoid=True),
          AsDiscreted(keys="pred", threshold_values=True, logit_thresh=0.5),
      ])

  def val_pre_transforms(self):
      return Compose([
          LoadImaged(keys=("image", "label")),
          AsChannelFirstd(keys=("image", "label")),
          ScaleIntensityRanged(keys="image", a_min=-57, a_max=164, b_min=0.0, b_max=1.0, clip=True),
          CropForegroundd(keys=("image", "label"), source_key="image"),
          ToTensord(keys=("image", "label")),
      ])

  def val_inferer(self):
      return SlidingWindowInferer(roi_size=(128, 128, 128))

Image Selection Strategy

Selecting the next image to load in the end-users client may be of importance to some labeling applications where the developer may want to allow the user to select one (of perhaps many) strategies to select the next image to annotate as a means to efficiently annotate the datastore by, for example, presenting the most representative image of an unlabeled subset of images.

To select the next image the user will have to implement at least selection Strategy (if not using one available with MONAI Label), which can be optionally supported by a ScoringMethod. The rule of thumb is:

  • if the next image selection strategy is lightweight (e.g. choose an unlabeled image at random) and can be performed as the client waits for MONAI Label server, you do NOT need a scoring method;

  • if the next image selection strategy is computationally bound (e.g. all unlabeled images must be scored) then it is recommended that the user implement a ScoringMethod that would ease the user’s interaction with MONAI Label server.

The example code below shows a simple image selection strategy where SelectImageWithMyScore returns the unlabeled image which scores with the highest my_score it finds in the Datastore based on the score generated by MyScoreGeneratorMethod.

from monailabel.interfaces import Datastore
from monailabel.interfaces.tasks.scoring import ScoringMethod

class MyScoreGeneratorMethod(ScoringMethod):

  def __call__(self, request, datastore: Datastore):
    result = {}

    scoring_model_timestamp = int(os.stat(self.scoring_model_path).st_mtime)
    scoring_model = torch.jit.load(self.scoring_model_path)

    if not scoring_model:
      return None

    scoring_model = scoring_model.to(self.device).eval()

    skipped = 0
    unlabeled_images = datastore.get_unlabeled_images()
    num_samples = request.get("num_samples", self.num_samples)

    for image_id in unlabeled_images:
        image_info = datastore.get_image_info(image_id)
        prev_timestamp = image_info.get("my_score_timestamp", 0)

        # if the timestamps match we dont' need to recompute score
        if prev_timestamp == scoring_model_timestamp:
          skipped += 1
          continue

        with torch.no_grad():
          data = {"image": datastore.get_image_uri(image_id)}
          my_score = scoring_model(data)

        if self.device == "cuda":
          torch.cuda.empty_cache()

        # add `my_score` in datastore to use later in `SelectImageWithMyScore`
        info = {
          "my_score": my_score,
          "my_score_timestamp": scoring_model_timestamp
        }
        datastore.update_image_info(image_id, info)
        result[image_id] = info

    return result

from monailabel.interfaces import Datastore
from monailabel.interfaces.tasks import Strategy

class SelectImageWithMyScore(Strategy):

    def __call__(self, request, datastore: Datastore):
      images = datastore.get_unlabeled_images()
      if not len(images):
          return None

      my_scores = {image: datastore.get_image_info(image).get("my_score", 0) for image in images}

      # default to picking at random if `my_score` is not available
      if sum(my_scores.values()) == 0:
          image = random.choice(images)
          logger.info(f"Randomly selected Image '{image}'")
      else:
          my_max_score, image = max(zip(my_scores.values(), my_scores.keys()))
          logger.info(f"Selected image '{image}' using `my_score` ({my_score})")
      return image

Developing a MONAI Label App

A MONAI Label app ties together inference, training, and image selection to provide the end-user with a seamless simultaneous model training and annotation experience, where a segmentation model learns how to segment the region of interest as the user annotates the data.

The labeling app in the example code below utilizes the tasks MyInfer, MyTrain, SelectImageWithMyScore and MyScoreGeneratorMethod we have defined so far. In the labeling app, the developer overrides

  • init_infers() method to define their own set of inferers

  • init_strategies() to define the next image selection strategies they want to make available to the end users

  • init_scoring_methods() to define the scoring method that assists the selection strategy

  • init_trainers() to define the training tasks that will update the various models required by the labeling app

from monailabel.interfaces import MONAILabelApp

import MyInfer, MyTrain, SelectImageWithMyScore, MyScoreGeneratorMethod

class MyApp(MONAILabelApp):

  def init_infers(self):
    return {
      "segmentation_spleen": MyInfer(self.final_model, load_from_mmar(self.mmar, self.model_dir)),
    }

  def init_strategies(self):
    return {
      "my_score": SelectImageWithMyScore(),
    }

  def init_trainers(self) -> Dict[str, TrainTask]:
    return {
      "segmentation": MyTrainTask(
        self.model_dir, self.network, load_path=self.pretrained_model, publish_path=self.final_model
      )
    }

  def init_scoring_methods(self) -> Dict[str, ScoringMethod]:
    return = {
      "my_scoring_method": MyScoreGeneratorMethod(
        model="/path/to/scoring_model",
      )
    }
Quickstart Application Deployment