Creating a Simple Image Processing App with MONAI Deploy App SDK¶

This tutorial shows how to develop a simple image processing application can be created with MONAI Deploy App SDK.

Creating Operators and connecting them in Application class¶

We will implement an application that consists of three Operators:

SobelOperator: Apply a Sobel edge detector.
- Input: a file path (DataPath)
- Output: an image object in memory (Image)
MedianOperator: Apply a Median filter for noise reduction.
- Input: an image object in memory (Image)
- Output: an image object in memory (Image)
GaussianOperator: Apply a Gaussian filter for smoothening.
- Input: an image object in memory (Image)
- Output: a file path (DataPath)

The workflow of the application would look like this.

%%{init: {"theme": "base", "themeVariables": { "fontSize": "16px"}} }%% classDiagram direction LR SobelOperator --|> MedianOperator : image...image MedianOperator --|> GaussianOperator : image...image class SobelOperator { <in>image : DISK image(out) IN_MEMORY } class MedianOperator { <in>image : IN_MEMORY image(out) IN_MEMORY } class GaussianOperator { <in>image : IN_MEMORY image(out) DISK }

Setup environment¶

# Install necessary image loading/processing packages for the application
!python -c "import PIL" || pip install -q "Pillow"
!python -c "import skimage" || pip install -q "scikit-image"
!python -c "import matplotlib" || pip install -q matplotlib
%matplotlib inline

# Install MONAI Deploy App SDK package
!python -c "import monai.deploy" || pip install -q "monai-deploy-app-sdk"

Download test input¶

We will use a test input from the following.

Case courtesy of Dr Bruno Di Muzio, Radiopaedia.org. From the case rID: 41113

!python -c "import wget" || pip install -q "wget"

from skimage import io
import wget

test_input_path = "/tmp/normal-brain-mri-4.png"
wget.download("https://user-images.githubusercontent.com/1928522/133383228-2357d62d-316c-46ad-af8a-359b56f25c87.png", test_input_path)

print(f"Test input file path: {test_input_path}")

test_image = io.imread(test_input_path)
io.imshow(test_image)

Test input file path: /tmp/normal-brain-mri-4.png

<matplotlib.image.AxesImage at 0x7ffa9326c130>

Setup imports¶

Let’s import necessary classes/decorators to define Application and Operator.

import monai.deploy.core as md  # 'md' stands for MONAI Deploy (or can use 'core' instead)
from monai.deploy.core import (
    Application,
    DataPath,
    ExecutionContext,
    Image,
    InputContext,
    IOType,
    Operator,
    OutputContext,
)

Creating Operator classes¶

Each Operator class inherits Operator class and input/output properties are specified by using @input/@output decorators.

Note that the first operator(SobelOperator)’s input and the last operator(GaussianOperator)’s output are DataPath type with IOType.DISK. Those paths are mapped into input and output paths given by the user during the execution.

Business logic would be implemented in the compute() method.

SobelOperator¶

SobelOperator is the first operator (A root operator in the workflow graph). op_input.get(label) (since only one input is defined in this operator, we don’t need to specify an input label) would return an object of DataPath and the input file/folder path would be available by accessing the path property (op_input.get().path).

Once an image data (as a Numpy array) is loaded and processed, Image object is created from the image data and set to the output (op_output.set(value, label)).

@md.input("image", DataPath, IOType.DISK)
@md.output("image", Image, IOType.IN_MEMORY)
# If `pip_packages` is specified, the definition will be aggregated with the package dependency list of other
# operators and the application in packaging time.
# @md.env(pip_packages=["scikit-image >= 0.17.2"])
class SobelOperator(Operator):
    """This Operator implements a Sobel edge detector.

    It has a single input and single output.
    """

    def compute(self, op_input: InputContext, op_output: OutputContext, context: ExecutionContext):
        from skimage import filters, io

        input_path = op_input.get().path
        if input_path.is_dir():
            input_path = next(input_path.glob("*.*"))  # take the first file

        data_in = io.imread(input_path)[:, :, :3]  # discard alpha channel if exists
        data_out = filters.sobel(data_in)

        op_output.set(Image(data_out))

MedianOperator¶

MedianOperator is a middle operator that accepts data from SobelOperator and pass the processed image data to GaussianOperator.

Its input and output data types are Image and the Numpy array data is available through asnumpy() method (op_input.get().asnumpy()).

Again, once an image data (as a Numpy array) is loaded and processed, Image object is created and set to the output (op_output.set(value, label)).

@md.input("image", Image, IOType.IN_MEMORY)
@md.output("image", Image, IOType.IN_MEMORY)
# If `pip_packages` is specified, the definition will be aggregated with the package dependency list of other
# operators and the application in packaging time.
# @md.env(pip_packages=["scikit-image >= 0.17.2"])
class MedianOperator(Operator):
    """This Operator implements a noise reduction.

    The algorithm is based on the median operator.
    It ingests a single input and provides a single output.
    """

    def compute(self, op_input: InputContext, op_output: OutputContext, context: ExecutionContext):
        from skimage.filters import median

        data_in = op_input.get().asnumpy()
        data_out = median(data_in)
        op_output.set(Image(data_out))

GaussianOperator¶

GaussianOperator is the last operator (A leaf operator in the workflow graph) and the output path of this operator is mapped to the user-provided output folder so we cannot set a path to op_output variable (e.g., op_output.set(Image(data_out))).

Instead, we can get the output path through op_output.get().path and save the processed image data into a file.

@md.input("image", Image, IOType.IN_MEMORY)
@md.output("image", DataPath, IOType.DISK)
# If `pip_packages` is specified, the definition will be aggregated with the package dependency list of other
# operators and the application in packaging time.
# @md.env(pip_packages=["scikit-image >= 0.17.2"])
class GaussianOperator(Operator):
    """This Operator implements a smoothening based on Gaussian.

    It ingests a single input and provides a single output.
    """

    def compute(self, op_input: InputContext, op_output: OutputContext, context: ExecutionContext):
        from skimage.filters import gaussian
        from skimage.io import imsave

        data_in = op_input.get().asnumpy()
        data_out = gaussian(data_in, sigma=0.2)

        output_folder = op_output.get().path
        output_path = output_folder / "final_output.png"
        imsave(output_path, data_out)

Creating Application class¶

Our application class would look like below.

It defines App class, inheriting Application class.

The requirements (resource and package dependency) for the App can be specified by using @resource and @env decorators.

@md.resource(cpu=1)
# pip_packages can be a string that is a path(str) to requirements.txt file or a list of packages.
@md.env(pip_packages=["scikit-image >= 0.17.2"])
class App(Application):
    """This is a very basic application.

    This showcases the MONAI Deploy application framework.
    """

    # App's name. <class name>('App') if not specified.
    name = "simple_imaging_app"
    # App's description. <class docstring> if not specified.
    description = "This is a very simple application."
    # App's version. <git version tag> or '0.0.0' if not specified.
    version = "0.1.0"

    def compose(self):
        """This application has three operators.

        Each operator has a single input and a single output port.
        Each operator performs some kind of image processing function.
        """
        sobel_op = SobelOperator()
        median_op = MedianOperator()
        gaussian_op = GaussianOperator()

        self.add_flow(sobel_op, median_op)
        # self.add_flow(sobel_op, median_op, {"image": "image"})
        # self.add_flow(sobel_op, median_op, {"image": {"image"}})

        self.add_flow(median_op, gaussian_op)

In compose() method, objects of SobelOperator, MedianOperator, and GaussianOperator classes are created and connected through self.add_flow().

add_flow(source_op, destination_op, io_map=None)

io_map is a dictionary of mapping from the source operator’s label to the destination operator’s label(s) and its type is Dict[str, str|Set[str]].

We can skip specifying io_map if both the number of source_op’s outputs and the number of destination_op’s inputs are one so self.add_flow(sobel_op, median_op) is same with self.add_flow(sobel_op, median_op, {"image": "image"}) or self.add_flow(sobel_op, median_op, {"image": {"image"}}).

Executing app locally¶

We can execute the app in the Jupyter notebook.

app = App()

app.run(input=test_input_path, output="output")

Going to initiate execution of operator SobelOperator
Executing operator SobelOperator (Process ID: 1713457, Operator ID: 926b6b99-14a9-4d31-a1dc-7ebe3ac8704b)
Done performing execution of operator SobelOperator

Going to initiate execution of operator MedianOperator
Executing operator MedianOperator (Process ID: 1713457, Operator ID: 57dca47d-e0d9-472a-be47-34db0ec058eb)
Done performing execution of operator MedianOperator

Going to initiate execution of operator GaussianOperator
Executing operator GaussianOperator (Process ID: 1713457, Operator ID: 8238e133-424c-4456-96d2-059c8454980e)

/home/mqin/src/monai-deploy-app-sdk/.venv/lib/python3.8/site-packages/skimage/_shared/utils.py:348: RuntimeWarning: Images with dimensions (M, N, 3) are interpreted as 2D+RGB by default. Use `multichannel=False` to interpret as 3D image with last dimension of length 3.
  return func(*args, **kwargs)
[2022-10-29 20:11:59,013] [WARNING] (imageio) - Lossy conversion from float64 to uint8. Range [0, 1]. Convert image to uint8 prior to saving to suppress this warning.

Done performing execution of operator GaussianOperator

!ls output

final_output.png  output.json

output_image = io.imread("output/final_output.png")
io.imshow(output_image)

<matplotlib.image.AxesImage at 0x7ffac05d98b0>

Once the application is verified inside Jupyter notebook, we can write the above Python code into Python files in an application folder.

The application folder structure would look like below:

simple_imaging_app
├── __main__.py
├── app.py
├── gaussian_operator.py
├── median_operator.py
└── sobel_operator.py

Note

We can create a single application Python file (such as simple_imaging_app.py) that includes the content of the files, instead of creating multiple files. You will see such example in MedNist Classifier Tutorial.

# Create an application folder
!mkdir -p simple_imaging_app

sobel_operator.py¶

%%writefile simple_imaging_app/sobel_operator.py
import monai.deploy.core as md
from monai.deploy.core import (
    DataPath,
    ExecutionContext,
    Image,
    InputContext,
    IOType,
    Operator,
    OutputContext,
)


@md.input("image", DataPath, IOType.DISK)
@md.output("image", Image, IOType.IN_MEMORY)
class SobelOperator(Operator):
    def compute(self, op_input: InputContext, op_output: OutputContext, context: ExecutionContext):
        from skimage import filters, io

        input_path = op_input.get().path
        if input_path.is_dir():
            input_path = next(input_path.glob("*.*"))  # take the first file

        data_in = io.imread(input_path)[:, :, :3]  # discard alpha channel if exists
        data_out = filters.sobel(data_in)

        op_output.set(Image(data_out))

Overwriting simple_imaging_app/sobel_operator.py

median_operator.py¶

%%writefile simple_imaging_app/median_operator.py
import monai.deploy.core as md
from monai.deploy.core import ExecutionContext, Image, InputContext, IOType, Operator, OutputContext, output


@md.input("image", Image, IOType.IN_MEMORY)
@md.output("image", Image, IOType.IN_MEMORY)
class MedianOperator(Operator):
    def compute(self, op_input: InputContext, op_output: OutputContext, context: ExecutionContext):
        from skimage.filters import median

        data_in = op_input.get().asnumpy()
        data_out = median(data_in)
        op_output.set(Image(data_out))

Overwriting simple_imaging_app/median_operator.py

gaussian_operator.py¶

%%writefile simple_imaging_app/gaussian_operator.py
import monai.deploy.core as md
from monai.deploy.core import (
    DataPath,
    ExecutionContext,
    Image,
    InputContext,
    IOType,
    Operator,
    OutputContext,
)


@md.input("image", Image, IOType.IN_MEMORY)
@md.output("image", DataPath, IOType.DISK)
class GaussianOperator(Operator):
    def compute(self, op_input: InputContext, op_output: OutputContext, context: ExecutionContext):
        from skimage.filters import gaussian
        from skimage.io import imsave

        data_in = op_input.get().asnumpy()
        data_out = gaussian(data_in, sigma=0.2)

        output_folder = op_output.get().path
        output_path = output_folder / "final_output.png"
        imsave(output_path, data_out)

Overwriting simple_imaging_app/gaussian_operator.py

app.py¶

%%writefile simple_imaging_app/app.py
import monai.deploy.core as md
from gaussian_operator import GaussianOperator
from median_operator import MedianOperator
from sobel_operator import SobelOperator

from monai.deploy.core import Application


@md.resource(cpu=1)
@md.env(pip_packages=["scikit-image >= 0.17.2"])
class App(Application):
    def compose(self):
        sobel_op = SobelOperator()
        median_op = MedianOperator()
        gaussian_op = GaussianOperator()

        self.add_flow(sobel_op, median_op)
        self.add_flow(median_op, gaussian_op)

# Run the application when this file is executed.
if __name__ == "__main__":
    App(do_run=True)

Overwriting simple_imaging_app/app.py

if __name__ == "__main__":
    App(do_run=True)

The above lines are needed to execute the application code by using python interpreter.

main.py¶

__main__.py is needed for MONAI Application Packager to detect the main application code (app.py) when the application is executed with the application folder path (e.g., python simple_imaging_app).

%%writefile simple_imaging_app/__main__.py
from app import App

if __name__ == "__main__":
    App(do_run=True)

Overwriting simple_imaging_app/__main__.py

!ls simple_imaging_app

app.py		      __main__.py	  __pycache__
gaussian_operator.py  median_operator.py  sobel_operator.py

In this time, let’s execute the app in the command line.

!python simple_imaging_app -i {test_input_path} -o output

Going to initiate execution of operator SobelOperator
Executing operator SobelOperator (Process ID: 1713625, Operator ID: 0980db0b-4d33-49e9-86aa-98a0f104045f)
Done performing execution of operator SobelOperator

Going to initiate execution of operator MedianOperator
Executing operator MedianOperator (Process ID: 1713625, Operator ID: 8b0d2bcc-752d-4c1e-8eee-d6ce88b205c5)
Done performing execution of operator MedianOperator

Going to initiate execution of operator GaussianOperator
Executing operator GaussianOperator (Process ID: 1713625, Operator ID: f6f19ca2-d4b4-4297-a018-f67615d9ab6a)
/home/mqin/src/monai-deploy-app-sdk/.venv/lib/python3.8/site-packages/skimage/_shared/utils.py:348: RuntimeWarning: Images with dimensions (M, N, 3) are interpreted as 2D+RGB by default. Use `multichannel=False` to interpret as 3D image with last dimension of length 3.
  return func(*args, **kwargs)
[2022-10-29 20:12:05,939] [WARNING] (imageio) - Lossy conversion from float64 to uint8. Range [0, 1]. Convert image to uint8 prior to saving to suppress this warning.
Done performing execution of operator GaussianOperator

Above command is same with the following command line:

!monai-deploy exec simple_imaging_app -i {test_input_path} -o output

Going to initiate execution of operator SobelOperator
Executing operator SobelOperator (Process ID: 1713667, Operator ID: dd931989-0613-4143-8a9a-d8194d990d60)
Done performing execution of operator SobelOperator

Going to initiate execution of operator MedianOperator
Executing operator MedianOperator (Process ID: 1713667, Operator ID: 752602c3-77af-4875-8c45-c62d48c4b215)
Done performing execution of operator MedianOperator

Going to initiate execution of operator GaussianOperator
Executing operator GaussianOperator (Process ID: 1713667, Operator ID: bb8d7535-0293-48e6-895b-b6d292b1f967)
/home/mqin/src/monai-deploy-app-sdk/.venv/lib/python3.8/site-packages/skimage/_shared/utils.py:348: RuntimeWarning: Images with dimensions (M, N, 3) are interpreted as 2D+RGB by default. Use `multichannel=False` to interpret as 3D image with last dimension of length 3.
  return func(*args, **kwargs)
[2022-10-29 20:12:11,084] [WARNING] (imageio) - Lossy conversion from float64 to uint8. Range [0, 1]. Convert image to uint8 prior to saving to suppress this warning.
Done performing execution of operator GaussianOperator

output_image = io.imread("output/final_output.png")
io.imshow(output_image)

<matplotlib.image.AxesImage at 0x7ffabc591c70>

Packaging app¶

Let’s package the app with MONAI Application Packager.

!monai-deploy package simple_imaging_app --tag simple_app:latest  # -l DEBUG

Building MONAI Application Package... -Done
[2022-10-29 20:12:16,845] [INFO] (app_packager) - Successfully built simple_app:latest

Note

Building a MONAI Application Package (Docker image) can take time. Use -l DEBUG option if you want to see the progress.

We can see that the Docker image is created.

!docker image ls | grep simple_app

simple_app                                                            latest                                     ee1cb868d265   About a minute ago   15GB

Executing packaged app locally¶

The packaged app can be run locally through MONAI Application Runner.

# Copy a test input file to 'input' folder
!mkdir -p input && rm -rf input/*
!cp {test_input_path} input/

# Launch the app
!monai-deploy run simple_app:latest input output

Checking dependencies...
--> Verifying if "docker" is installed...

--> Verifying if "simple_app:latest" is available...

Checking for MAP "simple_app:latest" locally
"simple_app:latest" found.

Reading MONAI App Package manifest...
Going to initiate execution of operator SobelOperator
Executing operator SobelOperator (Process ID: 1, Operator ID: 7bf99b18-df43-423b-ab1d-f183106f20e2)
[2022-10-30 03:12:27,405] [INFO] (matplotlib.font_manager) - generated new fontManager
Done performing execution of operator SobelOperator

Going to initiate execution of operator MedianOperator
Executing operator MedianOperator (Process ID: 1, Operator ID: 352120f0-f7b0-4677-8db5-607a6235b7e2)
Done performing execution of operator MedianOperator

Going to initiate execution of operator GaussianOperator
Executing operator GaussianOperator (Process ID: 1, Operator ID: ad03211f-5e3c-4a6e-baec-33a3ea0e1c96)
/root/.local/lib/python3.8/site-packages/skimage/_shared/utils.py:348: RuntimeWarning: Images with dimensions (M, N, 3) are interpreted as 2D+RGB by default. Use `multichannel=False` to interpret as 3D image with last dimension of length 3.
  return func(*args, **kwargs)
[2022-10-30 03:12:28,239] [WARNING] (imageio) - Lossy conversion from float64 to uint8. Range [0, 1]. Convert image to uint8 prior to saving to suppress this warning.
Done performing execution of operator GaussianOperator

output_image = io.imread("output/final_output.png")
io.imshow(output_image)

<matplotlib.image.AxesImage at 0x7ffabc50b820>

Creating a simple image processing app Creating MedNIST Classifier App

Creating a Simple Image Processing App with MONAI Deploy App SDK¶

Creating Operators and connecting them in Application class¶

Setup environment¶

Download test input¶

Setup imports¶

Creating Operator classes¶

SobelOperator¶

MedianOperator¶

GaussianOperator¶

Creating Application class¶

Executing app locally¶

sobel_operator.py¶

median_operator.py¶

gaussian_operator.py¶

app.py¶

__main__.py¶

Packaging app¶

Executing packaged app locally¶

main.py¶