ProcessTilesNNUNet

MLModule

genre	ML_Inference
author	Felix Thielke
package	FMEstable/ReleaseMeVis
dll	MLProcessTilesNNUNet
definition	ProcessTilesNNUNet.def
keywords	deep, learning, nnUNet, inference, prediction, apply, execute, RedLeaf

Purpose

Performs overlapping weighted tile-based inference for nnU-Net models analogous to the inference algorithm in the nnU-Net code base.

nnU-Net paper: https://www.nature.com/articles/s41592-020-01008-z

nnU-Net code base: https://github.com/MIC-DKFZ/nnUNet/tree/master/nnunetv2/inference

Usage

Connect a pre-processed image and a TileProcessor providing the trained nnU-Net (e.g. using an ONNXTileProcessor or RemoteTritonTileProcessor). Set the parameters according to the trained nnU-Net and press Update.

Details

Processes the tiles of an image and apply a U-Net trained by nnU-Net for inference on these tiles. The input image should already be preprocessed according to the trained nnU-Net model.

Debugging/Profiling of Individual Batch Processing Requests

Set the environment variable TILEPROCESSING_ENABLE_REQUEST_DEBUGGING to a non “0” value to enable sending a unique request ID string with each individual tile request. This simplifies tracking for debugging and profiling in the inference providers (i.e. tile processors).

Windows

Default Panel

Input Fields

inImage

name: inImage, type: Image, deprecated name: input0

The preprocessed image to be segmented.

inMask

name: inMask, type: Image

An optional input mask. It is used if Use Mask is set. Tiles outside the mask are not processed, so providing a mask can reduce the time needed for inference.

inTileProcessor

name: inTileProcessor, type: TileProcessor/TileProcessorContainer(MLBase)

TileProcessor wrapping the nnU-Net, e.g. ONNXTileProcessor or RemoteTritonTileProcessor.

Output Fields

outImage

name: outImage, type: Image, deprecated name: output0

The segmentation output

Parameter Fields

Field Index

[]: Trigger	Has Valid Output: Bool	Status Message: String
Apply Softmax to Model Outputs: Bool	In Page Extent: IntVector6	Update: Trigger
Batch Size: Integer	On Input Change Behavior: Enum	Use Mask: Bool
Clear: Trigger	Output Page Extent Mode: Enum	Used Output Page Extent: IntVector6
Custom Output: String	Output Selection Mode: Enum	X: Bool
doNotClearOnFailedUpdate: Bool	Patch Size: Vector3	Y: Bool
Enable Debug Logging: Bool	Selected Output: String	Z: Bool
Expected Inputs: String	Status Code: Enum

Visible Fields

Update

name: update, type: Trigger

Initiates update of all output field values.

Clear

name: clear, type: Trigger

Clears all output field values to a clean initial state.

On Input Change Behavior

name: onInputChangeBehavior, type: Enum, default: Clear, deprecated name: shouldAutoUpdate,shouldUpdateAutomatically

Declares how the module should react if a value of an input field changes.

Values:

Title	Name	Deprecated Name
​Update	​Update	​TRUE
​Clear	​Clear	​FALSE

Status Code

name: statusCode, type: Enum, persistent: no

Reflects module’s status (successful or failed computations) as one of some predefined enumeration values.

Values:

Title	Name
​Ok	​Ok
​Invalid input object	​Invalid input object
​Invalid input parameter	​Invalid input parameter
​Internal error	​Internal error

Status Message

name: statusMessage, type: String, persistent: no

Gives additional, detailed information about status code as human-readable message.

Has Valid Output

name: hasValidOutput, type: Bool, persistent: no

Indicates validity of output field values (success of computation).

[]

name: updateDone, type: Trigger, persistent: no

Notifies that an update was performed (Check status interface fields to identify success or failure).

Output Selection Mode

name: inModelOutputSelectionMode, type: Enum, default: Single

Choose how the module decides which of the processor outputs to pick. If it has only one output, just leave “Single”.

Values:

Title	Name	Description
​Single	​Single	​If the processor defines only a single output, it will be selected. Otherwise, an error will be reported.
​Custom	​Custom	​You have to specify the exact name of the output to select in Custom Output

Custom Output

name: inCustomModelOutput, type: String, deprecated name: inRequestedOutput

Name of the model output to pick in Output Selection Mode ‘Custom’

Apply Softmax to Model Outputs

name: inApplySoftmax, type: Bool, default: FALSE

Apply softmax to neural network outputs. This is required, if the neural network does not include a softmax layer by itself.

Batch Size

name: inBatchSize, type: Integer, default: 1

The batch size used for inference, i.e. the number of patches which are processed in parallel by the neural network. Increasing the batch size can increase the inference speed, but needs more memory inside the neural network.

Patch Size

name: inPatchSize, type: Vector3, default: 128 128 128, deprecated name: inTileSize

The size (extent) of the patch for processing. Should be the equal to the patch size the nnU-Net model used during training.

Output Page Extent Mode

name: inPageExtentMode, type: Enum, default: UseInputPageExtent

Selects a mode for choosing the output page extent. Larger page extents mean that more tiles need to be processed per page, which may improve (for larger batch sizes) or reduce (if only few parts of the image are needed) the inference performance.

Values:

Title	Name	Description
​Input Page Extent	​UseInputPageExtent	​Use the same page extent as the input image. Usually a good default.
​Fixed Page Extent	​UseFixedPageExtent	​Use the page extent given by In Page Extent, clamped to the image extent.

In Page Extent

name: inPageExtent, type: IntVector6, default: 128 128 1 1 1 1

The output page extent used if Output Page Extent Mode is UseFixedPageExtent. Will be clamped to the image extent.

X

name: inMirrorX, type: Bool, default: FALSE

Mirror X-axis for test time augmentation.

Y

name: inMirrorY, type: Bool, default: FALSE

Mirror Y-axis for test time augmentation.

Z

name: inMirrorZ, type: Bool, default: FALSE

Mirror Z-axis for test time augmentation.

Use Mask

name: inUseMask, type: Bool, default: FALSE

Apply a mask to the segmentation result.

Enable Debug Logging

name: enableDebugLogging, type: Bool, default: FALSE, deprecated name: debug

If true, timing and debug information will be provided to the console.

Expected Inputs

name: availableInputs, type: String, persistent: no

Comma-separated list of all inputs declared by the current TileProcessor

Selected Output

name: availableOutputs, type: String, persistent: no, deprecated name: outAvailableOutputs

Comma-separated list of all outputs declared by the current TileProcessor

Used Output Page Extent

name: outUsedPageExtent, type: IntVector6, persistent: no

Displays the used output page extent as set by Output Page Extent Mode and In Page Extent after clamping it to the output image extent.

Hidden Fields

doNotClearOnFailedUpdate

name: doNotClearOnFailedUpdate, type: Bool, persistent: no

Prevents automated clear after update failed. This does not affect status fields. It enables the developer to analyze module’s state after failure.