ProcessTiles

MLModule

genre	ML_Inference_PatchBased
author	Jan-Martin Kuhnigk
package	FMEstable/ReleaseMeVis
dll	MLApplyTileProcessorPageWise
definition	ApplyTileProcessorPageWise.def
see also	ApplyTileProcessorPageWise, TensorFlowServingTileProcessor, SetTileProcessorProperties, AutoProposeTileProcessorProperties, ApplyTileProcessor
keywords	classific, prediction, inference, machine, deep, learning, deeplearning, patch, batch, RemoteVoxelClassification, TensorFlowVoxelClassification, ApplyTileClassifier

Purpose

Applies a C++ it applies a C++ TileProcessor demand driven and a page-by-page to the input image.

It can also be used for “whole image” processing (see Tips section for more info), but usually, you may want to use ApplyTileProcessor instead for such cases.

If you need more than a single input or output, you need to use the module ApplyTileProcessor instead.

Usage

Connect an image and a processor with tile properties, chose an appropriate Custom Output Tile Size (if necessary) and press update.

Details

Tile Processor Properties

Unlike when using the module ApplyTileProcessor, the object inTileProcessor must be a TileProcessingContainer and must, on top of a TileProcessor, also contain a TileProcessorPropertiesSource object that can be queried for proposed tile processor properties (inputs, outputs, padding, stride, dimension mapping, etc.).

For more information on the TileProcessorProperties (strongly recommended reading, if you are not familiar), see the ‘Details’ section of the ApplyTileProcessorPageWise, or directly check out the TileProcessorProperties Schema Documentation.

The only tile property that can be modified by this module is the output tile size, which can be customized via Use and Custom Output Tile Size to best match the data and TileProcessor used.

In contrast to its more complex derivate, ApplyTileProcessorPageWise, ProcessTiles does not support any property overriding except for outputTileSize. Thus, it requires already the correct tile properties to be present and valid at the input. You can always change incorrect properties with ApplyTileProcessorPageWise or by prepending the module SetTileProcessorProperties to ProcessTiles, but it is recommended to fix the properties that come with the model.

See also SetTileProcessorPropertiesFromJson to see how to apply a complete set of properties from file.

Processing ROI

A region of interest (ROI, cf. Use, Output Restriction ROI [VC]) can be set to restrict processing to a sub-region of the input image, which is mostly useful for speeding up computation. In comparison to just applying the ROI to the input image before feeding it to the module, using this feature has the advantage that in case padding is used, it can make use of the actual input data where available, and only resort to filling (see above) where there is none. In comparison to just applying the ROI to the output image, it has the slight performance advantage that the page/tile borders will be optimally aligned to the required region.

Note that:

• the ROI box is to be given in voxel (index) coordinates relative to the image at the reference input (this is the format modules like BoundingBox or BoundingBoxWithMargin put out)

• the ROI box is ignored for the dimension specified as a channel dimension (in the tile properties externalDimensionForChannel1/2), as the ROI is not for clamping channel data

Sharing a TileProcessor between Multiple Modules

You can use two ProcessTiles or ApplyTileProcessorPageWise modules on the same tile processor if you take care to Clear the first one before calling Update on the second instance. Having two or more user modules doing interleaving processBatch calls on the same TileProcessor is currently not supported, the first one ‘updating’ locks the TileProcessor until it is cleared.

This limitation is due to the side effects of prepareProcessBatch calls and could be avoided if necessary, however with some implementation effort.

Debugging/Profiling of Individual Batch Processing Requests

Set the environment variable TILEPROCESSING_ENABLE_REQUEST_DEBUGGING to a non “0” value to enable sending a unique request ID string with each individual tile request. This simplifies tracking for debugging and profiling in the inference providers (i.e. tile processors).

Tips

Using ProcessTiles for “Full” (i.e. non-tiled) Image Processing

The stride tile property can actually be used to implement a single-tile processing of the entire image, instead of a “classic” localized tile processing. The trick is to ensure that the stride is always identical to the input image extent. You can use the module SetTileProcessorProperties so ensure that the input tile will comprise the entire input image, and the output tile only a single voxel (spanning the entire image region).

Windows

Default Panel

Input Fields

inTileProcessor

name: inTileProcessor, type: TileProcessor/TileProcessorContainer(MLBase), deprecated name: inTileClassifier

Used to provide an input processor (implementing the ml::TileProcessor interface) and proposed tile properties. The input must be of type BaseList (which serves as TileProcessorContainer) containing both the TileProcessor and a TileProperiesSource.

See SetTileProcessorProperties, SetTileProcessorPropertiesFromJson and AutoProposeTileProcessorProperties for more info on how to generate proposed tile properties.

Output Fields

outParameterInfoCpp (hidden)

name: outParameterInfoCpp, type: ParameterInfo(MLBase)

ParameterInfo for the connected processor as a C++ object. Hidden since C++ parameter info support is limited (mostly, there is a python wrapper that allows conversion to python, but you can use outParameterInfo instead).

For accessing this object via scripting, see the Scripting Reference: MLParameterInfoWrapper.

outParameterInfo

name: outParameterInfo, type: MLBase

ParameterInfo for the connected processor as a python object.

Parameter Fields

Field Index

[]: Trigger	inOutputSelectionMode: Enum	Status Code: Enum
Available Outputs: String	On Input Change Behavior: Enum	Status Message: String
Clear: Trigger	outMaxNumModuleInputs: Integer	Update: Trigger
Custom Output Tile Size: IntVector6	outMaxNumModuleOutputs: Integer	Use (inUseProcessingRoi): Bool
doNotClearOnFailedUpdate: Bool	Output Restriction ROI [VC]: String	Use (inUseCustomOutputTileSize): Bool
Expected Inputs: String	outUsedDimensions: String	Used Output Tile Size: IntVector6
Has Valid Output: Bool	Selected Output: String
inCustomOutput: String	Snap to valid: Bool

Visible Fields

Update

name: update, type: Trigger

Initiates update of all output field values.

Clear

name: clear, type: Trigger

Clears all output field values to a clean initial state.

On Input Change Behavior

name: onInputChangeBehavior, type: Enum, default: Clear, deprecated name: shouldAutoUpdate,shouldUpdateAutomatically

Declares how the module should react if a value of an input field changes.

Values:

Title	Name	Deprecated Name
​Update	​Update	​TRUE
​Clear	​Clear	​FALSE

Status Code

name: statusCode, type: Enum, persistent: no

Reflects module’s status (successful or failed computations) as one of some predefined enumeration values.

Values:

Title	Name
​Ok	​Ok
​Invalid input object	​Invalid input object
​Invalid input parameter	​Invalid input parameter
​Internal error	​Internal error

Status Message

name: statusMessage, type: String, persistent: no

Gives additional, detailed information about status code as human-readable message.

Has Valid Output

name: hasValidOutput, type: Bool, persistent: no

Indicates validity of output field values (success of computation).

[]

name: updateDone, type: Trigger, persistent: no

Notifies that an update was performed (Check status interface fields to identify success or failure).

Expected Inputs

name: availableInputs, type: String, persistent: no

Comma-separated list of all inputs declared by the current TileProcessorAll inputs declared by the current TileProcessor

Available Outputs

name: availableOutputs, type: String, persistent: no

Comma-separated list of all outputs declared by the current TileProcessor

Use (inUseProcessingRoi)

name: inUseProcessingRoi, type: Bool, default: FALSE

If enabled, you can restrict processing (and hence the output image) to a rectangular Region of Interest defined by Output Restriction ROI [VC].

Output Restriction ROI [VC]

name: inProcessingRoi_voxel, type: String, default: 0 0 0 0 0 0 -1 -1 -1 -1 -1 -1

If Use is enabled, this box defines the processing (and hence output image) region in voxel coordinates.

Snap to valid

name: inEnableOutputTileSizeSnapping, type: Bool, default: TRUE

If enabled (default) the module snaps the output tile size to a valid value greater or equal to the proposed one, if this is supported by the processor. Disabling the flag is only for those rare cases in which the processor errs on what he supports. Of course, it is better to fix the proposed properties than to disable this flag.

See modules SetTileProcessorProperties and SetTileProcessorPropertiesFromJson on how to adapt the proposed tile properties that snapping will always work correctly for you processor.

Use (inUseCustomOutputTileSize)

name: inUseCustomOutputTileSize, type: Bool, default: FALSE, deprecated name: inUseCustomTileSize

If enabled, Custom Output Tile Size is used (instead of the proposal from the connected processor).

Custom Output Tile Size

name: inCustomOutputTileSize, type: IntVector6, default: 128 128 1 1 1 1, deprecated name: inCustomTileSize

If Use is enabled, this overrides the processor’s proposal for the output tile size used for processing. If you have a multi-channel result, this must include the output channel extent in the CHANNEL dimension (indicated by the “Ch” label in the vector edit control). Any ‘0’ entry will be automatically mapped to the corresponding input image extent.

The “0 to full extent mapping” makes it also easier to use the module for whole image classification, but it is still recommended to use ApplyTileProcessor for that purpose.

See ApplyTileProcessorPageWise Details section for details.

Used Output Tile Size

name: outUsedOutputTileSize, type: IntVector6, persistent: no, deprecated name: outUsedTileSize

Actually used output tile size.

See ApplyTileProcessorPageWise Details section for details.

Selected Output

name: outUsedOutput, type: String, persistent: no, deprecated name: outUsedModelOutput

Name of the actually used model output.

Hidden Fields

doNotClearOnFailedUpdate

name: doNotClearOnFailedUpdate, type: Bool, persistent: no

Prevents automated clear after update failed. This does not affect status fields. It enables the developer to analyze module’s state after failure.

outMaxNumModuleInputs

name: outMaxNumModuleInputs, type: Integer, persistent: no

Maximum number of supported module inputs. The module will not be able to process more inputs than this at the same time.

Contact the author/maintainer if you need more.

outMaxNumModuleOutputs

name: outMaxNumModuleOutputs, type: Integer, persistent: no

Maximum number of supported module outputs. The module will not be able to process more outputs than this at the same time.

Contact the author/maintainer if you need more.

outUsedDimensions

name: outUsedDimensions, type: String, persistent: no, deprecated name: outExternalToProcessorDimensionMapping

Used dimension mapping.

inOutputSelectionMode

name: inOutputSelectionMode, type: Enum, default: Single, deprecated name: inModelOutputSelectionMode

Choose how the module decides which of the processor outputs to pick. If it has only one output, just leave “Single”.

Values:

Title	Name	Description
​Single	​Single	​If the processor defines only a single output, it will be selected. Otherwise, an error will be reported.
​Custom	​Custom	​You have to specify the exact name of the output to select in inCustomOutput

inCustomOutput

name: inCustomOutput, type: String, deprecated name: inCustomModelOutput,inRequestedOutputs

Name of the model output to pick in inOutputSelectionMode ‘Custom’