MLImageFormatFileCache

MLModule

genre	MLImageFormat
author	MeVis Medical Solutions AG
package	MeVisLab/Standard
dll	MLImageFormat
definition	MLImageFormat.def
see also	MLImageFormatLoad, MLImageFormatSave, MLImageFormatInfo, MemoryCache, BitImageArithmetic, ImagePropertyConvert, ImageLoad, ImageSave, LocalImage, BitImageLoad, BitImageSave, ImageIteratorStart, ImageIteratorEnd, LocalPath
keywords	save, load, store, read, write, storage, temporary, input, data, full, caching, global, buffer, disc, disk, import, export, io, lzf, lzw, lzma, bzip2, compressor, compression, tag

Purpose

The module MLImageFormatFileCache stores the content of a connected input image in a cache file and passes it to the output. Or, alternatively, it overwrites a specific region in an already existing image with new data with the Update Region button.

This module is essential for the composition, modification, and caching of very large volumes (see Processing Large Volumes in MeVisLab), which do not fit into memory or exceed the 2 GB limit.

Accessing a cached image content and passing it to the output image connector is possible even if the input connection is disconnected or invalidated. Thus MLImageFormatFileCache is comparable to the MemoryCache module, which caches the input image in memory.

MLImageFormatFileCache is especially useful to cache images, which are too large to be stored in memory and therefore cannot be handled by the module MemoryCache.

See also the BitImageArithmetic module, which can be configured to cache a BitImage and the general documentation about the MLImageFormat and MLDataCompressors.

Tips

There are some useful and different modes for which the MLImageFormatFileCache modules can be used.

1. The first mode is to store a large image temporarily in a file instead of storing it in a MemoryCache module to reduce the memory usage of an application. Since output pages and the file system also cache output information, access times are often as fast as if a MemoryCache module is used. Note that in many cases the application even becomes faster and more robust against out-of-memory problems with file caching, because more memory is available for other purposes.

2. The second mode is useful to handle and compose very large images. It is recommended to do this with a specifically defined file name in the Temp File Prefix and enabled Preserve Cache File flag. This approach first creates a file written from the input image, which already must have the final result extent. Often TestPattern is connected and Update stores it in the cache. Since pages with unique values are packed, those files are created and stored fast while the file remains very small if unique voxel values are defined as input image data. In the next step with the Update Region functionality subregions or slices can be written step-by-step into that file. Since everything works page-based, there is nearly no limitation about the file size. Even Terabyte-sized files can be composed on 32-bit systems.

3. This mode is similar to the previous one. It allows opening an existing .mlimage file with the Preserve Cache File flag enabled. This file then can be updated, partially overwritten, or changed with the Update Region functionality. Especially if you want to compose very large files, then consider creating the .mlimage file with the Create button in MLImageFormatSave before opening it in MLImageFormatFileCache. Although this only allows the creation of file contents with unique voxel values, it will be significantly faster for extreme-sized images, because it does not need to calculate any data for the created pages. Afterward, regions in such an empty file can be overwritten step-by-step as described in the previous paragraph.

4. Note that the default compressor - if available - is LZ4. This typically reduces the cached files while normally not reducing storage times. This, however, can lead to file fragmentation in cases when non-empty pages are overwritten and with less compression. This is not necessarily a problem; however, if this use case is heavily used, it might be safer to disable compression.

Windows

Default Panel

Input Fields

input0

name: input0, type: Image

The module has one optional input.

Therefore, an already cached file remains up-to-date when Update Mode is set to Off or Auto Update, even if the connection to the input is removed or the input is invalidated.

Output Fields

output0

name: output0, type: Image

The module has one image output, which is invalid if no data is cached; otherwise, it provides the internally cached image.

Parameter Fields

Field Index

boolHintVal6: Bool	dblHintVisible8: Bool	progress: Float
boolHintVal7: Bool	dblHintVisible9: Bool	Region Offset: IntVector6
boolHintVal8: Bool	Full cache file name: String	Remove: Trigger
boolHintVal9: Bool	hintName0: String	Status: String
boolHintVisible0: Bool	hintName1: String	Stop: Trigger
boolHintVisible1: Bool	hintName2: String	strHintVal6: String
boolHintVisible2: Bool	hintName3: String	strHintVal7: String
boolHintVisible3: Bool	hintName4: String	strHintVal8: String
boolHintVisible4: Bool	hintName5: String	strHintVal9: String
boolHintVisible5: Bool	hintName6: String	strHintVisible0: Bool
boolHintVisible6: Bool	hintName7: String	strHintVisible1: Bool
boolHintVisible7: Bool	hintName8: String	strHintVisible2: Bool
boolHintVisible8: Bool	hintName9: String	strHintVisible3: Bool
boolHintVisible9: Bool	intHintVal6: Integer	strHintVisible4: Bool
Cache Valid: Bool	intHintVal7: Integer	strHintVisible5: Bool
Clear: Trigger	intHintVal8: Integer	strHintVisible6: Bool
Compression: Enum	intHintVal9: Integer	strHintVisible7: Bool
dblHintVal6: Double	intHintVisible0: Bool	strHintVisible8: Bool
dblHintVal7: Double	intHintVisible1: Bool	strHintVisible9: Bool
dblHintVal8: Double	intHintVisible2: Bool	Suppress world position: Bool
dblHintVal9: Double	intHintVisible3: Bool	Tag List: String
dblHintVisible0: Bool	intHintVisible4: Bool	Temp File Prefix: String
dblHintVisible1: Bool	intHintVisible5: Bool	trueFileName: String
dblHintVisible2: Bool	intHintVisible6: Bool	Update: Trigger
dblHintVisible3: Bool	intHintVisible7: Bool	Update Mode: Enum
dblHintVisible4: Bool	intHintVisible8: Bool	Update Region: Trigger
dblHintVisible5: Bool	intHintVisible9: Bool	updateHintControls: Trigger
dblHintVisible6: Bool	isSaving: Bool
dblHintVisible7: Bool	Preserve Cache File: Bool

Visible Fields

Compression

name: compression, type: Enum, default: LZ4

Defines the compression algorithm. If available, it usually defaults to a fast compressor such as LZ4. For possible side effects, see Update Region and Tips section.

See MLDataCompressors for general information about data compressors.

Values:

Title	Name
​None	​None
​MLCTComp2	​MLCTComp2
​Lzf	​LZF
​Zlib	​ZLIB
​Lzma	​LZMA
​BZip2	​BZip2
​Lz4	​LZ4

Remove

name: remove, type: Trigger

When pressed, the cached image file is deleted (even in preserve file mode) and the output image is disabled.

Stop

name: stop, type: Trigger

When pressed, the file storing is terminated and the cached image invalidated.

Status

name: status, type: String, persistent: no

Shows information about the current cache state and/or the recent caching operation.

Tag List

name: tagList, type: String, persistent: no

Shows a list of tags in the file.

Region Offset

name: regionOffset, type: IntVector6, default: 0 0 0 0 0 0

Sets a 6D offset vector that is added to the position where an input region is written on Update Region.

If Suppress world position is disabled, the destination of the updated input region is calculated as follows: The origin of the input region in world coordinates is taken and the corresponding voxel coordinate in the cached image is determined. Then, Region Offset is added and the input region is written at that position.

Enable Suppress world position if position calculation shall be performed in voxel coordinates.

Suppress world position

name: suppressWorldPos, type: Bool, default: FALSE

If checked, world coordinates of an input region are used to determine the destination position in target image if Update Region is used.

Otherwise, the world coordinates of the origin of the input image + Region Offset are used as target position for subimages written into already written files. If this field is on, then only the position given by Region Offset will be used as target position for written subimages.

Cache Valid

name: valid, type: Bool, persistent: no

Shows whether a file is cached.

Preserve Cache File

name: preserveCacheFile, type: Bool, default: FALSE

If checked, an existing cache file is used and modified, but never cleared.

Otherwise, a cache file is automatically cleared/removed.

Clear

name: clear, type: Trigger

When pressed, the cached image file is cleared, or - in preserve file mode - the output image is only disabled.

Update

name: update, type: Trigger

When pressed, in preserve file mode with a valid cache file the modle loads the data from the cache file and passes it to the output.

If the cache file does not exist or if preserved file mode is off, then it stores the input image in the cache file.

Note: To update the content of an existing preserved cache file, use Remove to delete the file and then Update to save new data.

Update Region

name: updateRegion, type: Trigger

When pressed, the cached file is overwritten with data from the input image at its world or voxel location, depending on Suppress world position.

Note:

If non-empty and compressed pages are overwritten with new data that could not be compressed with the same or better ration, this can lead to fragmentation of the cache file. If this is used heavily and file sizes must be limited, then it might be safer to disable compression. This, however, is rarely a serious problem. In nearly all cases, using a fast compressor such as LZ4 is more beneficial than using no compression.

Update Mode

name: updateMode, type: Enum, default: AutoUpdate

Defines how changes of parameters of the the input image influence the currently cached image data and the output image.

Values:

Title	Name	Description
​Off	​Off	​Changes to parameters or input image do not influence the cached image or the output image.
​Auto Update	​AutoUpdate	​Changes to parameters or to the input image update the cache and the input image.
​Auto Clear	​AutoClear	​Changes to parameters or to the input image clear the cached image and invalidate the output image.

Temp File Prefix

name: tmpFileNamePrefix, type: String

Sets a preserved cache file name or prefix for automatic file name while using file buffering.

Full cache file name

name: outFullCacheFilePath, type: String, persistent: no

Shows the expanded path to the currently used cache file if is any. If not, then it is set to an empty string. This path is usually set before enabling the Cache Valid flag and after disabling it.

Hidden Fields

updateHintControls

name: updateHintControls, type: Trigger

hintName0

name: hintName0, type: String, persistent: no

dblHintVisible0

name: dblHintVisible0, type: Bool, persistent: no

intHintVisible0

name: intHintVisible0, type: Bool, persistent: no

strHintVisible0

name: strHintVisible0, type: Bool, persistent: no

boolHintVisible0

name: boolHintVisible0, type: Bool, persistent: no

hintName1

name: hintName1, type: String, persistent: no

dblHintVisible1

name: dblHintVisible1, type: Bool, persistent: no

intHintVisible1

name: intHintVisible1, type: Bool, persistent: no

strHintVisible1

name: strHintVisible1, type: Bool, persistent: no

boolHintVisible1

name: boolHintVisible1, type: Bool, persistent: no

hintName2

name: hintName2, type: String, persistent: no

dblHintVisible2

name: dblHintVisible2, type: Bool, persistent: no

intHintVisible2

name: intHintVisible2, type: Bool, persistent: no

strHintVisible2

name: strHintVisible2, type: Bool, persistent: no

boolHintVisible2

name: boolHintVisible2, type: Bool, persistent: no

hintName3

name: hintName3, type: String, persistent: no

dblHintVisible3

name: dblHintVisible3, type: Bool, persistent: no

intHintVisible3

name: intHintVisible3, type: Bool, persistent: no

strHintVisible3

name: strHintVisible3, type: Bool, persistent: no

boolHintVisible3

name: boolHintVisible3, type: Bool, persistent: no

hintName4

name: hintName4, type: String, persistent: no

dblHintVisible4

name: dblHintVisible4, type: Bool, persistent: no

intHintVisible4

name: intHintVisible4, type: Bool, persistent: no

strHintVisible4

name: strHintVisible4, type: Bool, persistent: no

boolHintVisible4

name: boolHintVisible4, type: Bool, persistent: no

hintName5

name: hintName5, type: String, persistent: no

dblHintVisible5

name: dblHintVisible5, type: Bool, persistent: no

intHintVisible5

name: intHintVisible5, type: Bool, persistent: no

strHintVisible5

name: strHintVisible5, type: Bool, persistent: no

boolHintVisible5

name: boolHintVisible5, type: Bool, persistent: no

hintName6

name: hintName6, type: String, persistent: no

dblHintVal6

name: dblHintVal6, type: Double, default: 0

intHintVal6

name: intHintVal6, type: Integer, default: 0

strHintVal6

name: strHintVal6, type: String

boolHintVal6

name: boolHintVal6, type: Bool, default: FALSE

dblHintVisible6

name: dblHintVisible6, type: Bool, persistent: no

intHintVisible6

name: intHintVisible6, type: Bool, persistent: no

strHintVisible6

name: strHintVisible6, type: Bool, persistent: no

boolHintVisible6

name: boolHintVisible6, type: Bool, persistent: no

hintName7

name: hintName7, type: String, persistent: no

dblHintVal7

name: dblHintVal7, type: Double, default: 0

intHintVal7

name: intHintVal7, type: Integer, default: 0

strHintVal7

name: strHintVal7, type: String

boolHintVal7

name: boolHintVal7, type: Bool, default: FALSE

dblHintVisible7

name: dblHintVisible7, type: Bool, persistent: no

intHintVisible7

name: intHintVisible7, type: Bool, persistent: no

strHintVisible7

name: strHintVisible7, type: Bool, persistent: no

boolHintVisible7

name: boolHintVisible7, type: Bool, persistent: no

hintName8

name: hintName8, type: String, persistent: no

dblHintVal8

name: dblHintVal8, type: Double, default: 0

intHintVal8

name: intHintVal8, type: Integer, default: 0

strHintVal8

name: strHintVal8, type: String

boolHintVal8

name: boolHintVal8, type: Bool, default: FALSE

dblHintVisible8

name: dblHintVisible8, type: Bool, persistent: no

intHintVisible8

name: intHintVisible8, type: Bool, persistent: no

strHintVisible8

name: strHintVisible8, type: Bool, persistent: no

boolHintVisible8

name: boolHintVisible8, type: Bool, persistent: no

hintName9

name: hintName9, type: String, persistent: no

dblHintVal9

name: dblHintVal9, type: Double, default: 0

intHintVal9

name: intHintVal9, type: Integer, default: 0

strHintVal9

name: strHintVal9, type: String

boolHintVal9

name: boolHintVal9, type: Bool, default: FALSE

dblHintVisible9

name: dblHintVisible9, type: Bool, persistent: no

intHintVisible9

name: intHintVisible9, type: Bool, persistent: no

strHintVisible9

name: strHintVisible9, type: Bool, persistent: no

boolHintVisible9

name: boolHintVisible9, type: Bool, persistent: no

trueFileName

name: trueFileName, type: String, persistent: no, deprecated name: autoFileName

progress

name: progress, type: Float, persistent: no

Shows the amount of stored image data in percent.

isSaving

name: isSaving, type: Bool, persistent: no