MLImageFormatFileCache¶
-
MLModule
¶ genre MLImageFormat
author MeVis Medical Solutions AG
package MeVisLab/Standard
dll MLImageFormat
definition MLImageFormat.def see also MLImageFormatLoad
,MLImageFormatSave
,MLImageFormatInfo
,MemoryCache
,BitImageArithmetic
,ImagePropertyConvert
,ImageLoad
,ImageSave
,LocalImage
,BitImageLoad
,BitImageSave
,ImageIteratorStart
,ImageIteratorEnd
,LocalPath
keywords save
,load
,store
,read
,write
,storage
,temporary
,input
,data
,full
,caching
,global
,buffer
,disc
,disk
,import
,export
,io
,lzf
,lzw
,lzma
,bzip2
,compressor
,compression
,tag
Purpose¶
The module MLImageFormatFileCache
stores the content of a connected input image in a cache file and passes it to the output.
Or, alternatively, it overwrites a specific region in an already existing image with new data with the Update Region
button.
This module is essential for the composition, modification, and caching of very large volumes (see Processing Large Volumes in MeVisLab
), which do not fit into memory or exceed the 2 GB limit.
Accessing a cached image content and passing it to the output image connector is possible even if the input connection is disconnected or invalidated. Thus MLImageFormatFileCache
is comparable to the MemoryCache
module, which caches the input image in memory.
MLImageFormatFileCache
is especially useful to cache images, which are too large to be stored in memory and therefore cannot be handled by the module MemoryCache
.
See also the BitImageArithmetic
module, which can be configured to cache a BitImage
and the general documentation about the MLImageFormat
and MLDataCompressors
.
Tips¶
There are some useful and different modes for which the MLImageFormatFileCache
modules can be used.
- The first mode is to store a large image temporarily in a file instead of storing it in a
MemoryCache
module to reduce the memory usage of an application. Since output pages and the file system also cache output information, access times are often as fast as if aMemoryCache
module is used. Note that in many cases the application even becomes faster and more robust against out-of-memory problems with file caching, because more memory is available for other purposes. - The second mode is useful to handle and compose very large images. It is recommended to do this with a specifically defined file name in the
Temp File Prefix
and enabledPreserve Cache File
flag. This approach first creates a file written from the input image, which already must have the final result extent. OftenTestPattern
is connected andUpdate
stores it in the cache. Since pages with unique values are packed, those files are created and stored fast while the file remains very small if unique voxel values are defined as input image data. In the next step with theUpdate Region
functionality subregions or slices can be written step-by-step into that file. Since everything works page-based, there is nearly no limitation about the file size. Even Terabyte-sized files can be composed on 32-bit systems. - This mode is similar to the previous one. It allows opening an existing .mlimage file with the
Preserve Cache File
flag enabled. This file then can be updated, partially overwritten, or changed with theUpdate Region
functionality. Especially if you want to compose very large files, then consider creating the .mlimage file with the Create button inMLImageFormatSave
before opening it inMLImageFormatFileCache
. Although this only allows the creation of file contents with unique voxel values, it will be significantly faster for extreme-sized images, because it does not need to calculate any data for the created pages. Afterward, regions in such an empty file can be overwritten step-by-step as described in the previous paragraph. - Note that the default compressor - if available - is LZ4. This typically reduces the cached files while normally not reducing storage times. This, however, can lead to file fragmentation in cases when non-empty pages are overwritten and with less compression. This is not necessarily a problem; however, if this use case is heavily used, it might be safer to disable compression.
Input Fields¶
input0¶
-
name:
input0
, type:
Image
¶ The module has one optional input.
Therefore, an already cached file remains up-to-date when
Update Mode
is set to Off or Auto Update, even if the connection to the input is removed or the input is invalidated.
Output Fields¶
Parameter Fields¶
Field Index¶
Visible Fields¶
Compression¶
-
name:
compression
, type:
Enum
, default:
LZ4
¶ Defines the compression algorithm. If available, it usually defaults to a fast compressor such as LZ4. For possible side effects, see
Update Region
and Tips section.See
MLDataCompressors
for general information about data compressors.
Values:
Title | Name |
---|---|
None | None |
MLCTComp2 | MLCTComp2 |
Lzf | LZF |
Zlib | ZLIB |
Lzma | LZMA |
BZip2 | BZip2 |
Lz4 | LZ4 |
Remove¶
-
name:
remove
, type:
Trigger
¶ When pressed, the cached image file is deleted (even in preserve file mode) and the output image is disabled.
Stop¶
-
name:
stop
, type:
Trigger
¶ When pressed, the file storing is terminated and the cached image invalidated.
Status¶
-
name:
status
, type:
String
, persistent:
no
¶ Shows information about the current cache state and/or the recent caching operation.
Region Offset¶
-
name:
regionOffset
, type:
IntVector6
, default:
0 0 0 0 0 0
¶ Sets a 6D offset vector that is added to the position where an input region is written on
Update Region
.If
Suppress world position
is disabled, the destination of the updated input region is calculated as follows: The origin of the input region in world coordinates is taken and the corresponding voxel coordinate in the cached image is determined. Then,Region Offset
is added and the input region is written at that position.Enable
Suppress world position
if position calculation shall be performed in voxel coordinates.
Suppress world position¶
-
name:
suppressWorldPos
, type:
Bool
, default:
FALSE
¶ If checked, world coordinates of an input region are used to determine the destination position in target image if
Update Region
is used.Otherwise, the world coordinates of the origin of the input image +
Region Offset
are used as target position for subimages written into already written files. If this field is on, then only the position given byRegion Offset
will be used as target position for written subimages.
Preserve Cache File¶
-
name:
preserveCacheFile
, type:
Bool
, default:
FALSE
¶ If checked, an existing cache file is used and modified, but never cleared.
Otherwise, a cache file is automatically cleared/removed.
Clear¶
-
name:
clear
, type:
Trigger
¶ When pressed, the cached image file is cleared, or - in preserve file mode - the output image is only disabled.
Update¶
-
name:
update
, type:
Trigger
¶ When pressed, in preserve file mode with a valid cache file the modle loads the data from the cache file and passes it to the output.
If the cache file does not exist or if preserved file mode is off, then it stores the input image in the cache file.
Note: To update the content of an existing preserved cache file, use
Remove
to delete the file and thenUpdate
to save new data.
Update Region¶
-
name:
updateRegion
, type:
Trigger
¶ When pressed, the cached file is overwritten with data from the input image at its world or voxel location, depending on
Suppress world position
.Note:
If non-empty and compressed pages are overwritten with new data that could not be compressed with the same or better ration, this can lead to fragmentation of the cache file. If this is used heavily and file sizes must be limited, then it might be safer to disable compression. This, however, is rarely a serious problem. In nearly all cases, using a fast compressor such as LZ4 is more beneficial than using no compression.
Update Mode¶
-
name:
updateMode
, type:
Enum
, default:
AutoUpdate
¶ Defines how changes of parameters of the the input image influence the currently cached image data and the output image.
Values:
Title | Name | Description |
---|---|---|
Off | Off | Changes to parameters or input image do not influence the cached image or the output image. |
Auto Update | AutoUpdate | Changes to parameters or to the input image update the cache and the input image. |
Auto Clear | AutoClear | Changes to parameters or to the input image clear the cached image and invalidate the output image. |
Temp File Prefix¶
-
name:
tmpFileNamePrefix
, type:
String
¶ Sets a preserved cache file name or prefix for automatic file name while using file buffering.
Full cache file name¶
-
name:
outFullCacheFilePath
, type:
String
, persistent:
no
¶ Shows the expanded path to the currently used cache file if is any. If not, then it is set to an empty string. This path is usually set before enabling the
Cache Valid
flag and after disabling it.