2.6. Tools

The following sections describe classes and toolboxes, and contain a lot of useful information for advanced ML programming.

2.6.1. MLLinearAlgebra(Vector2, ..., Vector10, Vector16, Matrix2, , ..., Matrix6, quaternion, ImageVector)

This project contains some basic classes for simple vector and matrix arithmetics as well as for quaternion support.

2.6.2.  MLUtilities

This project contains some basic classes, files and interfaces used by the ML which are not directly related to image processing. Some of them are explicitly explained in other chapters of this document.

  1. DateTime

    Defines the class DateTime for storing and processing date and time values.

  2. mlErrorMacros.h

    This file contains a number of macros for error handling, checking and tracing that should be used in all ML code. These macros are essential for source code quality control and for many other checks as well as for error tracing and reporting, and for exception handling.

  3. mlErrorOutput and mlErrorOutputInfos

    The class ErrorOutput is the main error handling and redirecting class of the ML. It uses mlErrorOutputInfos as an information container for error, debug or trace information that is passed to registered error handling routines. See Section 5.4, “The Class ErrorOutput and Configuring Message Outputs” for more information.

  4. mlFileSystem.h

    Defines a set of C functions for system independent file management (file open, close, etc.). All methods support UTF-8 unicode strings to access files that contain unicode characters in their absolute file names.

    See also Chapter 9, Unicode Support for more information on internationalization and management of files that contain international characters in their file names.

  5. mlLibraryInitMacros.h

    This file defines macros that are used for platform-independent library initializations with correct version checks. Most importantly, the macro ML_INIT_LIBRARY(initMethod) is defined in this file. This macro is used to initialize shared libraries independently of the underlying system (WIN32/Mac OS X/Linux). When the library has been loaded, the given init method is called as soon as possible. The name of the initMethod should be composed like < dllName > + 'Init'. This is necessary since this macro sets the name of the initialized DLL as well. Subsequent runtime types will use this name to register the originating DLL.

    See also Section 2.2.4, “The Runtime Type System” for macros to initialize module classes and runtime types of the ML.

  6. mlMemory.h

    Basic memory management class for the ML.

  7. mlNotify.h

    Class to notify registered instances of ML changes. With this class, registered classes will be notified of changes to ML internals. It is not intended to be used in normal modules, but can be very useful for diagnostics.

  8. mlRuntime.h , mlRuntimeDict.h , mlRuntimeSubClass.h , mlRuntimeType.h

    See Section 2.2.4, “The Runtime Type System” for more information on these classes and files.

  9. mlSystemIncludes.h

    This file includes many important system files, makes correct adaptations for some platforms and disables boring and unproductive warnings. It is designed to be independent of the ML or MLUtilities and does not need to link to any ML or MLUtilities binary. When this file is used, the most important program parts are provided platform-independently and without any warnings. Include this file instead of directly including system files.

  10. mlTypeDefs.h

    Header file that contains the most important ML types, constants and definitions; this file can be included without having to link the ML, MLUtilities or MLLinearAlgebra project (see Section 2.6.1, “MLLinearAlgebra(Vector2, ..., Vector10, Vector16, Matrix2, , ..., Matrix6, quaternion, ImageVector)”) and it also does not use any C++ functionality. Thus much ML stuff can be used without being really dependent on the ML.

  11. mlUnicode.h

    File that contains a set of C functions for converting and managing normal unicode strings.

    See also Chapter 9, Unicode Support for more information on internationalization and management of files that contain international characters in their file names.

  12. mlVersion.h

    Public header file for ML version support (Section A.7, “Version Control”). It provides some support for checking for the correct binary versions of the ML and the ML C-API.

2.6.3. Other Classes

2.6.3.1.  SubImageBoxd

Like SubImageBox , only with Vector6 corners. Manages a rectangular 6D box given by two Vector6. Permits intersections etc. See mlSubImgBoxf in project ML.

2.6.3.2. Other Classes and Types

The ML includes some frequently used types.

A detailed explanation of the following helper classes will be given in later editions of this document. Please refer to later versions of this document or to the header files:

  • Engine

    A class derived from Module and intended to build modules similar to Open Inventor™ engines by the use of ML fields. It does not provide any image input or output, i.e., only operations on fields can be implemented.

  • Plane

    Manages a geometric plane in 3D. Used as an encapsulated data type for a PlaneField.

  • Rotation

    Manages a geometric rotation in 3D. Used as an encapsulated data type for a RotationField.

  • Disc

    Manages a geometric 2D disk of a certain radius that is placed in 3D and that can also be voxelized into any image.

  • Sphere

    Manages a geometric sphere in 3D.

2.6.3.2.1. MLDataType

The MLDataType is an enumerator describing all voxel data types currently available in the ML. This includes built-in data types like MLint8Type, MLuint8Type, MLint16Type, MLuint16Type, MLint32Type, MLuint32Type, MLint64Type, MLuint64Type, MLfloatType and MLdoubleType as well as (pre)registered types like Vector2, Vector3, Vector4, Vector6, Vector8, Vector10, Vector16, Vector32, complexf, etc.

Values of this type are often used to request or to determine image data or voxels of a certain type.

Since the number of MLDataTypes can change during runtime, it is implemented as an integer, and the function MLNumDataTypes() returns the current number of available voxel data types. MLDataTypeNames() returns a pointer to the table of data type names corresponding to the MLDataType values.

[Note]Note
  • An MLDataType value may also be invalid or out of range. This should be taken into consideration when using such values. MLIsValidType(MLDataType) can be used to check a type's validity.

  • The number of data types may grow during runtime. MLNumDataTypes() can be used to retrieve the current number.

A number of other useful functions is available to query information about MLDataTypes:

  1. const char *MLNameFromDataType(MLDataType dt)

    Returns the C string name of data type dt if dt is a valid type. Otherwise "" is returned.

  2. MLDataType MLDataTypeFromName(const char * const name)

    Returns the MLDataType value of the data type with the name name. If name is not valid, -1 is returned.

  3. double MLDataTypeMax(MLDataType dt)

    Returns the maximum value of data type dt; if dt is invalid, 0 is returned.

  4. double MLDataTypeMin(MLDataType dt)

    Returns the minimum value of data type dt if dt is invalid, 0 is returned.

  5. size_t MLSizeOf(MLDataType dt)

    Returns the size of the data type dt in bytes. 0 is returned for invalid types.

  6. int MLIsValidType(MLDataType dt)

    Returns true(=1) if data type dt is valid. Otherwise false(=0) is returned.

  7. int MLIsSigned(MLDataType dt)

    Returns true(=1) if data type dt is signed. Otherwise false(=0) is returned.

  8. int MLIsIntType(MLDataType dt)

    returns true(=1) if data type dt is an integer data type. Otherwise false(=0) is returned.

  9. int MLIsFloatType(MLDataType dt)

    Returns true(=1) if data type dt is a floating point data type. Otherwise false(=0) is returned.

  10. int MLIsScalarType(MLDataType dt)

    Returns true(=1) if data type dt is a scalar (i.e., a built-in) type. Otherwise false(=0) is returned.

There are some more functions for the definition of features and properties related to data types. See the documentation in the file mlDataTypes.h for more information. A modern version to get compile-time information on MLDataType is to use the TypeTraits template class. See the documentation in the file mlTypeTraits.h for more information.

  • int MLRangeOrder (MLDataType dt)

  • int MLHolds (MLDataType dt1, MLDataType dt2)

  • MLDataType MLGetPromotedType (MLDataType d1, MLDataType d2)

  • MLDataType MLGetDataTypeForRange (double *min, double *max, int preferUnsigned)

  • MLDataType MLGetDataTypeForUncorrectedRange (double min, double max, int preferUnsigned)

  • MLDataType MLGetRangeAndPrecisionEquivalent (MLDataType dt)

  • MLDataType MLGetPromotedPrecision (MLDataType dt1, MLDataType dt2)

2.6.4.  MLBase

This project contains a set of classes that are useful when data structures like markers, lists, functions, diagram information, etc. are needed that are related to image processing, although they may not be an integral part of it. (See also project MLBase in the modules library.)

2.6.5.  MLKernel

A small template class library with some modules for managing a matrix of kernel elements, and for filtering or correlating/convoluting images. See Kernel Progamming for detailed information.

2.6.6.  MLTools

Class that contains a set of helper functions for different tasks. See mlTools.h in project MLTools for more information.

2.6.7.  MLDiagnosis

The ML project MLDiagnosis contains some modules that can prove to be helpful for module debugging and for changing the ML configuration at runtime.

  1. BadModule

    This module is designed to commit a large number of errors and to have many bugs. Hence applications, module networks, MeVisLab, etc. can be checked for stability on bad module behavior.

  2. CacheView

    This module shows the current state and load of the Memory Manager cache.

  3. Checksum

    This module calculates a checksum of an ML image at the input. This is a simple way to see whether two images differ. Saving just the checksum is sufficient for a later comparison of images. This is especially useful to see whether a module calculates the same image as some time before without storing the entire image.

  4. Console

    A module that shows all ML outputs in a console window.

  5. CoreControl

    Provides an interface to configure the ML error handling system (e.g., how to handle a fatal error, whether the ML continues or terminates), to enable/disable debugging symbols, to configure the ML caching and multithreading, and to obtain the version of the ML.

  6. ErrorTest

    Provides a simple interface to create messages, errors, and exceptions of user defined types. The exact behavior of the applications, error handlers, networks and the ML can be explicitly tested for any type of error.

  7. FieldTracer

    In module networks, fields of different modules are often connected and it can become quite difficult to see from where field changes are sent. The field tracer allows for the creation of a field change list in a certain period of time and by that, it allows to analyze changes in the network.

  8. MLLogFile

    A module that redirects ML output to a log file. The log file's content can be used for further diagnostic purposes (e.g. after crashes).

  9. ModuleView

    This module shows the currently instantiated modules and offers a view on the module interfaces, their fields, inputs and outputs, even when working with an ML release version that does not contain debug information.

  10. RuntimeDump

    This module allows for an installation of a dump function in the ML core that will be called when a runtime type causes a crash that is to be handled by the ML. The current state of the C++ interface of some runtime types like fields, modules derived from Module etc. will be dumped in the error output for further diagnostic purposes. This is intended especially for error diagnostics in release mode when the debugger cannot be used. This module can also remain in released applications so that log information on crashes that did not occur during application development in debug mode are available.

  11. RuntimeView

    This module shows all currently registered types in the runtime type system as well as the libraries they come from, their parent classes, whether they are abstract or not, etc.

  12. Tester

    This powerful module applies a number of tests to one or more modules. It checks for correct field names, memory leaks, stable behavior on many different input images with different (page) extents, data types, min/max values, etc. Parameter and base fields are tested with a large number of combinations of values. More test images are continually added to the module. Testing time, intensity, etc. can be controlled by parameters.

  13. TestInput

    This module generates test images of different types that can be addressed by indices. Thus a large number of different images that cover most image properties like image extent, page extent, min/max values, data type, etc. is available for testing.

2.6.8.  MLImageFormat

The ML project MLImageFormat contains file format classes for storing, loading and modifying an ML PagedImage or subimages in a file.

It stores all information of an up to 6D ML PagedImage, including extended voxel types, paging information and property extensions. It supports files of more than 4 GB, uses the registered MLDataCompressors classes for page-based compression, checks for pages containing only one voxel value to avoid file accesses and unnecessary compressor calls, and many more features. See Section 2.6.9, “ MLDataCompressors for more information on the MLDataCompressors.

The following classes are available as a programming interface (see mlImageFormatDoc.h and class headers of those classes for details):

  1. MLImageFormat

    Class to manage a stored file for saving, loading or retrieving image information. It is mainly used by the module classes.

  2. MLImageFormatTools

    Collection of independent static file IO classes that are mainly used by MLImageFormat.

  3. MLImageFormatFileCache

    Module class to cache an image in a file comparable to a MemCache module.

  4. MLImageFormatSave

    Module class to save a PagedImage and user tags.

  5. MLImageFormatLoad

    Module class to load a file and some of its information.

  6. MLImageFormatInfo

    Module class to get information about a file.

  7. MLImageFormatTag

    Tag class used in MLImageFormatTagList to store one pair of information items such as a name and an integer or a string.

  8. MLImageFormatTagList

    Class to describe the list of tag information stored in a file.

2.6.9.  MLDataCompressors

The ML project MLDatCompressors contains the base class DataCompressor that allows the implementation of new data compression algorithms. It also contains a factory class DataCompressorFactory that allows for the registration of user-derived classes. By that, any number of new compression classes can be implemented which are automatically detected by classes using compression algorithms, for example MLImageFormat modules (Section 2.6.8, “ MLImageFormat).

  1. DataCompressor

    Abstract base class for ML data compression algorithms. New data compressors can be derived from this class and then be registered in the DataCompressorFactory to become available for all other modules and classes that use data compression.

  2. DataCompressorFactory

    Factory class for ML data compression algorithms. It provides access to all registered data compressors, for example for file formats or memory managers using data compression.

See MLDataCompressorDoc.h and other header files in project MLDataCompressor for details.

2.6.9.1. How to Implement a New DataCompressor

Follow these steps to implement your own compressor (see MLDataCompressorDoc.h and other header files in project MLDataCompressor for details and code fragments):

  1. Be sure to implement everything in the namespace ML_UTILS_NAMESPACE.

  2. Derive your compressor from the DataCompressor class and override the following methods:

    1. virtual const std::string getTypeName() const = 0;

    2. virtual const std::string getVersion() const = 0;

    3. virtual const bool isSupportedVersion(const std::string &ver) const = 0;

    4. virtual MLErrorCode compress(const void *srcMem, size_t srcSize, void *&dstMem, MLint &dstNum) const = 0;

    5. virtual MLErrorCode decompress(const void *srcMem, size_t srcSize, void *&dstMem, MLint64 &resSize) const = 0;

  3. Register your DataCompressor (for example during dll/so registration) with YourDataCompressor::initClass() in the runtime type system of the ML first and then in the DataCompressorFactory.

  4. Be sure that classes that use your data compressor will find it registered in the DataCompressorFactory before they are instantiated.

    In MeVisLab, you can do this by specifying the PreloadDll flag in a .def file for your compressor.

  5. Optionally, you may also want to override the numUsedHints() method and initialize the following members appropriately to specify parameters for your compressor which might be detected and passed by some applications to control compression behavior: The parameters _hintType, _hintName, _rangeMin, and _rangeMax should be set by your derived class, the other parameters should be set to their default values.

  6. It is strongly recommended to implement the virtual methods getVersion(), isSupportedVersions(), getVendor(), getSuffix(), and isLossy() in order to provide additional information about classes using the compressor.

    Especially isLossy() should be implemented to make sure that other classes know that decompressed data need not be identical with compressed data. Otherwise, checksum tests done in those classes will fail.

All classes using DataCompressors via the DataCompressorFactory (the MLImageFormat class, for example) will automatically detect your compression algorithm and offer it as an option.

The following example shows a complete header file implementation of a data compressor that packs 16 bit words by removing bit 12 to 15. It is a potentially lossy compressor, because highest bits are removed. It, however, could be useful for CT data, for example, which do not use those bits, or for cases where other compressors do not reach high compression ratios, because data is too noisy:

Example 2.9. CT Data Compressor Packing 12 of 16 Bits


#ifndef __mlCTPackDataCompressor_H
#define __mlCTPackDataCompressor_H

#include "MLCTPackDataCompressorSystem.h"
#include "mlDataCompressor.h"

ML_UTILS_START_NAMESPACE

//! CTPackDataCompressor example for the ML.
class MLCTPackDATA_COMPRESSOR_EXPORT CTPackDataCompressor : public DataCompressor
{

public:

  //! Constructor (no destructor needed).
  CTPackDataCompressor() : DataCompressor() { }

  //! Returns name of compression scheme, used e.g., "RLE", or "LZW".
  virtual std::string     getTypeName()     const { return "CTPack";    }

  //! Returns the version string, e.g., "1.1.4" or "1.1"; compatibility
  //! check needs to be done in isSupportedVersion().
  virtual std::string     getVersion()      const { return "1.0";       }

  //! Returns true if the passed version ver is supported by the
  //! implemented compressor class and false otherwise.
  virtual bool            isSupportedVersion(const std::string &ver) const
  { return ver == getVersion(); }

  //! Return the name of the vendor providing the compressor code
  //! or algorithm, something like "MeVis", the author or the
  //! company selling the algorithm.
  virtual std::string     getVendor()       const { return "ML Guide";  }

  //! Returns the suffix describing the compression scheme, for
  //! example "rle" or "lzw".
  virtual std::string     getSuffix()       const { return "cpk";       }

  //! Returns true if compression is lossy, false if not, base class
  //! default is false.
  //! We have to enable lossy, because we throw away highest nibble
  //! which could cause check sum errors in file formats if not denoted.
  virtual bool            isLossy()         const { return true;        }

  //! Number of hints used by the derived compressor class (defaults
  //! to 0 in base class).
  virtual MLuint8         numUsedHints()    const { return 0;           }
  //! Compresses a chunk of memory to be decompressed later with decompress().
  //! \param  srcMem  is the pointer of data to be compressed.
  //! \param  srcSize is the size of the data pointed to by srcMem in bytes.
  //! \param  dstMem  the pointer to the compressed data.
  //!                 The compressor will allocate the required memory and
  //!                 overwrites the dstMem pointer which then must be freed
  //!                 by the caller with MLFree() or Memory::freeMemory().
  //! \param  dstNum  returns size of compressed data chunk in bytes or 0 on error.
  //! \return         ML_RESULT_OK on successful compression or an error code
  //!                 describing the error.
  virtual MLErrorCode compress(const  void  *srcMem,
                               size_t        srcSize,
                               void        *&dstMem,
                               MLint        &dstNum) const
  {
    MLErrorCode errCode = ML_NO_MEMORY;
    dstMem = NULL;
    dstNum = 0;

    if (srcMem && (srcSize>0)){

      // Determine size of destination buffer, it requires 4 byte at begin to
      // store original data size and in worst case 1 bit more per CTPack more
      // if no CTPack can be compressed. Add four bytes for rounding securely.

      const size_t packedSize =
        static_cast<size_t>(sizeof(MLint64) + (srcSize * 3) / 4 + 4);

      dstMem = Memory::allocateMemory(packedSize, ML_RETURN_NULL);

      if (dstMem != NULL) {

        // Get byte pointer to output memory and clear data.
        unsigned char *targetData = static_cast<unsigned char *>(dstMem);
        memset(targetData, 0, packedSize);

        // Store size of source data in little endian format at buffer start.
        (static_cast<MLint64*>(dstMem))[0] = static_cast<MLint64>(srcSize);

        if (!MLIsLittleEndian()) {
          MLSwapBytes(targetData, sizeof(MLint64), sizeof(MLint64));
        }

        // Traverse all nibbles/half bytes.
        srcSize *= 2;
        size_t oNibble = 16; // Set start to first nibble after stored startDstSize.
        for (size_t n = 0; n < srcSize; ++n) {

          // Get nibble from source data.
          const unsigned char nib =
            (static_cast<const unsigned char*>(srcMem)[n>>1] >>
             ((n & 1)*4)) & 0xf;

          // Add nibble to output data.
          if ((n & 3) != 3) {
            targetData[oNibble>>1] |= (oNibble & 1 ? (nib << 4) : nib);
            ++oNibble;
          }
        }

        // Calculate number of really used bytes in destination buffer and add 1
        // byte as buffer zone for check of buffer overrun during decompression.
        dstNum = (oNibble >> 1) + (oNibble & 1 ? 1 : 0);

        // Return success.
        errCode = ML_RESULT_OK;
      }
    }
    return errCode;
  }
  //! Decompresses a chunk of memory created with compress().
  //! \param  srcMem  is the pointer to the compressed data to be decompressed.
  //! \param  srcSize is the size of the data pointed to by srcMem in bytes.
  //! \param  dstMem  returns the pointer to the decompressed data; it is
  //!                 overwritten with the pointer to the allocated and
  //!                 uncompressed data which must be freed by the caller
  //!                 with MLFree() or Memory::freeMemory().
  //! \param  resSize returns the size of the decompressed data
  //!                 memory in bytes or -1 on error.
  //! \return         ML_RESULT_OK on successful decompression or
  //!                 an error code describing the error.
  virtual MLErrorCode decompress(const  void  *srcMem,
                                 size_t        srcSize,
                                 void        *&dstMem,
                                 MLint64      &resSize) const
  {
    // Pointer to working and result buffers.
    dstMem              = NULL;
    resSize             = -1;
    MLErrorCode errCode = ML_BAD_POINTER_OR_0;

    // Check uncompressed size for at least the four size bytes at start.
    if (srcSize <= sizeof(MLint64)) {
      errCode = ML_FILE_OR_DATA_STRUCTURE_CORRUPTED;
    } else {

      // Get size of decompression data from start of compressed data.
      MLint64 uncompressedSize = (static_cast<const MLint64*>(srcMem))[0];

      if (!MLIsLittleEndian()) {

        // Swap data to local endian format, the data is always stored
        // in little endian.
        MLSwapBytes(reinterpret_cast<unsigned char*>(&uncompressedSize),
                    sizeof(MLint64),
                    sizeof(MLint64));
      }

      if (uncompressedSize < 0) {

        // Should not happen, data is probably corrupted.
        errCode = ML_FILE_OR_DATA_STRUCTURE_CORRUPTED;
      } else {

       // Size seems to be valid, allocate return buffer.
        dstMem = Memory::allocateMemory(static_cast<size_t>(uncompressedSize),
                                        ML_RETURN_NULL);
        if (!dstMem) {
          errCode = ML_NO_MEMORY;
        } else {

          // Unpack all packed nibbles from source data into cleaned result buffer.
          memset(dstMem, 0, uncompressedSize);
          MLint64 oNibble = 0;
          for (size_t n = 16; n < srcSize*2; ++n) {
            // Get nibble from packed data.
            const unsigned char nib =
              (static_cast<const unsigned char*>(srcMem)[n>>1] >>
               ((n & 1)*4)) & 0xf;

            // After unpacking 3 nibbles add a fourth empty one.
            if ((oNibble & 3) == 3) { ++oNibble; }

            // Add nibble to output, shifted by 4 bits if necessary.
            static_cast<unsigned char*>(dstMem)[oNibble >> 1] |=
             (oNibble & 1) ? (nib << 4) : nib;

            ++oNibble;
          }

          // Return success and number of uncompressed bytes.
          errCode = ML_RESULT_OK;
          resSize = uncompressedSize;
        } // else ML_NO_MEMORY;
      } // else if ((uncompressedSize < 0))
    } // else if (srcSize <= sizeof(MLint64) + 1)

    // Clean up on error.
    if (ML_RESULT_OK != errCode) {
      MLFree(dstMem);
      dstMem  = NULL;
      resSize = -1;
    }
    return errCode;
  }


private:

  //! Implements interface for the runtime type system of the ML.
  ML_CLASS_HEADER(CTPackDataCompressor)
};

ML_UTILS_END_NAMESPACE
#endif // __mlCTPackDataCompressor_H

Do not forget to implement the registration code in the ML runtime type system with the typical ML_CLASS_SOURCE macro in the .cpp file:

ML_CLASS_SOURCE(CTPackDataCompressor, DataCompressor);

Also, the registration of the classes in the runtime type system and in the factory for ML data compressors need to be called before using them for the first time (normally in the initialization code while loading the module code):

CTPackDataCompressor ::initClass();
DataCompressorFactory::registerCompressor(CTPackDataCompressor::getClassTypeId());