7.2. Implementing Image Processing on extended Voxel Data Types

7.2. Implementing Image Processing on extended Voxel Data Types
	Chapter 7. Registered Voxel Data Types

This section gives detailed information on programming with extended voxel types.

This includes

configuring your module to work fine with extended voxel types,
handling compile and runtime decisions between scalar and extended voxel types and their properties,
getting and managing metadata on extended voxel types,
working with templates on extended voxel types outside the template function calculateOutputSubImage,
handling generalized registered voxel types and module parameters with DataTypeField and UniversalTypeField, and
advanced configuration and programming issues.

See Section 7.2.3, “Examples with Registered Voxel Types” for examples.

7.2.1. Important Functions For Voxel Types

The ML provides many helpful functions that support managing different voxel types and using them for programming (see Section 7.5.2, “Getting and Managing Metadata About Registered Voxel Types” for a detailed discussion).

The most important functions are:

size_t MLSizeOf(MLDataType dt)
returns the size of the data type dt in bytes. On invalid types 0 is returned.
MLDataType MLGetDataTypeFromName("data_type_name")
determines the data type id of the type to be handled, because it is not available as a precompiled constant.
bool MLIsValidDataType(MLDataType dt)
checks whether the data type is registered.
bool MLIsStandardType(MLDataType dt)
checks whether the data type dt is a normal built-in compiler type.
MLTypeInfos* MLGetTypeInfosForDataType(MLDataType outDType)
returns a pointer to the MLTypeInfos object, which describes features and properties of the data type, or returns NULL if outDType is an undefined data type.

The following methods of the class PagedImage are normally used in the calculateOutputImageProperties method of self-developed ML module classes when data types are not appropriate for the implemented algorithm:

PagedImage::setInvalid()
invalidates the module output if the module cannot operate, because e.g., the type does not exist or the data type is not appropriate for the algorithm.
PagedImage::setStateInfo(<message>, ML_TYPE_NOT_REGISTERED)
specifies the reason in <message> why the output image has been invalidated. A connected Info module, for example, will show the reason in its state information.

See Section 7.5.2, “Getting and Managing Metadata About Registered Voxel Types” for information on further functions.

7.2.2. The Basic Concept of Calculating the Output SubImage

After the output properties were evaluated in calculateOutputImageProperties the output image will be requested by calling the derived function calculateOutputSubImage(SubImage *outSubImg, int outIndex, SubImage *inSubImgs) of the module. This function was generated in MeVisLab before versions 3.6 by a set of preprocessor macros, e.g. ML_CALCULATEOUTPUTSUBIMAGE_NUM_INPUTS_1_SCALAR_TYPES_CPP. Those macros ensured that for all possible input and output voxel types the function temmplate calculateOutputSubImage gets instantiated and called during runtime. These macros and not necessary anymore with the possibilities of C++17 but are kept for backward compatibility. The file mlTSubImageVariant.h contains a set of functions. The description of each function contains an example of its usage. These functions can create from a SubImage a variant kind type that can be passed to std::visit and that does the necessary dispatching to the different instantiations of calculateOutputSubImage

7.2.3. Examples with Registered Voxel Types

The following examples contain many useful code fragments for handling and using registered voxel types. For advanced examples see

Section 7.2.5, “Handling Generalized Registered Voxel Types as Module Parameters” to implement general module fields that handle parameters for any registered or any standard voxel type.
Section 7.5.3, “Reducing Generated Code and Compile Times” to have module implementations compile only standard, only registered or only self-defined voxel types and by that to reduce both compile time and size of the generated code.
Section 7.4, “Traps and Pitfalls When Using Registered Voxel Types” to see the difference between correct and incorrect pointer incrementation when traversing and accessing registered voxels in a templated calculateOutputSubImage function.
Section 7.4, “Traps and Pitfalls When Using Registered Voxel Types” to implement and register your own voxel type.

Example 7.1. How to Check and Set a Registered Type Safely as the Output Voxel Type in calculateOutputImageProperties

  MLDataType dt = MLDataTypeFromName("vecf3");
  if (!MLIsValidType(dt)){
    outImage->setInvalid();
    outImage->setStateInfo("Could not find type 'vecf3'", ML_TYPE_NOT_REGISTERED);
    return;
  }
  outImage->setDataType(dt);

This example shows how to select a specific voxel type for the output image. Note that a registered voxel type is used whose id is unknown at compilation time. That is why the voxel type id is determined by using the function MLDataTypeFromName.

Example 7.2. How to Write calculateOutputSubImage without Macros

void SetVoxelValue::calculateOutputSubImage(SubImage *outSubImg, int outIndex, SubImage *inSubImgs)
{
  auto imagePair =
      createTSubImageVariantPair<MLuint8, MLint8, MLuint16, MLint16, MLuint32, MLint32, MLuint64,
                                 MLint64, std::complex<MLfloat>, std::complex<double>, Vector2f,
                                 Vector2d, Vector3f, Vector3d, Vector6f, Vector6d, Matrix2f,
                                 Matrix2d, Matrix3f, Matrix3d>(*outSubImg, inSubImgs);

  auto visitor = [this, outIndex](auto& ip){ calculateOutputSubImage(ip.output, outIndex, ip.input); };

  std::visit(visitor, imagePair);
}

The template parameters of the function createTSubImageVariantPair specifies all possible voxel types that this module can support.

template <class DATATYPE>
void SetVoxelValue::calculateOutputSubImage(TSubImage<DATATYPE>& outImg, int /*outIdx*/, const TSubImage<DATATYPE>&  /*inImg*/) const
{        
  if (outImg.getBox().contains(_inputVoxelPos))
  {
    outImg.setImageValue(_inputVoxelPos, *(reinterpret_cast<DATATYPE*>(_writeValueFld->getUniversalTypeValue())));
  }
}

Example 7.3. How to Write calculateOutputSubImage for Different Input and Ouput Voxel Types Without Macros

void DifferentTypesInputOutputExample::calculateOutputSubImage(SubImage *outSubImage, int outIndex,
                                                             SubImage *inSubImage)
{
  auto input =
      createTSubImageVariant<MLuint8, MLint8, MLuint16, MLint16, MLuint32, MLint32, MLuint64, MLint64,
                       MLfloat, MLdouble>(inSubImage);
  auto output =
      createTSubImageVariant<MLuint8, MLint8, MLuint16, MLint16, MLuint32, MLint32, MLuint64, MLint64,
                       MLfloat, MLdouble>(outSubImage);

  auto visitor = [this, outIndex](auto& out, const auto& in){ calculateOutputSubImage(out, outIndex, in); };

  std::visit(visitor, output, input);
}

The C++ function std::visit creates the cross product of all possible input- and output types.

template <typename T, typename U>
void DifferentTypesInputOutputExample::calculateOutputSubImage(TSubImage<T>& outputSubImage,
                                                             int outputIndex,
                                                             const TSubImage<U>& inputSubImage)
{
  const T constantValue = static_cast<T>(_constantValueFld->getDoubleValue());

  // Clamp box of output image against image extent to avoid that unused areas are processed.
  const SubImageBox validOutBox = outputSubImage.getValidRegion();

  // Process all voxels of the valid region of the output page.
  ImageVector p;
  for (p.u = validOutBox.v1.u; p.u <= validOutBox.v2.u; ++p.u)
  {
    for (p.t = validOutBox.v1.t; p.t <= validOutBox.v2.t; ++p.t)
    {
      for (p.c = validOutBox.v1.c; p.c <= validOutBox.v2.c; ++p.c)
      {
        for (p.z = validOutBox.v1.z; p.z <= validOutBox.v2.z; ++p.z)
        {
          for (p.y = validOutBox.v1.y; p.y <= validOutBox.v2.y; ++p.y)
          {

            p.x = validOutBox.v1.x;
            // Get pointers to row starts of input and output sub-images.
            const U *inVoxel0 = inputSubImage.getImagePointer(p);

            T *outVoxel = outputSubImage.getImagePointer(p);

            const MLint rowEnd = validOutBox.v2.x;

            // Process all row voxels.
            for (; p.x <= rowEnd; ++p.x, ++outVoxel, ++inVoxel0)
            {
              *outVoxel = *inVoxel0 + constantValue;
            }
          }
        }
      }
    }
  }
}

Example 7.4. How to Accept Non-Standard Input Voxels Only

  MLDataType dt = getInputImage(0)->getDataType();
  if (MLIsValidType(dt) && !MLIsStandardType(dt)){
    outImage->setDataType(dt);
  }
  else{
    // Invalidate output image if we have an invalid or a standard voxel data type.
    outImage->setInvalid();
    outImage->setStateInfo("Bad input voxel type", ML_BAD_PARAMETER);
  }

This is a similar example which demonstrates how to configure an ML module to accept only registered voxel types in the input image. (The "!" before MLIsStandardType() can be removed in order to have the ML module accept only standard types).

Example 7.5. How to Implement a Flip of a Vector3f in calculateOutputSubImage

template <typename DTYPE>
  void Vecf3Flip::calculateOutputSubImage(TSubImage<DTYPE> *outSubImg,
                                  int outIndex,
                                  TSubImage<DTYPE> *inSubImg1)
{
  // NOTE: In this example we assume that we have set to operate only on Vector3f voxels.

  // Clamp our page region to the image extent to avoid processing of regions outside the image.
  SubImageBox outBox = outSubImg->getValidRegion();

  // Iterate over all voxels of the valid area of the output subimage.
  ImageVector p;
  for (p.u=outBox.v1.u; p.u<=outBox.v2.u; ++p.u) {
    for (p.t=outBox.v1.t; p.t<=outBox.v2.t; ++p.t) {
      for (p.c=outBox.v1.c; p.c<=outBox.v2.c; ++p.c) {
        for (p.z=outBox.v1.z; p.z<=outBox.v2.z; ++p.z) {
          for (p.y=outBox.v1.y; p.y<=outBox.v2.y; ++p.y) {

            // Get start position of voxel rows in input and in output image.
            p.x = outBox.v1.x;
            DTYPE *iVoxel = inSubImg1->getImagePointer(p);
            DTYPE *oVoxel = outSubImg->getImagePointer(p);

            // Flip all voxels in the row.
            // Warning: Do not iterate with vef3 pointers, because they might have
            // smaller size than DTYPE.
            for (;    p.x <= outBox.v2.x;    ++p.x, ++iVoxel, ++oVoxel ) {

              // Flip Vector3f components from input to output.
              ( *reinterpret_cast<Vector3f*>(oVoxel) )[0] = ( *reinterpret_cast<Vector3f*>(iVoxel) )[2];
              ( *reinterpret_cast<Vector3f*>(oVoxel) )[1] = ( *reinterpret_cast<Vector3f*>(iVoxel) )[1];
              ( *reinterpret_cast<Vector3f*>(oVoxel) )[2] = ( *reinterpret_cast<Vector3f*>(iVoxel) )[0];
            }
          }
        }
      }
    }
  }
}

This example shows a possible way of how to implement the template function calculateOutputSubImage to flip the three components of a Vector3f.

Example 7.6. How to Request a Specific Voxel Type

void ExampleModule::calculateOutputImageProperties(int outIndex, PagedImage* outImg)
{
  // Force the input voxel data type to be of Vector2f; set it for image at index 0
  // because we have only one input image.

  outImg->setInputSubImageDataType(0, MLDataTypeFromName("vecf2"));

}

This example demonstrates how to implement calculateOutputImageProperties to request a specific voxel type for the input subimage. If the input type does not match the requested type, the ML will automatically cast the voxels. Normally, this is done component-wise for registered voxel types. Be aware of the following:

The algorithm must use the ML_CALCULATE_OUTPUTSUBIMAGE_NUM_INPUTS_*_DIFFERENT_INPUT_DATATYPES macros to be able to handle different types of input and output subimages.
The template function calculateOutputSubImage must use two template parameters to distinguish the two types. See documentation of the ML_CALCULATE_OUTPUTSUBIMAGE_NUM_INPUTS_*_DIFFERENT_INOUT_DATATYPES macros.
Some compilers have problems with the large amount of generated code. See Traps And Pitfalls When Using Registered Voxel Types for solutions.
Implicit casts between registered voxel type are relatively slow, because they are done component-wise.

Example 7.7. How to Convert a vecf2 to a vecf3

// Use a macro to call the template function with different input and output template arguments.
ML_CALCULATEOUTPUTSUBIMAGE_NUM_INPUTS_1_DIFFERENT_DEFAULT_INOUT_DATATYPES_CPP(Vecf2ToVecf3Converter);

template <typename OTYPE, typename ITYPE>
  void Vecf2ToVecf3Converter::calculateOutputSubImage(TSubImage<DTYPE> *outSubImg,
                                              int outIndex,
                                              TSubImage<DTYPE> *inSubImg1)
{
  // NOTE: In this example we assume that we have set Vector2f as input and Vector3f as output type.

  // Clamp our page region to the image extent to avoid processing of regions outside the image.
  SubImageBox outBox = outSubImg->getValidRegion();

  // Iterate over all voxels of the valid area of the output subimage.
  ImageVector p;
  for (p.u=outBox.v1.u; p.u<=outBox.v2.u; ++p.u) {
    for (p.t=outBox.v1.t; p.t<=outBox.v2.t; ++p.t) {
      for (p.c=outBox.v1.c; p.c<=outBox.v2.c; ++p.c) {
        for (p.z=outBox.v1.z; p.z<=outBox.v2.z; ++p.z) {
          for (p.y=outBox.v1.y; p.y<=outBox.v2.y; ++p.y) {

            // Get start position of voxel rows in input and in output image.
            p.x = outBox.v1.x;
            ITYPE *iVoxel = inSubImg1->getImagePointer(p);
            OTYPE *oVoxel = outSubImg->getImagePointer(p);
            for (;    p.x <= outBox.v2.x;    ++p.x, ++iVoxel, ++oVoxel ) {

              ( *reinterpret_cast<Vector3f*>(oVoxel) )[0] = ( *reinterpret_cast<Vector2f*>(iVoxel) )[0];
              ( *reinterpret_cast<Vector3f*>(oVoxel) )[1] = ( *reinterpret_cast<Vector2f*>(iVoxel) )[1];
              ( *reinterpret_cast<Vector3f*>(oVoxel) )[2] = ( *reinterpret_cast<Vector2f*>(iVoxel) )[0] *
                                                         ( *reinterpret_cast<Vector2f*>(iVoxel) )[1];
            }
          }
        }
      }
    }
  }
}

This example shows how to implement calculateOutputSubImage to convert a vec2f to a vec3f by writing the product of the first two vector components into the third one. Note that this example still compiles all possible combinations of input and output voxel types, although only one specific combination is used. This version might be useful when other algorithm parts still use other type combinations, otherwise the following version is recommended.

Example 7.8. How to Convert a Vector2f to a Vector3f Without Template Code

void Vecf2ToVecf3Converter::calculateOutputSubImage(SubImage *outSubImg, int outIndex, SubImage *inSubImgs)
{
  // NOTE: In this example we assume that we have set Vector2f as input and Vector3f as output type.

  // Clamp our page region to the image extent to avoid processing of regions outside the image.
  SubImageBox outBox = outSubImg->getValidRegion();

  // Get the sizes of the input and output voxels.
  const size_t iVoxSize = MLSizeOf(inSubImgs->getDataType());
  const size_t oVoxSize = MLSizeOf(outSubImg->getDataType());

  // Iterate over all voxels of the valid area of the output subimage.
  ImageVector p;
  for (p.u=outBox.v1.u; p.u<=outBox.v2.u; ++p.u) {
    for (p.t=outBox.v1.t; p.t<=outBox.v2.t; ++p.t) {
      for (p.c=outBox.v1.c; p.c<=outBox.v2.c; ++p.c) {
        for (p.z=outBox.v1.z; p.z<=outBox.v2.z; ++p.z) {
          for (p.y=outBox.v1.y; p.y<=outBox.v2.y; ++p.y) {

            // Get start position of voxel rows in input and in output image.
            p.x = outBox.v1.x;
            MLTypeData *iVoxel = static_cast<MLTypeData*>(inSubImgs->getImagePointer(p));
            MLTypeData *oVoxel = static_cast<MLTypeData*>(outSubImg->getImagePointer(p));
            for (;  p.x <= outBox.v2.x;  ++p.x) {

              ( *reinterpret_cast<Vector3f*>(oVoxel) )[0] = ( *reinterpret_cast<Vector2f*>(iVoxel) )[0];
              ( *reinterpret_cast<Vector3f*>(oVoxel) )[1] = ( *reinterpret_cast<Vector2f*>(iVoxel) )[1];
              ( *reinterpret_cast<Vector3f*>(oVoxel) )[2] = ( *reinterpret_cast<Vector2f*>(iVoxel) )[0] *
                                                         ( *reinterpret_cast<Vector2f*>(iVoxel) )[1];
              iVoxel += iVoxSize;
              oVoxel += oVoxSize;
            }
          }
        }
      }
    }
  }
}

This example explicitly implements the virtual method calculateOutputSubImage without using any ML_CALCULATE_OUTPUTSUBIMAGE macro. Note that we do not have explicit voxel types anymore. We must use the untyped (void) versions to get voxel positions to the raw data and the sizes of the voxels to move pointers correctly. However, the amount of generated code is considerably smaller, and the compile times are faster.

7.2.4. Compile and Runtime Decisions on Standard and Registered Voxel Types

In order to optimize an algorithm, either with regard to performance or with regard to precision, it is sometimes useful to distinguish between data types or between data type properties. A typical example is: the programmer would like to know whether the template type is an integer, a floating point, a registered, a signed or an unsigned type.

The ML provides a number of functions that return flags depending only on the pointer type; the pointer value is ignored:

MLIsStandardTypePtr (const T* ptr),
MLIsSignedTypePtr (const T* ptr),
MLIs8_16_Or_32BitIntegerTypePtr (const T* ptr),
MLIs8BitIntegerTypePtr (const T* ptr),
MLIs16BitIntegerTypePtr (const T* ptr),
MLIs32BitIntegerTypePtr (const T* ptr),
MLIs64BitIntegerTypePtr (const T* ptr),
MLIsBuiltInIntegerTypePtr (const T* ptr),
MLIsBuiltInFloatingPointTypePtr (const T* ptr),

The following functions return other values such as data type enumerators and sizes, or they activate function tables for registered types:

MLGetDataTypeFromPtr (const T* ptr),
MLGetDataTypeSizeFromPtr (const T* ptr),

	Note
	The above functions are traits, i.e., they are constant at compile time and can be "optimized away" by compilers. Hence, these functions can even be used in time-critical code.

7.2.5. Handling Generalized Registered Voxel Types as Module Parameters

Some modules require an arbitrary voxel type and its values to be selected and handled. The ML offers the fields MLDataTypeField and UniversalTypeField to meet this requirement.

An EnumField can simply be configured to offer a selectable list of all standard and registered voxel types to the user.
A UniversalTypeField allows to handle a value from a freely selectable MLDataType; it also - with certain limitations - implicitly converts data from one type to another when its data type is changed. The filling of values in arbitrarily typed images, for example, can easily be specified, even for registered voxel types.
An MLDataTypeField stores an MLDataType value; it is useful whenever any data type needs to be specified, for example for output images and internal buffers. It is rarely used because in most cases the first version with an EnumField version is safer and easier for module users, because there is no need to write the string name of the type correctly.

The following code fragments show how to configure the output image of an ML module with one output and with a fill value of an arbitrary standard or registered voxel type.

Header file:

  //! Field containing the type of the selected voxel type. Default is MLdoubleType.
  EnumField *_voxTypeFld;

  //! Field containing the type of the selected voxel type. Default is 0.
  UniversalTypeField *_voxValFld;

C++-File, Constructor:

  handleNotificationOff();

  // Add voxel type field by using the string table of all standard and registered voxel
  // types and its size. Also set the default to the double voxel type.
  _voxTypeFld = addEnum("voxelType", MLDataTypeNames(), MLNumDataTypes());
  _voxTypeFld->setEnumValue(MLdoubleType);
  _voxTypeFld->attachField(getOutputImageField(0));

  // Add a field to the module which contains a value of the selected data type.
  _voxValueFld = addUniversalType("voxelValue");
  _voxValueFld->setDataType((MLDataType)(_dataTypeFld->getEnumValue()));
  _voxValueFld->setStringValue("0");
  _voxValueFld->attachField(getOutputImageField(0));

  handleNotificationOn();

C++-File, handleNotification:

  // Be sure that the UniversalType field is always of the selected voxel type.
  if (field == _voxTypeFld){
    _voxValueFld->setDataType((MLDataType)(_voxTypeFld->getEnumValue()));
  }

  if (field == _voxValueFld){
    // Get the value of the selected data type as string.
    std::string strVal = _voxValueFld->getStringValue();

    // Get a pointer to memory containing the value of the selected type.
    MLTypeData *fillVal = _voxValueFld->getUniversalTypeValue();
  }

C++-File, calculateOutputImageProperties:

  // Set output image to the selected data type.
  outImg->setDataType ((MLDataType)(getDataTypeFld()->getEnumValue()));

C++-File, calcOutSubmage:

  // Fill output subimage with the user defined value.
  outSubImg->fill(*((DATATYPE*)(getFillExtValueFld()->getUniversalTypeValue())));


Chapter 7. Registered Voxel Data Types		7.3. Limitations of Registered Data Types