ThirdPartyLicenseReport¶

MacroModule¶

genre	`Info`
author	`Julius Bierbaum`
package	`FMEstable/ReleaseMeVis`
definition	ThirdPartyLicenseReport.def
keywords	`thirdparty`, `license`, `report`

Purpose¶

Compares the licenses of all third parties delivered with the respective release of MeVisLab against a provided “allow/deny-list”.

Usage¶

Via the MeVisLab GUI - Instantiate the module and provide a path to a folder containing one or more license list files. Also specify an output destination for the report files. Then press Update.
Via the command-line - Provide path to the license list files and an output directory. Optionally you can instruct the module to return a non-zero exit code in case any ThirdParty license conflicts with license list.
```
MeVisLab.exe -noide -runappbatch ThirdPartyLicenseReport --license-list-location <path> -o license_report --fail-on-license-issue
```

Details¶

License lists

The license list (formerly known as allow-list or whitelist) files are searched for in a directory path passed via input parameter. Each file consists of a set of license identifiers. The file format is YAML and file names must match the pattern *.<status>.licenses.yaml, where <status> can be either “allowed” or any arbitrary string.

Licenses that are not listed in any license list or are explicitly listed in a file with a <status> string different than “allowed” are considered not allowed / denied - which may be interpreted as:: “The license is not generally allowed and may require a manual case-specific approval apart from this automatic check.”

Licenses can optionally have constraints, which indicate that the respective license is only allowed under specific circumstances (validated in a manual process for each affected third party). Possible types of constraints are conditions and requirements. Both types of constraints are noted as lists and can contain an arbitrary number of elements (plain text). This script will only check for (non-)existence of any constraint - the text itself is ignored.

Example file:

MIT:
LGPL-2.1-only:
BSL-1.0:
  conditions:
    - Not allowed if you provide modified boost source code with your software
    - ...
MEVIS-XY-MIT:
  requirements:
    - Some requirements ...
MEVIS-35-BSD-LGPL-3:
BSD-3-Clause:

Clearance list

License lists specify general policies which may need to be assessed and adjusted for each individual use-case. You can provide a clearance list file with a filename matching the pattern *.clearance.yaml. Any entry allows a specific ThirdParty (in a specific version) to have a license even though it is not generally allowed.

Example file:

PythonQt:
  version: 3.0
  license: LGPL-2.1-only

xylib:
  version:     1.4
  license:     LGPL-2.1-only
  review_jira: TPL-1338

mlinfo Ingestion

The information (license, homepage, …) about the thirdparty components is read from .mlinfo files. There are some special cases:

If Patch third party information is enabled, the contents from an external YAML file specified through In Ml Info Override File be used to override specific .mlinfo fields.
If an .mlinfo file contains a base_purl or package_source entry, the homepage entry is overwritten from the one in base_purl (e.g. github/google/protobuf results in homepage = https://github.com/google/protobuf) or package_source (e.g. pypi for component pydantic results in homepage = https://pypi.org/project/pydantic).

Output

The generated output consists of multiple CSV files and a folder containing a copy of the license text of each third party that has a conflicting license.

report.csv contains the license identifiers (usually SPDX) of all licenses applied to any third party. For each license it provides info of it’s status (e.g. “allowed”, “unknown”, “allowed with constraints”, or an arbitrary status text that was defined via license list file (in the filename).
thirdparties.csv contains details from .mlinfo files of all detected third parties (which are delivered as part of the MeVisLab instance).
optionally license-list.csv lists all licenses found in the license list file(s) merged together. Does not include text of constraints.

Tips¶

The license list files that apply to Fraunhofer MEVIS are located at svn.fme.lan/dev/trunk/FMEstable/General/ThirdPartyInformation.

Windows¶

Default Panel¶

../../../Projects/ThirdPartyLicenseReport/Modules/mhelp/Images/Screenshots/ThirdPartyLicenseReport._default.png

Parameter Fields¶

Field Index¶

`[]`: `Trigger`	`Include license list in report`: `Bool`	`Update`: `Trigger`
`Apply Clearance list`: `Bool`	`License list location`: `String`	`Use 'license-report' subfolder at destination`: `Bool`
`Clear`: `Trigger`	`On Input Change Behavior`: `Enum`
`Destination folder`: `String`	`Patch third party information`: `Bool`
`doNotClearOnFailedUpdate`: `Bool`	`Report summary`: `String`
`Has Valid Output`: `Bool`	`Restrict subfolder with license texts to those with unclear 'allow' status`: `Bool`
`In Clearance List Location`: `String`	`Status Code`: `Enum`
`In Ml Info Override File`: `String`	`Status Message`: `String`

Visible Fields¶

Update¶

name: update, type: Trigger¶: Initiates update of all output field values.

Clear¶

name: clear, type: Trigger¶: Clears all output field values to a clean initial state.

On Input Change Behavior¶

name: onInputChangeBehavior, type: Enum, default: Clear, deprecated name: shouldAutoUpdate,shouldUpdateAutomatically¶: Declares how the module should react if a value of an input field changes.

Values:

Title	Name	Deprecated Name
Update	Update	TRUE
Clear	Clear	FALSE

[]¶

name: updateDone, type: Trigger, persistent: no¶: Notifies that an update was performed (Check status interface fields to identify success or failure).

Has Valid Output¶

name: hasValidOutput, type: Bool, persistent: no¶: Indicates validity of output field values (success of computation).

Status Code¶

name: statusCode, type: Enum, persistent: no¶: Reflects module’s status (successful or failed computations) as one of some predefined enumeration values.

Values:

Title	Name
Ok	Ok
Invalid input object	Invalid input object
Invalid input parameter	Invalid input parameter
Internal error	Internal error

Status Message¶

name: statusMessage, type: String, persistent: no¶: Gives additional, detailed information about status code as human-readable message.

License list location¶

name: inLicenseListLocation, type: String, default: $(MLAB_FMEstable_General)/ThirdPartyInformation¶: Path to a directory containing one or more license list files matching a certain name pattern.

Apply Clearance list¶

name: inUseClearanceList, type: Bool, default: FALSE¶: If checked, a file path to a clearance list is accepted as input parameter.

In Clearance List Location¶

name: inClearanceListLocation, type: String, default: $(HOME)/my-project-specific-clearance-list.yaml¶: Path to a clearance list containing a list of ThirdParties whose license usage will be considered allowed.

Destination folder¶

name: inDestinationFolder, type: String, default: $(HOME)¶: Path to a directory to put in the license report files. Depending on Use 'license-report' subfolder at destination, an addition subfolder will be created/used.

Use ‘license-report’ subfolder at destination¶

name: inUseLicenseReportSubFolderAtDestination, type: Bool, default: TRUE¶: If set, the module will always append an additional subfolder named ‘license-report’ to Destination folder, and if it does not yet exist, it will be auto-created (but the parent folder must already exist)

Include license list in report¶

name: inIncludeLicenseList, type: Bool, default: FALSE¶: If checked, the applied license list read from YAML files is dumped to a CSV file in the report as well. Useful for testing.

Restrict subfolder with license texts to those with unclear ‘allow’ status¶

name: inRestrictLicenseTextFolderToLicensesWithUnclearStatus, type: Bool, default: FALSE, deprecated name: inRestrictLicenseTextFolderToUnknownLicenses¶: If checked, only the information from licenses with status other than ‘allowed’ (with or without constraints) or ‘cleared’

Patch third party information¶

name: inUseMlInfoOverrideFile, type: Bool, default: TRUE¶

Enable overriding selected mlinfo entries as specified through an external file.

Sometimes the information from the .mlinfo files is not 100% up-to-date, or in a different way than it should be presented in the thirdparty/license report. For such cases, you can override specific properties of specific thirdparty component through an external a YAML file.

In Ml Info Override File¶

name: inMlInfoOverrideFile, type: String, default: C:/dev/cmake/fme/FMEstable/ReleaseMeVis/Projects/ThirdPartyLicenseReport/Modules/resources/mlInfoOverridesForLicenseReport_default.yaml¶

YAML file with a (TP) dictionary of (mlinfo property) dictionaries. Note that only known mlinfo properties will be used.

Example Override file:

lzma_sdk:
  license: "LZMA-SDK-9.22"
  licenseComment: "See license section on the page, the 'Public Domain' corresponds to https://spdx.github.io/license-list-data/LZMA-SDK-9.22.html"
newmat:
  licenseComment: "License mentioned in Section 1.1 on this webpage: http://www.robertnz.net/nm10.htm"

Report summary¶

name: outSummary, type: String, persistent: no¶: A short summary of the resulting license report.

Hidden Fields¶

doNotClearOnFailedUpdate¶

name: doNotClearOnFailedUpdate, type: Bool, persistent: no¶: Prevents automated clear after update failed. This does not affect status fields. It enables the developer to analyze module’s state after failure.