TOLERANT Match Release 8.0

Notices

• The usage of MongoDB databases is now supported for original data. This enables the usage of higher data volumes for this use case.
• The SQLite database driver is now fully integrated in Tolerant products.
• Starting with version 8.0 of Tolerant Match, the product needs a valid license file to start. Please contact our support team to get a license file suitable for your use case.
• The admin-command „get environment“ returns a json object instead of a string for the key „ENVIRONMENT_VARIABLES“. This change breaks the backward compatibility for the admin GUI and service.

New Product Features

Configuration and Administration GUI

• The correct paths are being used now, when editing a batch configuration of an installation with the admin GUI of another installation.
• The admin GUI now tries to detect the codeset of input files.
• The function used to determine output and/or input fields from files was extended, to enable the generation of fields even if the file does not have a header line.

Client GUI

• With the new result mode resultMode=5, it is now possible to get information about the internal search process of the Match kernel in a service response.
• The query history in the client GUI now shows the project of the request that has been sent.

Batch

• The console output of batch processes was extended to show the path of input and output files.
• Match batch statistics were extended to show the hit count of all rules (even if the hit count is zero).

Service

• The usage of MongoDB databases is now supported for original data.
• New serviceOutputField attributes allow changing the default sorting of service results:
  • sortMode: defines how the comparision between field’s values is done.
  • sortFieldOrder: determines the order of the field in the sorting criteria.
  • sortAsc: ascending or descending sorting.
• The perfomance of the initial load was optimized. It can now use the parallelization in the Match kernel more effectively.
• The admin-command „get environment“ returns a json object instead of a string for the key „ENVIRONMENT_VARIABLES“.

Security

• An example which can be used to change the default constraints on the ‚dir.list‘ right was added to the example ’security.xml‘.

API

• The Java API does not use a certain logging library. An internal implementation is used for basic logging. The logging level can now be changed using the java property tolerant.api.logLevel. Please refer to the manual for further details about possible logging configurations.
• The Java API has now new methods to change the project and profile of a connection instance.
• The CAPI now provides two setter methods to change the project (matchSetProject) and the profile (matchSetProfile) of an active CAPI connection.

General

• The audit logfile now contains the IP address of the caller.
• The configuration check now checks the runtime section only once.
• The ‚checkinstallation‘ tool will now compare checksums generated for TLCONFIG to determine whether configuration files were changed after the installation.
• The Attribut ‚tokenSubset‘ in ‚matchProfileField‘ has now three new possible values:
  • INDEX_MIN_1: at least one token of the search query should be found in Match index
  • INDEX_MIN_2: at least two tokens of the search query should be found in Match index
  • INDEX_MIN_3: at least three tokens of the search query should be found in Match index
• The SQLite database driver is now fully integrated in Tolerant products.
• The log messages of third party libraries can be redirected to the Tolerant trace file. For example, adding the following ‚parameter‘ element will change the log level of the hikaricp connection pool to TRACE: The default log level of integrated opensource libraries is ‚ERROR‘.
• Tolerant Match now uses a license file to control the used volume and the validity period of the product. The product cannot be started without a valid license file.
• An example of how to adjust the java heap memory setting can be found in the ‚tolerantEnv‘ script file located in the configuration directory of a new installation.
• The distribution of available memory between different projects and fields right at the start of Tolerant Match was optimized.

Fixed Bugs

Configuration and Administration GUI

• TPR-557: When trying to save an invalid configuration, the user now has the option to discard the last changes.
• TPR-4432: The environment variables were substituted using the local system variables, if the admin GUI user does not have the right ‚environment.get‘. After the bug fix, the admin GUI will not try to substitute environment variables unless the user has sufficient rights.
• TPR-4460: Extra measures were implemented to avoid javascript injection attacks.
• TPR-4557: The action of editing a project using the admin GUI now checks whether the current user has sufficient rights.
• TPR-4742: Problems when starting the admin GUI with a different locale than the one stored in the cookies have been resolved.
• TPR-4850: A problem with deleting faulty method configurations (for example reference or negative) in the admin GUI was resolved.
• TPR-4911: The function ‚User Column definition from File‘ now handles files with BOMs correctly.
• TPR-4922: Deletion of Match output in the admin GUI now works properly.
• TPR-5034: Creating a new project for a service instance in the admin GUI now uses the templates from the service instance, not those of the local instance.
• TPR-5054: The console output no longer shows an exception when the execution of a command line task was terminated from the admin GUI.
• TPR-5117: Validation errors now are only written once for the same database connection.

Client GUI

• TPR-4388: The query history drop down now contains the last search by key requests.
• TPR-4392: Extra measures were taken to prevent a concurrent modification error, when handling error responses from server.
• TPR-4727: The client GUI no longer checks the connection to the configured databases.
• TPR-5038: When logging out from the client GUI, the cached connections to the server are now resetted to overrule old credentials .

Batch

• TPR-4410: The attribute ‚maxTotalScore‘ is now checked to have a bigger value than ‚minTotalScore‘.
• TPR-4809: The validation for the filepath attribute in summaryFile element was corrected, so that a warning is written instead of an error, since the attribute has a default value.
• TPR-4978: Errors that occur while writing the output in database are now counted correctly in statistics on console.

Service

• TPR-4265: The responses of delete with wildcards bulkrequests now contain the list of deleted keys.
• TPR-4443: The order of projects‘ states written during the start of the service is now deterministic.
• TPR-4519: The environment variable BUILDNUMBER will now be set correctly in case the product is registered as a windows service.
• TPR-4612: Deletion of entries in the original database now work correctly.
• TPR-4628: The proxy engine now uses an internal data structure that preserves the order of internal and original data fields
• TPR-4641: The evaluation of the ‚active‘ flag in Match project was corrected. If active flag is set, the incative project will be ignored during the configuration phase. Please note that this flag is only relevant for Match service.
• TPR-4730: The service state will be changed from UNKNOWN to CONFIGURING right after reading the configuration file.
• TPR-4773: Some command line tools, for example ‚matchServiceTool‘ didn’t interpret the ‚active‘ flag in the project bean configuration correctly.
• TPR-4774: The admin commandline tool no longer returnes ’null‘ for product name when trying to execute unimplemented commands.
• TPR-4795: The codeset of bulk responses is now always set to UTF-8.
• TPR-4854: Error messages logged after failed service requests were extended to include the cause of the failure.
• TPR-4923: The standard input/output streams of the command line tools can now be redirected correctly.
• TPR-5015: Starting Match service as part of a cluster and with a corrupted paging database is now handled correctly.
• TPR-5070: The path to the directory containing the backlog files is now resolved correctly.
• TPR-5076: Logging initialization for initial load was adjusted to show exceptions correctly in case of an error.
• TPR-5081: The initial load now checks the validity of the configuration before starting the load process.

API

• TPR-4445: Service result codes are now handled correctly in PL/SQL API.
• TPR-4455: The paths of the specification and body scripts in the installation package for PL/SQL client were corrected.
• TPR-4703: ‚resultOverflow‘ in JSON result is now only parsed if the ‚resultCount‘ is bigger than zero.
• TPR-5046: The integrated ‚curl‘ library in Match C-API no longer links unnecessary and on some systems not existing libraries.

General

• TPR-4369: The installer no longer changes the default value of JETTY_HOST when upgrading an older version of the product.
• TPR-4377: The option show of the support script now writes the list of changed files to the console.
• TPR-4545: New measures were implemented to prevent a memory leak from occurring when logging messages or exceptions from internal kernel threads.
• TPR-4734: On deletion of index entries far too many pages were treated as modified. Now only really changed pages are marked as modified and thus written to the paging database afterwards.
• TPR-4823: An optimization was implemented to avoid excessive adjustments of the available memory between the different page controllers.
• TPR-4853: Fixed access violation in Match kernel after out of memory error.
• TPR-4904: All hits are now returned correctly for compare mode ‚DICECOEFF‘.
• TPR-4920: Token scores are now correctly limited by ‚tokenMinScore‘ and not by ‚minScore‘ in case of multistring fields with phonetics.
• TPR-5016: The creation of a full save point now correctly considers the case, which prevents the Match service from starting without a save point.
• TPR-5023: Match service no longer freezes after service timeout in some cases.
• TPR-5043: Protocol and port entered during the installation will now be used for the PL/SQL client example.
• TPR-5077: All resources will now be properly released if a failure occurres while executing command line tools.

Known Bugs

• TPR-5089: A service inactive project cannot be reactivated and started from the administrative GUI. A workaroung would be to stop the project, set active flag to ‚Y‘, save it and start it again.
• TPR-5138: Match proxy engine does not handle bulk insert requests correctly.
• TPR-5143: The new resultMode (5) cannot be used when original data is configured.