TOLERANT Match Release 3.0

2015/06/03

Notices

  • The german manuals were updated to reflect the 3.0 product changes.
  • Attribute wildcardDelete in matchProject is deprecated. New attribute wildcardOperation is to be used instead.
  • Optimized structure in REST and SOAP responses for getByKey request.
  • Changed default values of some profile attributes: distanceScore (House number field, now 75, was 100), wildcardScore (Plain field, now 80, was 100), prefixScore (Plain field, now 80, was 100).
  • Before this release, when DB request logging was activated, service requests were only logged to the database if either the loglevel of LOGFILE or TRACE was set to at least DEBUG. Starting with this release, the DB request logging, will always log to the database. To disable the request logging to the DB, just unconfigure the defined database in the attribute reqLogDbRef of the runtime.
  • In Release 2.1, socket timeout (default 60 seconds) was introduced to Oracle database connections. For some customers, this default value caused problems. The new default is 0, meaning „no timeout“.

New Product Features

Configuration and Administration GUI

  • For integer fields which are not mandatory sets the default value automatically when removes the current value.
  • The configuration of test mode and number of test lines was also added to execution dialog when execute an initial load, like a batch run.
  • Add field validation for the number of test lines field at the execution dialog.

Client GUI

  • Regular expressions henceforth also be used for a search by key request.
  • New attribute wildcardOperation in matchProject indicates if delete and search by key operations can be performed on multiple keys that matches the given regular expression for Id/Reference.
  • A new column „match.Key“ is returned at the output table result tab for a search by key request.

Batch

  • No new features

Service

  • getByKey supports regular expression as match.Key input to perform search on multiple keys that match the pattern.

General

  • No new features

Security

  • No new features

Performance

  • No major improvements

Miscellaneous

  • If Match Batch is configured to use batch update when writing in a database, the following should be taken into consideration:
    • If some statements in the batch could not be executed, then the whole batch is rolled back.

Fixed Bugs

Configuration and Administration GUI

  • Bug #270: Integer field which refers to configuration attributes which are not mandatory by dtd are no more marked as mandatory fields.
  • Bug #2731: No default value was set for the number of test lines after activate the test mode directly before process a batch run.
  • Bug #2755: Empty delimiters are not written in the configuration file.
  • Bug #2784: File paths under windows were changed to lower case after a apply click at file selector dialog.
  • Bug #2796: Copy a project at a service instance does not work.
  • Bug #2809: After opened the output preview for a database output in a batch configuration, it was possible that the next batch run hangs up.
  • Bug #2825: Creation of synonyms file from template fails if ‚config‘ directory is other than default
  • Bug #2826: Logging configuration in Admin-GUI leads to inconsistent configuration file
  • Bug #2827: Adding of logging configuration to Match Batch project fails in Admin GUI

Client GUI

  • Bug #2733: If the system locale not German, some systems cannot read the external configuration file.
  • Bug #2814: Query History entries in Match GUI do not reflect activ flag if field was empty
  • Bug #2833: Statues shows No hits found. even when results were found

Service

  • Bug #2736: ClusterNode does not synchronize after reconfigure Project.
  • Bug #2751: The windows service registration script winserviceRegister.bat does not work correctly if external jar-files are used.
  • Bug #2804: Admin commands may fail, if db request logging is activated for a database where an empty string is treated as NULL value (eg. ORACLE).

Batch

  • no bugs were fixed

Installer

  • Bug #2776: Silent installation for https failed with exception
  • Bug #2778: Installation to / fails with exception

General

  • Bug #2754: Internal error when searching in PLAIN field with PREFIX mode.
  • Bug #2793: SingleScore 255 (instead of 0) for MatchFields that are inactive in rule.

Known Bugs

  • Bug #2797: Attribute outputRecordType in element does not work correctly for the values „HEAD“ and „FOLLOW“.

Requirements

  • Please refer to the manual

Installation/De-Installation

  • Please refer to the manual

Release 2.1, 2015/04/10

Notices

  • There is a plethora of new parameters for the silent installation:
    • windowsServiceName and windowsServiceDisplayName. These are relevant for the windows installation only and can be set to change the windows service name. If they are unset, the default name (tolerant_match_service) will be used.
    • doHttps If set to true, then HTTPS will be activated for the web server (if configured) and a self-signed certificate will be created.
    • httpsPort If doHttps is set, then this will be the HTTPS port.
    • switchHTTPOff If set to true, then the HTTP port will be disabled. Only valid of doHttps is set.
    • https.password If doHttps is set, then this will be the password for the key-/truststore
    • https.hostname If doHttps is set, then this name value be used for the certificate generation (CN)
  • Windows only: During the installation, new questions will appear. If the question „Change Windows Service Name“ is answered positively, the internal windows service and the display name for the windows service will be asked and the installer will then change the service registration scripts.
  • All platforms: During the installation, new questions will appear. HTTPS configuration can now be handled by the installer directly.
  • The response parameter „runtimeState“ for the administration service command request „state service“ was renamed to „serviceState“.
  • The default value for codepage attributes were changed from „ISO8859-1“ to „ISO-8859-1“.
  • Time spent on garbage collection will be monitored periodically every 5 seconds. A warning is shown if moving average time spent for garbage collection exceeds threshold. Threshold is 5 percent by default or can be set with system property „tl.gcruntimethreshold“
  • The password column was removed from the databases table-view in the Administration GUI.
  • New security rights synonyms.check, normalizer.check and clusterfile.check have been added. They are only used by the Administration GUI for the new file validation feature.
  • The original data column in the databases view of Match Administration GUI was removed and replaced with a new tree item.

New Product Features

Configuration and Administration GUI

  • The Configuration of database usage for Match batch input/output and Match service initial load can now be carried out using the administration GUI. This new feature requires that the user has both database.select and database.check rights.
  • A new button for synonym, normalizer and cluster file configuration has been added. Click this new button to check the configured file for correctness.
  • In the window „Execute batch job?“ shown when starting a batch process, a new checkBox was added to enable the testmode. When the testmode is enabled, the number of lines used for testing can be entered.

Client GUI

  • No new features

Batch

  • The batch process can now be started with the option –test . This activates – regardless of the setting of testmode in the configuration – the test mode and only the first   will be processed.
  • The number of maximum file partitions was increased to 2000

Service

  • When a Match service that uses a paging database for its projects is started after an unexpected termination (e.g. system error) now the database locks are automatically removed and re-created, if the locks originally have been created by the same projects on the same host (ip address) and port number.
  • Implementation of service cluster has been re-designed. Now (including others) the following use cases are supported
    • Adding of naked cluster nodes without data with following data synchronization.
    • Restart of nodes with following re-synchronization.
    • Re-synchronization after initial load of one cluster node, bi-directional transfer of differences.
    • Re-synchronization after initial load of one cluster node, uni-directional transfer of differences (all nodes have initial-load-data afterwards, see new attribute „clusterMode“ below).
    • „Reconfigure runtime“ and „reconfigure project“ work with service instances in cluster.
    • Use of files to store index.
    • Error messages will be issued if configurations differ between same project in different cluster nodes.
  • Checking runtime or project status for service also delivers cluster specific information, like clusterState, clusterName, clusterNodeName, etc.
  • The initial load can now be started with the option –test . This activates – regardless of the setting of testmode in the configuration – the test mode and only the first   will be processed.
  • New attribute „isMandatory“ of tag „matchRuleField“ (values „Y“/“N“, default „N“): if set to „Y“, the corresponding rule is activated only if this match field is enabled in the Match query.
  • New attribute „clusterNodeName“ of Runtime tag (default empty): improves readability of log/trace files, because automatic node names are (mostly) replaced by the configured name. In addition this gives more control for the (very rare) case when conflicting events for the same Match-key occur at different cluster nodes. Then the node with the lexicographically higher name ‚wins‘.
  • New attribute „clusterMode“ of tag „matchProject“ (values „MASTER_SLAVE“/“MASTER_FORCE_OVERWRITE“, default „MASTER_SLAVE“): if set to „MASTER_FORCE_OVERWRITE“ you can start a node after initial load and its data will replace the data in all cluster nodes.
  • New attribute „clock“ of Runtime tag (values „SYSTEM“/“CLUSTER“, default „SYSTEM“): for „SYSTEM“ the system clock of the respective cluster node is used. This is OK in most situation. If time synchronization of nodes is bad, „CLUSTER“ can be used. Then a cluster protocol synchronizes the clocks.

Installer

  • The default „Windows Service Name“ can now be changed during the installation.
  • HTTPS configuration can now be handled by the installer directly.

General

  • There is a new attribute allowEmptyResults for the synonymFile tag. If set to „N“, a synonym replacement that leads to an empty result will not be performed.
  • Improve the validation output for cluster and normalizer files.

Security

  • The security tool has new options to hash passwords.

Performance

  • No major improvements

Miscellaneous

  • No new features

Fixed Bugs

Configuration and Administration GUI

  • Bug #978: No error popUp occurred when opened the project view with a wrong service inputFieldmap mode.
  • Bug #1781: Project-Id and profile-Id fields are checked for invalid characters.
  • Bug #2541: It was not possible to change the configuration file path of a service file product instance, after changed the installation path.
  • Bug #2556: If codepage attribute not set, then an unnecessary save popUp occurred when leaving the project configuration without changes.
  • Bug #2564: If ruleCut=“N“ is set in profile and one of its child rules has set ruleCut=“Y“, then the Administration GUI saves the ruleCut of the child rule with „N“ instead of „Y“.
  • Bug #2566: Opened the project list of a service instance shows an error PopUp with a wrong failure message, in case the configuration file on service has an error.
  • Bug #2567: Administration service request which can call at service instances and needs the maintenance service state, do not work if service already at maintenance state.
  • Bug #2568: If GUI starts with an anonymous user then cannot create a new product instance from type service instance in case of security was activated for the service area.
  • Bug #2575: Cluster tab was not displayed at runtime settings.
  • Bug #2597: The wrong Batch-Process was called, if installation- and execution path were changed.
  • Bug #2599: A PopUp with an error message which tells the project is opened in read mode occurred, when edit a product instance with a relative configuration file path. This is wrong. Now the PopUp occurred not anymore.
  • Bug #2607: The login popUp was not opened if the user role have no „project.list“ right to run a batch command from execute menu at project list.
  • Bug #2610: Paging view does not update its databases selection box when new databases are configured.
  • Bug #2611: The create, save, copy and delete project operation on a service instance writes batch only attributes to the remote configuration file.
  • Bug #2638 Textfield for matchrulecondition (with regexp) does not allow certain characters.
  • Bug #2674: The GUI encrypts the entered password without checking whether the password is already encrypted.
  • Bug #2683: If security is activated and a user has no right to edit a synonym, normalizer or cluster file, then login popUp is shown. Clicks the user the cancel button on the login popUp, the popUp is closed, but any more click to the edit button does nothing. Now the login popUp is showed again.
  • Bug #2685: A internal error is occurred if remove a synonym or normalizer file path and change the focus to another field.
  • Bug #2701: After try to create a new product instance without required rights, it is not possible to create another product instance with the same name.
  • Bug #2707: A canceled Batch process at project list view cannot restart anymore without leaving the view.
  • Bug #2730: Expanding the filepath at file selector dialog do not work in case of a new product instance, windows operation system and a TOLERANT_HOME which starts with a lower case character.
  • Bug #2732: Detailed view for the input and output files miscounts the line number.
  • Bug #2789: File preview shows the last line twice.

Client GUI

  • no bugs were fixed

Service

  • Bug #1954: Error while starting up one project prevents other projects to start up and service status becomes ERROR
  • Bug #2574: IMPORT-function in matchServiceTool does not work on empty database.
  • Bug #2581: Tolerant Match search using PLSQL-Client doesn’t work properly with all result modes.
  • Bug #2603: Reading database input for initial load returned a row as a line with null values if corresponding columns had null values. This led to NullPointerException while processing the input for synonyms.
  • Bug #2605: Endless loop in Match kernel in serialization function for key map caused initial load to fail when using multiple aggregations in partitioning.
  • Bug #2623: Option DELETE_SAVEPOINTS of tool matchAdjustPagingDatabase does not work for SQL databases.
  • Bug #2624: Synchronization on startup between cluster nodes did transfer too often full save points.
  • Bug #2634: Internal error „No existing mapping for in-key ‚0‘ in partition ‚65535‘“ during initial load.
  • Bug #2647: Match Service crashes during insert operation.
  • Bug #2648: Consistency check shows wrong error for fields with no values.
  • Bug #2649: Warning on Service stop because of wrong book keeping of used page memory.
  • Bug #2664: Wrong error message after DELETE during save operation if INSERT was also during same save operation.
  • Bug #2665: Cluster operations fail after reconfigure project.
  • Bug #2692: Unnecessary connection proxies created for DBInput/Output databases when project runs in service
  • Bug #2693: admin command „state project“ for not correctly configured project results in exception.
  • Bug #2694: admin command „stop project“ for not correctly configured project results in exception.
  • Bug #2699: admin command „stop project“ for not being able to stop projects in state other than RUNNING.
  • Bug #2700: Reconfigure project not possible if it is in any other state than RUNNING.
  • Bug #2719: Wrong encoding for multibyte Unicode characters in transport of Backlog data between cluster nodes.

Batch

  • Bug #344: Maximal number of partitions was restricted by threadcount. Partition count is now limited to 2000.
  • Bug #2584 Match Batch might have aborted with an error, that a file could not be found, if the testmode was used.
  • Bug #2603: Reading database input for batch processes returned a row as a line with null values if corresponding columns had null values. This led to NullPointerException while processing the input for synonyms.
  • Bug #2641: In case of an internal match kernel error, the batch processing did not count the errors correctly
  • Bug #2718: File partitioning did not work if the first column of the input file was used for partitioning.

Installer

  • Bug #1300: If the installer was started in a directory called „install“, the installer deleted itself.
  • Bug #1540: The installer now checks for write-only files and disk space before installing. If a problem occurs the installer will warn about the problem, but continue.
  • Bug #2563:
    arch
    command is required for the installer script under Linux.
  • Bug #2583: The installer will now check each file after the installation, if it matches the file in the installer package. In case a file was not overwritten, a warning will be generated.

General

  • Bug #1499: csv file reader has problem escaping a character if the quote character was not used.
  • Bug #2298: synonymList does not work with non character fields. The use of synonymList for a field of type D or L is no longer allowed.
  • Bug #2569: If STOREONLY fields were used, the singlescores were output in the wrong order.
  • Bug #2576: Sporadic internal error during initial load when duplicate keys exist.
  • Bug #2577: Poor performance in fuzzy index if most values have same length.
  • Bug #2587: Cluster node does not start up if empty local backlog exists.
  • Bug #2593: If an initial load fails in the very last steps, program reports ’success‘ on console despite the failure.
  • Bug #2645: If a project configuration contains errors the project state will now be „ERROR“ instead of „UNKNOWN“.
  • Bug #2652: Rule does match if field score>max-score in case of min-score=0.
  • Bug #2659: while adding a new user, the security tool does not check if the user already exists.
  • Bug #2670: matchEmptyFieldvalues=“N“ does not work correctly if index and search value are both empty and minScore=“0″.
  • Bug #2673: If outputFieldmapItem is used more than on time, it should reference fields that can be concatenated.
  • Bug #2675: Error when phoneticUsage has different values for one field in different profiles.
  • Bug #2702: For linux systems the entropy gathering device was set to /dev/urandom. This should help prevent problems with ORACLE connections, that seem to hang forever. See also http://docs.oracle.com/cd/E13153_01/wlcp/wlss40/configwlss/jvmrand.html

Known Bugs

  • No known bugs

Requirements

  • Please refer to the manual

Installation/De-Installation

  • Please refer to the manual

Archived release notes