Tokenization process in Master Data Management

           
This article provides detailed information on the tokenization process in Master Data Management (MDM). What happens internally during the tokenization process? What are the performance challenges and how to fix these during the tokenization process? All these questions will be answered in this article.

What is tokenization?
         For example, people visit the Dept. of Motor Vehicles (DMV) office for various purposes: drivers license application, vehicle registration, or identity card acquisition, etc. How are these different kinds of requests handled at the DMV? In order to address different types of requests, first similar requests are grouped together by providing particular types of tokens. Similar kinds of tokens will be addressed in one queue.
         In a similar manner the MDM tokenization process works. Different kinds of of tokens are assigned to different kinds of data. Similar kinds of tokens are grouped together and matched with each other during the matching process.
         In the MDM tokenization process, similar records as well as exact records are grouped together. In order to achieve tokenization, we need to create fuzzy match rules. Exact match rules will not generate tokens as exactly matched records.

What are match tokens?
These are encoded and non-encoded representations of the data in base object records. The tokenization process generates one or more tokens for each record.

Match tokens include following things:

1. Match keys are fixed-length, compressed strings consisting of encoded values built from all of the columns in the Fuzzy Match Key of a fuzzy-match base object. Match keys contain a combination of the words and numbers in a name or address such that relevant variations have the same match key value.
2. Non-encoded strings consist of flattened data from the match columns (Fuzzy Match Key as well as all fuzzy-match columns and exact-match columns).

How are tokens generated?

      Token generation is a very complex process. The generation of tokens is based on several factors: the match columns configurations, fuzzy criteria, and the length or width of tokens. For more details you can watch this video.

In which table are tokens stored?

       Various temporary tables are created during the token generation process. Temporary tables are dropped at the end of the process. Generated tokens are stored in the C_<BO Table >_STRP table.  It contains the columns below.

1. SSA_KEY : Match key for the record. It is the encoded representation of the value in the fuzzy match key column, such as the name, address, or organization name, for the associated base object record. The string consists of fixed-length, compressed, and encoded values built from a combination of the words and numbers in a name or address.
2. SSA_DATA : Non-encoded, plain text, string representation of the concatenated match columns defined in the associated base object table. Concatenated match columns include the fuzzy match key as well as all fuzzy match columns and exact match columns.

Why does the tokenization process cause performance issues?

      Performance issues either in the tokenization process or other processes depend on the factors below:

• Volume of records getting processed
• Hardware configuration
• MDM Hub configuration 
• Application server configuration
• Data and Hotspot issues

      Out of the above mentioned causes, the MDM configuration-related problems can be fixed by tuning the MDM hub and the other configurations in the properties files. Similarly, the application server- related configuration can be fixed.

     However, to address volume or data related issues, we need to take inputs from business users and data stewards. Any architectural or hardware level issues must be addressed in the designing phase.

How to write set method for classes generated by wsdl2java

If we generate classes using the wsdl2java tool then some of the classes do not generate the setter method since the get method will act as the setter method. Few of the calling applications need the setter method.

Below is an example of how to write the setter method for the classes generated by the wsdl2java tool.

e.g. Let's consider the example of the AddressList calls which contains the object of the Address class.

public class Address {
    protected String addressLine1;
    protected String addressLine2;
    protected String cityName;
    protected String stateName;
    protected String countryCode;
    protected String countryName;
    protected String zipcode;

    /**
     * Gets the value of the addressLine1 property.
     * 
     * @return
     *     possible object is
     *     {@link String }
     *     
     */
    public String getAddressLine1() {
        return addressLine1;
    }

    /**
     * Sets the value of the addressLine1 property.
     * 
     * @param value
     *     allowed object is
     *     {@link String }
     *     
     */
    public void setAddressLine1(String value) {
        this.addressLine1 = value;
    }

    /**
     * Gets the value of the addressLine2 property.
     * 
     * @return
     *     possible object is
     *     {@link String }
     *     
     */
    public String getAddressLine2() {
        return addressLine2;
    }

    /**
     * Sets the value of the addressLine2 property.
     * 
     * @param value
     *     allowed object is
     *     {@link String }
     *     
     */
    public void setAddressLine2(String value) {
        this.addressLine2 = value;
    }

    /**
     * Gets the value of the cityName property.
     * 
     * @return
     *     possible object is
     *     {@link String }
     *     
     */
    public String getCityName() {
        return cityName;
    }

    /**
     * Sets the value of the cityName property.
     * 
     * @param value
     *     allowed object is
     *     {@link String }
     *     
     */
    public void setCityName(String value) {
        this.cityName = value;
    }

    /**
     * Gets the value of the stateName property.
     * 
     * @return
     *     possible object is
     *     {@link String }
     *     
     */
    public String getStateName() {
        return stateName;
    }

    /**
     * Sets the value of the stateName property.
     * 
     * @param value
     *     allowed object is
     *     {@link String }
     *     
     */
    public void setStateName(String value) {
        this.stateName = value;
    }

    /**
     * Gets the value of the countryCode property.
     * 
     * @return
     *     possible object is
     *     {@link String }
     *     
     */
    public String getCountryCode() {
        return countryCode;
    }

    /**
     * Sets the value of the countryCode property.
     * 
     * @param value
     *     allowed object is
     *     {@link String }
     *     
     */
    public void setCountryCode(String value) {
        this.countryCode = value;
    }

    /**
     * Gets the value of the countryName property.
     * 
     * @return
     *     possible object is
     *     {@link String }
     *     
     */
    public String getCountryName() {
        return countryName;
    }

    /**
     * Sets the value of the countryName property.
     * 
     * @param value
     *     allowed object is
     *     {@link String }
     *     
     */
    public void setCountryName(String value) {
        this.countryName = value;
    }

    /**
     * Gets the value of the zipcode property.
     * 
     * @return
     *     possible object is
     *     {@link String }
     *     
     */
    public String getZipcode() {
        return zipcode;
    }

    /**
     * Sets the value of the zipcode property.
     * 
     * @param value
     *     allowed object is
     *     {@link String }
     *     
     */
    public void setZipcode(String value) {
        this.zipcode = value;
    }
  }

Below is example for writing setter method -

public class AddressList {

    protected List<Address> address;

    /**

     * Gets the value of the address property.

     * 

     * <p>

     * This accessor method returns a reference to the live list,

     * not a snapshot. Therefore any modification you make to the

     * returned list will be present inside the JAXB object.

     * This is why there is not a <CODE>set</CODE> method for the address property.

     * 

     * <p>

     * For example, to add a new item, do as follows:

     * <pre>

     *    getAddress().add(newItem);

     * </pre>

     * 

     * 

     * <p>

     * Objects of the following type(s) are allowed in the list

     * {@link Address }

     * 

     * 

     */

    public List<Address> getAddress() {

        if (address == null) {

            address = new ArrayList<Address>();

        }

        return this.address;

    }

    

    public void setAddress(Address oaddress) {

     if (null != address)

     address.add(oaddress);

     else {

     List<Address> listAddress = new ArrayList<Address>();

     listAddress.add(oaddress);

     this.address = listAddress;

     }

    }

How to Handle Informatica MDM errors?

This article helps you to understand the different errors which occur during the various executions of the MDM process. This article also gives the solutions for such errors and issues.

Error 1 : SIP-28070
Scenario :  While executing Match Job through Job Viewer you get the following error:

Executing ... failed.

Cannot execute procedure at this time.
Refresh the status of this procedure.
Check the application server log for more information.
See Informatica MDM Hub documentation for more information.

Solution : 
Check the application logs. If the error below is noticed then it means that you have not configured the cleanse server.

17:02:04,993 INFO  [stdout] (AsyncAppender-Dispatcher-Thread-10 in BATCH mode.

17:02:04,993 INFO  [stdout] (AsyncAppender-Dispatcher-Thread-102)       MATCH;  BATCH;2) Caused by: com.siperian.common.SipException: SIP-28070: No Cleanse-Match server availab
le for MATCH

In order to resolve this issue, go to the Utilities section of MDM Hub and select the Cleanse Server (in MDM 9.5 version) or the Process Server (in MDM 9.7) and configure the cleanse server.

What are the MDM Best Practices for Working With Batch Jobs?

In this article we will see different Informatica MDM Best Practices for Batch Jobs. This article gives  the overall idea about various points to be taken into account while running Batch Jobs such as Stage Jobs, Load Jobs, Match Jobs, Merge Jobs.

1. Schema : The very first point is to create the proper schema as per the business requirement. The schema is very important for any activity in Informatica MDM Hub. If the schema is not present then there is nothing to do with any batch job.
2. Parallel Processes : In order to avoid issues with the MDM Hub related to the processes, limit the number of parallel processes. If more parallel processes are created then it will lead to insufficiency in the resources. 
3. Mappings in the Stage Jobs : Define mapping first before running stage jobs. Mappings are used to perform the transformation of data in the stage tables.
4. Match Rules : Define the match rules before running the Match Jobs. If no match rules are set then there will not be match records found after the Match Job is run.
5. Production Jobs :

• Before running any production batch jobs, run the test with small data sets.
• Run the individual component such as the cleanse engine component to determine whether each component is working properly or not.
• After the individual component test is successful, integrate all the components and test the entire module with the sample data and check whether everything is working well or not.

The major points above need to be considered while running the Batch Jobs in MDM Hub.

How to handle NoSuchMethodException exception

Scenario : 
The exception - NoSuchMethodException occurs if the setter or the getter method does not present in classes created during the WSDL skeleton or the client generation.

public class ElectronicAddressRequest {
    protected List<ElectronicAddressSO> electronicAddress;

    public List<ElectronicAddressSO> getElectronicAddress() {
        if (electronicAddress == null) {
            electronicAddress = new ArrayList<ElectronicAddressSO>();
        }
        return this.electronicAddress;
    }

Here is what the ElectronicAddressSO class looks like below :

public class ElectronicAddressSO {

    protected String electronicAddress;
    protected String electronicAddressType;
    public String getElectronicAddress() {
        return electronicAddress;
    }
     public void setElectronicAddress(String value) {
        this.electronicAddress = value;
    }
     public String getElectronicAddressType() {
        return electronicAddressType;
    }
     public void setElectronicAddressType(String value) {
        this.electronicAddressType = value;
    }
}

In the above code, the setter method is not created for the ElectronicAddressRequest if you use the wsdl2j tool to create the skeleton.

Error  :

When you try to run the application you will get the error below.

weblogic.wsee.jaxws.framework.policy.advertisementimpl.AdvertisementHelperImpl registerExtension
WARNING: Registering oracle.j2ee.ws.wsdl.extensions.addressing.AddressingExtensionRegistry extension failed; java.lang.NoSuchMethodException: oracle.j2ee.ws.wsdl.extensions.addressing.AddressingExtensionRegistry.registerSerializersAndTypes(com.ibm.wsdl.extensions.PopulatedExtensionRegistry)
Mar 11, 2015 12:22:12 PM weblogic.wsee.jaxws.spi.WLSServiceDelegate addWsdlDefinitionFeature
SEVERE: Failed to create WsdlDefinitionFeature for wsdl location: http://rmv-eap-spr-d11.ddc.dot.state.ma.us:57453/MARMV_MainFrame_Provider/LicenseUpdateService?wsdl, error: com.sun.xml.ws.wsdl.parser.InaccessibleWSDLException, message: 2 counts of InaccessibleWSDLException.

Solution :
In order to handle this error, add the method below in the ElectronicAddressRequest class :


public void setElectronicAddress(ElectronicAddressSO oElectronicAddressSO) {
        if (null != electronicAddress) {
            electronicAddress.add(oElectronicAddressSO);
        } else {
            List<ElectronicAddressSO> listContactInformation = new ArrayList<ElectronicAddressSO>();
            listContactInformation.add(oElectronicAddressSO);
            this.electronicAddress = listContactInformation;
        }
    }

How to configure JMS in Informatica MDM Hub?

Are you looking for an article to better understand the configuration of JMS in Informatica MDM 9.7, 10+ version with JBOSS-eap-6.2.0? This article will explain the step by step procedure to configure JMS in the MDM Hub console and the JBOSS configuration files.

Step 1 : JMS configuration in JBOSS Configuration files

 The configuration file used for the JMS queue configuration is "standalone-full.xml". This file is present at the location - Installation directory\jboss-eap-6.2\standalone\configuration. Open this file and include the code below to add the new queue to the JBOSS server.

<jms-destinations>

      <jms-queue name="queue/siperian.sif.jms.queue">

             <entry name="queue/siperian.sif.jms.queue" /> 

             <entry name="jboss/exported/queue/siperian.sif.jms.queue" /> 

      </jms-queue>

      <jms-queue name="queue/OutboundQueue">

            <entry name="queue/OutboundQueue" /> 

            <entry name="jboss/exported/queue/OutboundQueue" /> 

      </jms-queue>

</jms-destinations>

Step 2 : Start the JBOSS server

Use the command below to start the Jboss server.

standalone.bat -c Standalone-full.xml -b 0.0.0.0 -bmanagement 0.0.0.0

Step 3 : MDM Hub Configuration 

a)  Queue Configuration :

      1. The JMS queue server configuration -
         Right click on the Message Queues and select "Add Message Queue Server". Use the Connection Factory Name as siperian.mrm.jms.xaconnectionfactory. Also enable the "Data Changes Monitoring". If it is not enabled then use the button "Toggle Data Changes Monitoring Status" to change from disabled to enabled.

 Fig 1: JMS queue server configuration

    2. The JMS queue configuration -

       Right click on the connection factory configured and select "Add Message Queue". Use the Queue name which configured in  "standalone-full.xml" file. i.e. jboss/exported/queue/OutboundQueue. Select the radio button Use with message triggers. After configuration is completed, test the queue connection by using the Test button provided at the bottom of the right panel. 

 Fig 2: JMS queue configuration

  3. The JMS queue - SIF Manager configuration -

      Select the SIF Manager and press the button "Generate and Deploy ORS Specific Schema". After pressing this button, Schema Url will be populated. Check if any out of sync objects are present in the ORS by pressing "Find Out of Sync Objects" button. 

Fig 3 : JMS Queue configuration in SIF Manager

   4. Set the Triggers for the Base object : 

       Create the triggers for the base object and select the appropriate events for it.

  

Fig 4 : Trigger configuration for Base Objects

Once you have completed all of the above configuration the queue is ready for use. The message will be sent on the queue whenever the event is triggered.

How to Clean Java Cache for Informatica MDM hub application

This article elaborates how to clear the Java Cache. I have encountered several situations where the MDM Hub application is unable to launch. Sometimes it launches but it isn't possible to see the content on the Hub console. In all such cases the temporary files and the Java Cache must be removed.

Below are the three main steps which need to be performed.

A. Remove temporary files and cookies from the browser.

You can go to the internet browser setting and clear its temporary files and cookies.

B. Clear your Java cache in Windows:

In order to clear Java Cache follow these steps:

1. Below are the steps as per the operating system:

      1.1 Windows 7/Vista: Click on Start > Control Panel

      1.2 Windows 8: On the Start screen, type Control Pane and  then click the Control Panel icon

2. Select the Java icon present on the window and open it. You can locate the Java icon on the window as below: 

     2.1 Select Programs. Then click the Java icon.

     2.2 Enter Java in the search box which is available in the upper-right corner of the Control Panel window. After providing Java in the search box press the Enter button. 

    2.3 After that double-click Java 32-bit to open it.

3. Select the Settings button under Temporary Internet Files on the General tab of the dialog.

4. Select Delete Files and the Delete Temporary Files window appears. Make sure all boxes are selected with check marks and click OK.

5. Click the OK button again.

6. Click the OK button again to close the Java Control Panel window.

C. Clear Java Applications:
1. Use command 'javaws -viewer' to view the applications which are accessing Java.
2. Remove the application available on the Java Viewer.
3. Clear the browser history.
4. Restart the system.

I hope this article is helpful if you face a similar kind of issue in your MDM hub application. Below is a good video which explains the Informatica MDM Hub.