Tuesday, May 5, 2015

Mounting WSO2 ESB registry partitions into MySQL

By default WSO2 ESB comes with an embedded H2 database where the config and governance partitions of the registry is embedded there. In most of the production scenarios it is required to externalize these partitions to out of ESB for the convenience of managing registry resources. This post is a step by step guide on how to mount ESB registry partitions into a MySQL database.

1. Download MySQL JDBC connector
1. Installing MySQL
2. Create new registry database
3. Configure ESB to use externalized MySQL database

1. Download MySQL JDBC connector
The jdbc connector for MySQL can be downloaded by the link provided here. 
[https://dev.mysql.com/downloads/connector/j/5.0.html]
Extract the zip file and obtain the mysql-connector-java-5.0.8-bin.jar file.

2. Installing MySQL

There are two ways to install MySQL server in Windows 7. One is by downloading the MSI installer and the other option is downloading the mysql zip file.
I have used the option two below as it was quick. You can download the mysql ""Windows (x86, 64-bit), ZIP Archive"" using this link. 
[https://dev.mysql.com/downloads/mysql/5.5.html#downloads]

Once downloaded, extract it and go to \mysql-5.5.43-winx64\bin. Run the below command to start mysql server. This will start the MySQL service.

> mysqld.exe

After that execute below command to log in to mysql as root user. The default password is empty, so once prompted for password you can just press enter and login. If you need set a new root password later for increased security.

> mysql.exe -u root -p
Enter password: [Press Enter here as there is no password to type]
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.5.43 MySQL Community Server (GPL)


That is all, you can use any mysql commands now to work with databases here.

3. Create a new database in MySQL server, which will be used as the external database for WSO2 ESB. I have given the name reg_db for this.

mysql> create database reg_db;
Query OK, 1 row affected (0.00 sec)


4. Configure ESB to use externalized MySQL database

The last step to do is pointing ESB to use this newly created MySQL database. Here we are going to link ESB with new sql database, so the entries written to config and governance spaces will be stored in reg_db mysql database from here onwards.

4.1  Copy the previously download jdbc connector jar into ESB_HOME/repository/components/lib folder.
4.2  Goto ESB_HOME/repository/conf/datasources and open the master-datasources.xml file. Add the below entry for a new datasource. Do not remove any existing entries too.
     We have to give username and password for the database. Since my MySQL password is empty i have kept <password></password>  without any value here.

        <datasource> 
            <name>WSO2_CARBON_DB_Reg</name> 
            <description>External DB used for registry and config spaces</description> 
            <jndiConfig> 
                <name>jdbc/WSO2CarbonDB_Reg</name> 
            </jndiConfig> 
            <definition type="RDBMS"> 
                <configuration> 
                    <url>jdbc:mysql://localhost:3306/reg_db</url> 
                    <username>root</username> 
                    <password></password> 
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                    <maxActive>50</maxActive> 
                    <maxWait>60000</maxWait> 
                    <testOnBorrow>true</testOnBorrow> 
                    <validationQuery>SELECT 1</validationQuery> 
                    <validationInterval>30000</validationInterval> 
                </configuration> 
            </definition> 
        </datasource>

4.3 Goto ESB_HOME/repository/conf/registry.xml file and add a new dbConfig for remote MySQL database.

    <dbConfig name="mounted_registry"> 
        <dataSource>jdbc/WSO2CarbonDB_Reg</dataSource> 
    </dbConfig>
   
4.4 Add a new remoteInstance entry to the same registry.xml.

    <remoteInstance url="https://localhost:9443/registry"> 
        <id>instanceid</id> 
        <dbConfig>mounted_registry</dbConfig> 
        <readOnly>false</readOnly> 
        <enableCache>true</enableCache> 
        <registryRoot>/</registryRoot> 
        <cacheId>root@jdbc:mysql://localhost:3306/reg_db</cacheId> 
    </remoteInstance>   

4.5 Add the mount configurations to registry.xml as below.

    <mount path="/_system/config" overwrite="true"> 
        <instanceId>instanceid</instanceId> 
        <targetPath>/_system/nodes</targetPath> 
    </mount> 
    <mount path="/_system/governance" overwrite="true"> 
        <instanceId>instanceid</instanceId> 
        <targetPath>/_system/governance</targetPath> 
    </mount>
   
That is all. Now when you restart the ESB server, go to the Registry menu from the admin console. When you 'Browse' the registry the icons for config and governance spaces would look like below.

 



The blue arrow in the icons mean those partitions are now pointed into an external database.

Now if you goto the mysql console and try to see the tables that in the 'reg_db' database, it is now loaded with list of new tables as below.

mysql> use reg_db;
Database changed
mysql>
mysql> show tables;
+-----------------------+
| Tables_in_reg_db     |
+-----------------------+
| reg_association       |
| reg_cluster_lock      |
| reg_comment           |
| reg_content           |
| reg_content_history   |
+-----------------------+
39 rows in set (0.00 sec)

Wednesday, October 29, 2014

[WSO2 ESB] Comparing two XMLs and find matching elements using WSO2 ESB 4.8.0

Imagine we are getting a list of xml data from an external service and it is required to match this response against another xml payload and find the matching xml elements inside ESB 4.8.0. For e.g. a user send the below request into ESB.
<Accounts>
    <acctNum>123452</acctNum>
    <acctNum>678912</acctNum>
</Accounts>

Inside ESB we are calling an Account DB service which retrieves all the Account elements inside the database and return us. Assume the BE response with Account details is as follows.

<a123: Accounts xmlns:a123="http://test123.com>

    <a123:Account>
        <a123:AcctNum>123451</a123:
AcctNum>
        <a123:Date>2014-03-26</a123:
Date>
    </a123:Account>
    <a123:Account>
        <a123:AcctNum>123452</a123:
AcctNum>
        <a123:Date>2014-04-20</a123:
Date>
    </a123:Account>
</a123:Accounts>
  
 

   
Once this response comes into ESB it is required to compare the two xmls and retrieve Account elements that matches with account numbers which are in incoming request. We can do this through an iterate mediator inside ESB too, however the way that is explained here is rather simple to be used, specially for a scenario when it is not possible to use Iterate mediator.

1. When the incoming request comes, we can use xpath to get a comma separated String with all account numbers in the request. Then assign it to a Property mediator.

<property name="accountsList" expression="string-join(//Accounts/acctNum, ',')" scope="default" type="STRING"/>
This XPath expression uses string-join() function from Xpath 2.0. Therefore it is required to enable the below property in ESB/repository/conf/synapse.properties file in order to support Xpath 2.0

# Uncomment following to support fallback XPATH 2.0 support with DOM and Saxon
synapse.xpath.dom.failover.
enabled=true   
   
2. Once this property is set we can call the BE and get the Account details response. Then the response payload will be passed to the given XSLT stylesheet using XSLT mediator in ESB.
Also as there is comparison needs to be done here, i am passing the previous 'accountsList' property as a parameter into XSLT stylesheet as well.

<xslt key="MatchingAccounts">
    <property name="AccountsList" expression="$ctx:accountsList"
/>
</xslt>

   
3. The stylesheet is added as a local-entry in the ESB with the name 'MatchingAccounts'. If needed you can add this as a Registry resource too.

<localEntry xmlns="http://ws.apache.org/ns/synapse" key="MatchingAccounts">
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:a123="http://test123.com xmlns:fn="http://www.w3.org/2005/xpath-functions" version="2.0">
        <xsl:param name="AccountsList"/>
        <xsl:output method="xml" indent="yes"/>
        <xsl:template match="/">
            <Result xmlns="">
                <xsl:for-each select="//a123:Account">
                    <xsl:if test="matches($AccountsList,
a123:AcctNum)">
                        <xsl:copy-of select="."/>
                    </xsl:if>
                </xsl:for-each>
            </Result>
        </xsl:template>
</xsl:stylesheet>   
</localEntry>

   
4. Inside the style sheet if process through each a123:Account/a123:AcctNum element and checks whether the 'accountsList' String contains that account number. If there is a match that <Account> node is taken as results.

5. For the above two XMLs the final payload that comes outside XSLT mediator will be like this, with one matching Account node.
<Result xmlns="">
    <a123:Account>
        <a123:AcctNum>123452</a123:
AcctNum>
        <a123:Date>2014-04-20</a123:
Date>
    </a123:Account>
</Result>

Sunday, September 7, 2014

[WSO2 ESB] Sending Form Data through WSO2 ESB with x-www-form-urlencoded content type

This post is about how to post form data into a REST service from WSO2 ESB 4.8.1.
Imagine that we have the following key values pairs to be passed into a REST service  which accepts x-www-form-urlencoded type data.

name=ishara&company=wso2&country=srilanka

Now when we going to send these data into ESB, it is needed to set them as key values pairs through adding a PayloadFactory mediator in the following format.  

<property name="name" value="ishara" scope="default" type="STRING"/>
<property name="company" value="wso2" scope="default" type="STRING"/>/>
<property name="country" value="srilanka" scope="default" type="STRING"/>/>

            
<payloadFactory media-type="xml">
                <format>
                    <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
                        <soapenv:Body>
                            <root>
                                <name>$1</name>
                                <company>$2</company>
                                <country>$3</country>                               
                            </root>
                        </soapenv:Body>
                    </soapenv:Envelope>
                </format>
                <args>
                    <arg evaluator="xml" expression="$ctx:name"/>
                    <arg evaluator="xml" expression="$ctx:company"/>
                    <arg evaluator="xml" expression="$ctx:country"/>
               </args> 

</payloadFactory>           

Then set the messageType property as 'application/x-www-form-urlencoded'. This is how ESB can identify these key-value pairs as form data and it will do the transformations. Then it is also required to disable chunking too.
            
<property name="messageType" value="application/x-www-form-urlencoded" scope="axis2" type="STRING"/>
<property name="DISABLE_CHUNKING" value="true" scope="axis2" type="STRING"/>

                       
Now we are all set to call the REST endpoint with this message data as below. You can use either send or a call mediator.

<call>
   <endpoint key="conf:endpoints/EmployeeDataServiceEndpoint.xml"/>
</call>

                       

Sunday, August 31, 2014

[WSO2 ESB] Enrich Mediator Patterns

WSO2 ESB Enrich Mediator can process a message based on a given source configuration and then perform the specified action on the message by using the target configuration. The following post specifies some of the source/target patterns that can be used with Enrich mediator. The flow is explained in 'description' element of each mediator entry to make it easily understandable.

1. Adding a property value into message body as a sibling

<property name="failureResultProperty" scope="default" description="FailureResultProperty">
        <result xmlns="">failure</result> 

</property>
<enrich>
        <source xpath="$ctx:failureResultProperty"/>
        <target type="body" action="sibling"/> 

</enrich>   
   
2. Append new elements into a property using two enrichs

<property name="testMessages" scope="default" description="Test Messages Property">
        <
testMessages/> 
</property>
          
<property name="testMsg" expression="//testMessage" scope="default" type="OM"/>

<enrich description="Get testMessages into current msg body">
            <source type="property" property="testMessages"/>
            <target type="body"/> 

</enrich>
<enrich description="Append testMsg into testMessages property">
            <source xpath="$ctx:testMsg"/>
            <target type="body" action="child"/> 

</enrich>

<property name="testMessages" description="Set testMessages property back"  expression="//testMessages" scope="default" type="OM"/>
       
       
The final output for 'testMessages' property after enriching will be like this.

 




 
3 . Append direct to a property using single Enrich.
   
<property name="testMessages" scope="default" description="Test Messages Property">
        <
testMessages/> 
</property>
<enrich>
            <source xpath="//testMessage"/>
            <target action="child" xpath="$ctx:testMessages"/>
 </enrich>

    

[WSO2 ESB] How to Read Local Entries and Registry Entries

This post is rather a quick note to my self. In WSO2 ESB we can have external reference to a file using either a local entry or a registry entry. 

Local-entry :

An entry stored in the local registry. The local registry acts as a memory registry where you can store text strings, XML strings, and URLs. These entries can be retrieved from a mediator.

Registry-entry :
WSO2 ESB makes use of a registry to store various configurations and artifacts such as sequences and endpoints. A registry is a content store and a metadata repository. Various SOA artifacts such as services, WSDLs, and configuration files can be stored in a registry and referred to by a key, which is a path similar to a UNIX file path.

1. This is how we can read some value stores in a local-entry. Let's say i have a value stored in local-entry file 'testEntry'.

<localEntry xmlns="http://ws.apache.org/ns/synapse" key="testEntry">12345</localEntry>

 This is how we can read this value into a mediator.

<property name="testProp" expression="get-property('testEntry')" scope="default" type="STRING"/>

2. If this 'testEntry' file is stored in the registry we have to use 'registry' scope with get-property() XPath extension function and read entry as below.

<property name="testProp" expression="get-property('registry', 'conf://custom/testEntry')" scope="default" type="STRING"/>

[WSO2 ESB] Property mediator : Performing XPath Expressions with and without Namespaces

Assume that we have an XML payload where there are several namespaces defined and we need to retrieve values of some elements from the payload into Properties using XPath. There are several ways to do this. We can either define the namespace in the property mediator and then refer to XML element via XPath with adding the namespace itself into the Xpath operation. let's get started with this.

Below is our payload.

<abc  xmlns="http://abc.xyz.com" >
         <A>YES</A>
         <B>{"abc":"Y","d":"N","e":"N"}</B>
</abc>

For e.g i need to read the value 'A' into a property.  Following are the two ways for this However the 2nd option is more cleaner approach since we can get rid of adding namespace references every where. In addition to Property mediator, this can be used with other mediators in WSO2 ESB whether XPath operations are supported.

1. Use name space entries with property mediator

<property  xmlns:ns="http://abc.xyz.com" name="aValue" expression="//ns:abc/ns:A" scope="default" type="STRING"/>

2. Use local-name() option in XPath in order to avoid name spaces and get the same element value with property mediator.

<property name="aValueWithoutNS" expression="//*[local-name()='A']" scope="default" type="STRING"/>


 

[WSO2 ESB] XSLT mediator : Writing a simple style sheet

The XSLT Mediator of WSO2 ESB applies a specified XSLT transformation to a selected element of the current message payload. In this post i am going to write a simple XSLT style sheet which will read values from the current XML payload inside ESB using XPath and populate them into style sheet to create a new or different payload.

Below is our original XML payload.



We will be passing this payload into XSLT mediator with specifying a certain drink name as a parameter to the style sheet. For e.g i am passing drink name as 'Coffee'. At the style sheet it will traverse through incoming payload and find the <lunch> elements which contains 'Coffee' as drink name. If matches found we add the prices of those elements under a new <Payment> element. So when we come out of XSLT mediator the payload will be now changed to the <Payment> entry where it contains drinkPrices of matching elements.

The style sheet 'discountPayment.xsl' is like this.

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:fn="http://www.w3.org/2005/02/xpath-functions" xmlns:m0="http://services.samples" version="2.0" exclude-result-prefixes="m0 fn">
        <xsl:output method="xml" omit-xml-declaration="yes" indent="yes"/>
        <xsl:param name="drink_name"/>
        <xsl:template match="/">
            <Payment>
                <xsl:for-each select="//Order/lunch[contains(drinkName, $drink_name)]">
                    <discount>
                        <xsl:value-of select="drinkPrice"/>
                    </discount>
                </xsl:for-each>
            </Payment>
        </xsl:template>
</xsl:stylesheet>

Add this style sheet into local-entries of ESB and it can be referred from XSLT mediator as this.

After coming out from XSLT mediator now our current message payload is changed to below.