[GSoC] FOAF Co-reference based Entity Disambiguation for Apache Stanbol

It's been an exciting 1.5 months so far with Apache Stanbol in my GSoC project. I have learnt so much about how semantic content management and enhancement are done in Stanbol. Thanks to Rupert, Rafa and Andreas my mentors I have successfully passed my mid-term evaluation in GSoC 2013. This post is to give you all what I have been doing for the last 1.5 months.

Apache Stanbol is a Semantic content management management system..Enhancement Engine, Entityhub, ContentHub, Reasoners and CMS adapters are main components of Stanbol. Stanbol evolved from the 'Interactive Knowledge Stack' (IKS) research project launched to develop a technology platform for content and knowledge management in European IT organizations.

Apache Stanbol's enhancement engines are the components that actually enhance a given content by identifying entities mentioned in the content and giving entity references. When identifying entities, many semantic technologies such as natural-language-processing, part-of-speech identification and entity-linking are used. Different entities can sometimes referred by same name, same description leading to ambiguity in entity linking. This is where entity disambiguation techniques come into play. At the moment Apache Stanbol has several entity disambiguation engines, including a SolrMLT based engine and DBpedia Spotlight engine. My project aims at introducing a new disambiguation engine to use FOAF (friend-of-a-friend) data as the source and FOAF co-reference based techniques as the disambiguation technique.

At mid-term I have successfully produced 3 milestones.
1. Selection of a sufficient FOAF dataset and integrate it as a Stanbol Reference Site.
2. Configure Stanbol Entityhub engine to use the FOAF data and link entities.
3. Design a disambiguation workflow using FOAF data elements in an effective manner.

I have written down a guide to discuss : how to integrate a FOAF dataset in Apache Stanbol as a EntityhubLinking engine. Please refer my github project with all the project resources and wiki for a step-by-step guide to integrate FOAF data in Stanbol.

Research Life...

It's been a while.. :) A lot of things changed during past few months and I started a new page in my life as a researcher. I'm now a full-time research student working on my Msc by Research degree at Computer Science Department, UOM. I felt I need to have a solid research background, hence started this and hope for the best :)

My research topic is "Collaborative Reputation Management in Email Networks". The initial email architecture didn't account for reputation of the users. Maybe the inventors never thought more than 60% of email exchanged today would be spam! With time, so many research has been carried out in the  field of reputation management in email networks and associated social networks. My aim is to study such relevant projects and develop a sustainable, user-friendly and effective model for reputation management in emails. The benefits of such project will be enormous. Spam filtering, social profiles, email categorizing are few good examples. I will be writing more in coming days about the progress of my research project... Stay tuned..

By the way..
I'm also doing a Google Summer of Code project for Apache Stanbol this year. Apache Stanbol is a semantic enhancement engine for content management. It integrates so many opensource projects such as Apache Solr, Gena, Clerezza, OpenNLP under one umbrella in a component-based architecture. I find the project amazing and there are so many application areas of it. My project is about developing a "Entity Disambiguation Engine using FOAF Co-reference".  You can find my project propsal here.I will write about my adventures in this project too :)
  

Multiple P2 Profile Support in Carbon

In this blog-post I'm going to discuss about the proposed multiple P2 Profile support in Carbon. To get a better understanding of this effort, we first need to understand the basics about P2.

P2 is the provisioning framework used by WSO2 Carbon platform to install features. To get an overall understanding of P2 concepts please refer the Eclipse wiki 

The main areas to discuss here are, P2 Profile, how we generate P2 data using P2-Director and how we manage the Eclipse runtime by passing relevant configurations to start an Eclipse based product.

P2 Profile represents the state of all the installation units (IUs) that constitute an Eclipse based product. P2 Profile keeps track of all installed, uninstalled and updated IUs in the product.

In Carbon platform the IUs are features, and a Carbon product is constituted with Carbon Kernel + a set of features. 

P2 Director is the a command line tool for installing additional software or uninstalling IUs from an Eclipse-based product. This is the tool we use underneath the carbon-p2-plugin to build & provision features our products. 

Until now, Carbon products have been using a single P2 Profile:  WSO2CarbonProfile. In all our products we have been using WSO2CarbonProfile to install different features and constitute different Carbon products.

Now we will look at how we can support multiple P2 profiles in a single product distribution. This will make it possible to have multiple Profiles pre-installed in a single product and select the profile to start which will load a separate Product at runtime. 

Let me explain how we generate our product distributions using carbon-p2-plugin to have a multi-profile installation.

1. How we generate product distributions  

To see how we can generate the new structure to support multiple-profiles we should first look at how the product distributions are created currently via carbon-p2-plugin using default WSO2CarbonProfile

1. In Kernel distribution 

We first constitute the carbon product using the carbon product configuration file: carbon.product . 

carbon.product file has the initial configurations of the osgi-bundles to start and other required configurations necessary to generate the config.ini (the main configuration file for an eclipse product)

It also defines the minimal equinox runtime-feature to be installed first into the default WSO2Carbon profile.

So in the first 2 phases in carbon-p2-plugin (publish-product, materialize-product) we invoke the p2-director to use above carbon.product and materialize the default WSO2CarbonProfile with a minimal runtime.

Then in the profile-gen phase we install carbon core feature to the WSO2CarbonProfile.

2. In product distributions

We extract the carbon-core distribution and install additional features to the default WSO2CarbonProfile in profile-gen phase.

So in the final output the target, the product distribution structure with a single Profile looks like this:

When supporting multiple-profiles, we have to first do the process of materializing the a Profile, before installing features to it.

This can be achieved using the same carbon.product configuration file as all our products use the same minimal runtime to install features to the Profile afterwards. 

Which means for each new P2 Profile we will have to to do the initial materializing part using carbon.product configuration file and install features on top of it.

For this, we'll have to repeat the materialize-product and p2-profile-gen goals for each Profile we generate. 

Please refer this pom in the POC done. Here in the p2-profile-gen module of the carbon product I'm creating 2 profiles. 

WSO2CarbonProfile and WSO2SampleProfile.

So the new product distribution structure now looks like below; 

Each profile installed will have a separate configuration directory, and all the Profiles will share the same P2 data-area to store P2 data.

Now let's look at how we can generate above structure using the carbon-p2-plugin. Underneath the p2-plugin we invoke the p2-director to generate above structure.

We can do this by passing below arguments to generate separate profile configuration folder (-destination) and share the same p2 data location (-shared) by all the Profiles.

Arguments to invoke P2-director;

-metadataRepository : metadataRepository.toExternalForm() 

-artifactRepository : metadataRepository.toExternalForm()

-installIU : installIUs

-profileProperties :  org.eclipse.update.install.features=true 

-destination : ${destination}/${profile}

-bundlepool : ${destination}

-shared : ${destination}/p2

-profile :  ${profile}

-roaming

To have a clear understanding of what each of above argument is used for, please refer the p2-director documentation.

eg: 

-nosplash -application org.eclipse.equinox.p2.director -metadataRepository file:/dileepa/kernel/trunk/distribution/product/modules/p2-profile-gen/target/p2-repo -artifactRepository file:/dileepa/kernel/trunk/distribution/product/modules/p2-profile-gen/target/p2-repo -profileProperties org.eclipse.update.install.features=true -installIU org.wso2.carbon.core.feature.group/4.1.0.SNAPSHOT,org.wso2.carbon.tryit.feature.group/4.1.0.SNAPSHOT,org.wso2.carbon.soaptracer.feature.group/4.1.0.SNAPSHOT,org.wso2.carbon.styles.feature.group/4.1.0.SNAPSHOT, -bundlepool /dileepa/kernel/trunk/distribution/product/modules/p2-profile-gen/target/wso2carbon-core-4.1.0-SNAPSHOT/repository/components -shared /dileepa/kernel/trunk/distribution/product/modules/p2-profile-gen/target/wso2carbon-core-4.1.0-SNAPSHOT/repository/components/p2 -destination /dileepa/kernel/trunk/distribution/product/modules/p2-profile-gen/target/wso2carbon-core-4.1.0-SNAPSHOT/repository/components/WSO2SampleProfile -profile WSO2SampleProfile -roaming

In the POC done for this, in carbon-p2-plugin I have modified the MaterialzeProductMojo and ProfileGenMojo classes by passing above arguments to the p2-director to support multiple-profile installations with a shared p2 area.

What I basically did was, changing the values for the below parameters in addArguments() method;

"-destination", destination + File.separator + profile

"-shared" , destination + File.separator + "p2"

With -destination set like above, a separate directory will be created to hold the Eclipse configuration directory for the Profile(eg: ${carbon.home}/respository/components/WSO2CarbonProfile/configuration, ${carbon.home}/respository/components/WSO2SampleProfile/configuration)

And in the separate configuration folder we will have the config.ini and org.eclipse.equinox.simpleconfigurator/bundles.info file which has the list of all the bundles to start, for this particular runtime represented by the P2 profile. 

Note: 
Since we are changing the Product distribution structure by changing install-root to point to /repository/components/${Profile}, the relative-paths for copying files during feature installation should be changed accordingly in the Platform features. We copy various configuration files during feature installations, and this this done using P2 touchpoints instructions (org.eclipse.equinox.p2.touchpoint.natives.copy touchpoint is used here).
In Carbon features we have a p2.inf file to define these instructions and the new relative paths from install directory should be changed as below;

eg:
org.eclipse.equinox.p2.touchpoint.natives.copy(source:${installFolder}/../features/org.wso2.carbon.logging.mgt.server_${feature.version}/conf/logging-config.xml,target:${installFolder}/../../conf/etc/logging-config.xml,overwrite:true);  

After making changes to features which copy files at installation and creating the product distribution like above the next thing to look at is, how we are going to launch the Equinox runtime to point to different P2 profiles, at Carbon startup.

2. How to manage the Multi-Profile Carbon runtime

Equinox finds the necessary files and directory locations to start and load the Equinox runtim using some parameters.

Some of the important parameters are below; 

1. osgi.install.area  (the root of a Eclipse product installation)

2. osgi.configuration.area (the configuration folder of an Eclipse product)

3. eclipse.p2.data.area (the P2 data area of the Profile)

So when we select a particular Profile, at startup by passing a parameter (eg: -Dprofile=WSO2SampleProfile), we need to pass above parameters based selected Profile.  Basically we need to point Equinox to the relevant Profile root and configuration directories at startup. And if a profile is not given at startup, Carbon should start with the default WSO2CarbonProfile. 

Please note that the default WSO2CarbonProfile is passed as a system property at carbon.server.Main class if no profile is given at startup. 

So the changing parameter values are

osgi.install.area : ${carbon.home}/respository/components/${profile}

osgi.configuration.area :  ${carbon.home}/respository/components/${profile}/configuration

Please see how these parameters are passed in the POC's modified carbon.server CarbonLauncher class.

So with above changes we can successfully start different Carbon profiles from a pre-installed multi-profile distribution.   

Things to improve and fix 

1. Feature installation with copying resources at runtime

The feature installation at runtime have issues with features having various files to copy, during installation. The problem is again with relative-paths to find /features directory.

If we generate profiles with -roaming enabled (which allows a Eclipse product to be moved) at runtime P2 updates the bundle-pool location to point to the same directory as the p2.installFolder.  Because of this, when we install features at runtime, /features and /plugins directories will be created inside ${carbon.home}/repository/components/${profile}

In P2 the location of the installFolder is defined by org.eclipse.equinox.p2.installFolder and the location of the bundle.pool is calculated using org.eclipse.equinox.p2.cache property. At runtime P2 engine updates the p2.cache property to point to the installFolder and the /features and /plugins are extracted there. 

eg: 

At build-time when we invoke p2-director the bundle-pool location is created at ${carbon.home}/repository/components/features

${carbon.home}/repository/components/plugins

But at runtime when we install  features it will be extracted under the installFolder as below;

${carbon.home}/repository/components/WSO2SampleProfile/features

${carbon.home}/repository/components/WSO2SampleProfile/plugins

So the same file-paths we changed in p2.inf above will be invalid and feature installation will fail during copy instruction execution.

To fix this issue, we need to either find a way to override the p2.cache location at runtime to point to the bundle-pool at ${carbon.home}/repository/components/features, /plugins

Or use some other way to derive the file-path in the p2.inf to copy files during feature installations.

2. Improve carbon-p2-plugin to a have a new goal to merge product materializing and profile-gen goals so that new Profiles can be generated in single execution without iterating the process of materializing the product and installing features afterwards.

Running Carbon 4.0.2 in Geronimo as a web-application

In this post I'll explain how to deploy Carbon 4.0.2 as a web-application in Apache Geronimo V3. I'm using the Geronimo J2EE distribution with Tomcat 7 integration for Linux platform downloaded from here. 

Please Note: 

WSO2 Carbon platform is a fully fledged cloud-native middleware platform with it's own set of complexities which are greater than a typical web-application or other deployable artifact.

Running WSO2 products in web-app mode or any other deployable form in 3rd party AppServers can greatly limit the capabilities of the the platform and can introduce new complexities when combined with the host AppServer's environment and complexities.
Therefore running WSO2 Products in other AppServers is not encouraged and subject to specific support conditions only.

The Steps 

The basic steps are similar to what is explained by Pradeep in his blog on running Carbon 4.0 as a web-app in Apache Tomcat. So I would suggest reading his blog to understand the basics on running Carbon based products in web-app-mode on any other Web Server.

Here I will explain the steps to deploy Carbon web-app in Geronimo and explain what are the tweaks and changes we need to do in Geronimo to run Carbon web-app.

1. Download Carbon 4.0.2 from the previous releases page here

2. Extract carbon-4.0.2 in the file-system. 
The extracted directory will be considered $CARBON_HOME from here onwards.

3. The web-app directory for Carbon is in $CARBON_HOME/webapp-mode directory. Inside it you will find the WEB-INF folder which contains the web-application descriptors and required libraries to run Carbon web-application. The structure of the WEB-INF directory is as below;


WEB-INF/
├── eclipse
│   └── launch.ini
├── lib
│   └── org.wso2.carbon.bridge-4.0.0.jar
└── web.xml

The web.xml is the web-application descriptor that includes details about the Carbon web-app. Please note that the web-app ID will be taken as the web-context root if you don't include a geronimo-web.xml descriptor specifying the Web-Context information, under your web-application. So in this case the Carbon web-app ID is : WebApp

4. Create a .war archive file with above content in WEB-INF directory so that we can deploy it in Geronimo. I'm creating the archive as carbon.war.

eg:  jar -cvf carbon.war WEB-INF/

5. Now we need to configure the carbon-repository to be used by our carbon web-application. For this create a directory named carbon_repo and copy the repository sub-directory under $CARBON_HOME, to carbon_repo directory.

We'll call this directory $CARBON_REPO from here onwards.
For me this location is at: /home/dileepa/work/carbon_repo

 carbon_repo/

                          └── repository

                                        ├── components

                                        ├── conf

                                        ├── data

                                        ├── database

                                        ├── deployment

                                        ├── logs

                                        ├── README.txt

                                        ├── resources

                                        └── tenants

Now;

• Remove all the tomcat related jars from the  '$CARBON_REPO/repository/components/plugins' directory.  As Geronimo provides Tomcat servlet container by default we don't need these bundles in runtime.

           org.wso2.carbon.tomcat_4.0.1.jar 

           org.wso2.carbon.tomcat.ext_4.0.2.jar

           org.wso2.carbon.tomcat.fragment.dummy_4.0.0.jar

           org.wso2.carbon.tomcat.patch_4.0.1.jar

           tomcat-ha_7.0.28.wso2v1.jar

           tomcat_7.0.28.wso2v1.jar

• Copy the jars found under $CARBON_HOME/webapp-mode/bundles to '$CARBON_REPO/repository/components/plugins'  directory. namely org.wso2.carbon.http.bridge-4.0.0.jar and org.wso2.carbon.servletbridge-4.0.0.jar

• Now we need to configure the bundles.info to add these 2 new bundles in the runtime. Add the below 2 lines to $CARBON_REPO/repository/components/configuration/org.eclipse.equinox.simpleconfigurator/bundles.info

org.wso2.carbon.http.bridge,4.0.0,plugins/org.wso2.carbon.http.bridge-4.0.0.jar,4,true org.wso2.carbon.servletbridge,4.0.0,plugins/org.wso2.carbon.servletbridge-4.0.0.jar,4,true

6. We now need to setup the HTTPS connector configuration for tomcat in Geronimo to handle connections to carbon web-app.

To do this; add a HTTPS Connector configuration to $GERONIMO_HOME/var/catalina/server.xml

I'm configuring a new HTTPS connector for port : 8445 in Geronimo's tomcat server using the default carbon-keystore as below;

eg:

Then configure Axis2 transports in $CARBON_REPO/repository/conf/axis2.xml

    8080

 

    8445

Now configure the serverURL and the web-context root (the web-context root for our scenario is /WebApp) in carbon.xml

/WebApp
https://localhost:8445/WebApp/services/

Also change the JMX ports in carbon.xml to some other accepted port number to avoid address conflicts with host Geronimo's RMI ports.

            
            9998
            
            11112

7. Endorsing JVM provided classes
Copy the endorsed lib jars found under $CARBON_HOME/lib/endorsed to $GERONIMO_HOME/lib/endorsed to provide the endorsed libraries required for XML parsing/API etc. 

8. Configuring the Carbon Datasource in Geronimo.

We need to configure a Carbon datasource in Geronimo for the Registry and User Manager in Carbon. For this purpose please follow the blog-post on how to create an embedded-derby Carbon datasource in Geronimo. Similarly you can create the datasource using any database type and configure the master-datasources.xml to use it.

Now use the JNDI name and the URL of the created datasource with the relevant database driver in $CARBON_REPO/repository/conf/datasources/master-datasources.xml

eg: 

            WSO2_CARBON_DB
            The datasource used for registry and user manager
            
                jca:/console.dbpool/WSO2CarbonDB/JCAConnectionManager/WSO2CarbonDB
            
            
                
                    jdbc:derby:/home/dileepa/work/geronimo-tomcat7-javaee6-web-3.0.0/var/derby/WSO2_CARBON_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000
                    wso2carbon
                    wso2carbon
                    org.apache.derby.jdbc.EmbeddedDriver
                    50
                    60000
                    true
                    SELECT 1
                    30000
                
            
        

To support embedded derby database connection, you  need to add the derby embedded driver bundle to $CARBON_REPO/repository/components/plugins and add below entry to bundles.info.

derby,1.0.0,plugins/derby_1.0.0.jar,4,true
You can download the derby_1.0.0 bundle from the hosted location here.

To enable Carbon Logging, you need to make the log4j.property file available for carbon logging bundle in $CARBON_REPO/repository/components/plugins. In standalone Carbon 4 based products, this is achieved by bundling the log4j property file as a fragment bundle and deploying it on the fly as from dropins. But as we are deploying the carbon runtime in bridge mode we cannot install dropins, so we have to manually copy it to $CARBON_REPO/repository/components/plugins directory and add the relevant entry in the bundles.info as below;

org.wso2.carbon.logging.propfile,1.0.0,plugins/org.wso2.carbon.logging.propfile_1.0.0.jar,4,true 
You can download the org.wso2.carbon.logging.propfile_1.0.0.jar bundle from the hosted location here.

9. Now configure $CARBON_REPO/repository/conf/user-mgt.xml and registry.xml using the above created datasource JNDI name.

• In user-mgt.xml change the datasource property as below;

<Property name="dataSource">jca:/console.dbpool/WSO2CarbonDB/JCAConnectionManager/WSO2CarbonDB</Property>

• In registry.xml change the dataSource as below;

 <dataSource>jca:/console.dbpool/WSO2CarbonDB/JCAConnectionManager/WSO2CarbonDB</dataSource>

10. Now the $CARBON_REPO is configured. 
You now need to set the carbon.home system property required to run Carbon. 
Set the carbon.home system property in  $GERONIMO_HOME/etc/system.properties as below;

carbon.home=/home/dileepa/work/carbon_repo

10. Now the environment is set for deploying carbon web-app in Geronimo. So start Geronimo server by executing below command;

$GERONIMO_HOME/bin$ ./geronimo run

To deploy the carbon.war we created in step 4, deploy the carbon.war in Geronimo server by going to the Deployer section in console and giving the carbon.war archive to install.


Now when deploying carbon.war if you see below error it's due to a aries-jndi bug in 0.3.0 version discussed in ARIES-554. 

ERROR [DataSourceRepository] Error in registering data source: WSO2_CARBON_DB - Error in creating JNDI subcontext 'javax.naming.InitialContext@2702c693/jca:: Unable to determine caller's BundleContext
org.wso2.carbon.ndatasource.common.DataSourceException: Error in creating JNDI subcontext 'javax.naming.InitialContext@2702c693/jca:: Unable to determine caller's BundleContext
 at org.wso2.carbon.ndatasource.core.DataSourceRepository.checkAndCreateJNDISubContexts(DataSourceRepository.java:232)
 at org.wso2.carbon.ndatasource.core.DataSourceRepository.registerJNDI(DataSourceRepository.java:257)
 at org.wso2.carbon.ndatasource.core.DataSourceRepository.registerDataSource(DataSourceRepository.java:360)
 at org.wso2.carbon.ndatasource.core.DataSourceRepository.addDataSource(DataSourceRepository.java:474)
 at org.wso2.carbon.ndatasource.core.DataSourceManager.initSystemDataSource(DataSourceManager.java:182)
 at org.wso2.carbon.ndatasource.core.DataSourceManager.initSystemDataSources(DataSourceManager.java:154)
 at org.wso2.carbon.ndatasource.core.internal.DataSourceServiceComponent.initSystemDataSources(DataSourceServiceComponent.java:166)
 at org.wso2.carbon.ndatasource.core.internal.DataSourceServiceComponent.setSecretCallbackHandlerService(DataSourceServiceComponent.java:152)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Geronimo V3 comes with aries-jndi-0.3.0 installed.


To fix the JNDI issue you need to update the aries-jndi bundle and it's dependencies.
For that please follow below steps;

• Go to OSGI bundles section in console and search for aries.jndi bundle;

• Now you need to uninstall this bundle and install the latest available org.apache.aries.jndi.1.0.0 and it's required dependencies: org.apache.aries.util.1.0.0 and org.apache.aries.proxy.1.0.0. 

• You can download these aries bundles from the official Aries download site.After installing the 3 new aries bundles and starting them you can rectify the JNDI error you got before.

11. Now you can re-start the Geronimo server and access the Carbon management console at : https://localhost:8445/WebApp/carbon/

You can telnet to the osgi-console of Carbon server as below;
telnet localhost 19444

Deploying Carbon bundles in Geronimo

In this post I'll explain the steps to test deploying WSO2 Carbon bundles in Apache Geronimo V3 server. Before deploying the bundles we need to setup the configuration environment required by the Carbon bundles. 
Please note that deploying Carbon bundles in Geronimo has several bundle resolution issues which need to be manually identified and rectified in order to setup all Carbon bundles to function properly. 

For this effort, I'm using the Geronimo J2EE distribution with Tomcat 7 integration for Linux platform downloaded from here. 
I'm using carbon-4.0.2 bundles for the deployment. You can download carbon-4.0.2 distribution from previous releases found here. 

1. Create CARBON_HOME directory under Geronimo home. Copy the repository directory in the carbon distribution to Geronimo home. I have highlighted the important sub-directories and configuration files for our use-case.  

Mainly we need to focus on datasources/master-datasources.xml, registry.xml and user-mgt.xml to configure the Carbon datasource and components/plugins directory to point Geronimo to find the bundle set to deploy.
eg:
GERONIMO_HOME
 |
 |── CARBON_HOME
│   └── repository│    
│       │   └── conf
│       │       ├── axis2
│       │       ├── carbon.xml
│       │       ├── datasources
│       │       │   └── master-datasources.xml
│       │       ├── etc
│       │       ├── log4j.properties
│       │       ├── osgi-debug.options
│       │       ├── README
│       │       ├── registry.xml
│       │       ├── security
│       │       ├── tomcat
│       │       │   ├── carbon
│       │       │   │   ├── META-INF
│       │       │   │   │   └── context.xml
│       │       │   │   └── WEB-INF
│       │       │   │       └── web.xml
│       │       │   ├── catalina-server.xml
│       │       │   ├── tomcat-users.xml
│       │       │   └── web.xml
│       │       └── user-mgt.xml
│       ├── carbonapps
│       ├── components
│       │   ├── dropins
│       │   ├── extensions
│       │   ├── features
│       │   ├── lib
│       │   ├── p2
│       │   ├── patches
│       │   ├── plugins

The repository directory includes all the main artifacts, bundles and configurations for Carbon.


2. Remove the tomcat related bundles from $GERONIMO_HOME/CARBON_HOME/repository/components/plugins as Geronimo provides tomcat bundle by default; 
(remove below bundles)
  org.wso2.carbon.tomcat_4.0.1.jar 
           org.wso2.carbon.tomcat.ext_4.0.2.jar
           org.wso2.carbon.tomcat.fragment.dummy_4.0.0.jar
           org.wso2.carbon.tomcat.patch_4.0.1.jar 
           tomcat_7.0.28.wso2v1.jar
           tomcat-ha_7.0.28.wso2v1.jar

Also remove org.eclipse.osgi_3.7.0.v20110613 bundle as Geronimo provides the bundle by default.


3. Create a database for WSO2 UserManager and Registry and configure WSO2Carbon DataSource in $GERONIMO_HOME/CARBON_HOME/repository/conf/datasources/master-datasources.xml.

To create the database you can use the db-scripts from CARBON_HOME/dbscripts.
For more details on this, you can refer this blog post on configuring an embedded Derby Carbon datasource in Geronimo and how to configure user-manager and registry to use it.

4. Configure carbon.home system.property required by Carbon in $GERONIMO_HOME/etc/system.properties. 
carbon.home=${karaf.base}/CARBON_HOME

5. Setup Carbon bundle installation location in Geronimo by configuring $GERONIMO_HOME/etc/hotbundles-config/org.apache.felix.fileinstall-deploy.cfg to point to CARBON_HOME/repository/components/plugins to install Carbon bundles at startup.
felix.fileinstall.dir=${karaf.base}/CARBON_HOME/repository/components/plugins

6. Optionally you can modify Geronimo server's log4j configuration to support Carbon logging by adding the configurations to $GERONIMO_HOME/var/log/server-log4j.properties.
You can find the Carbon log4j configurations in :$CARBON_HOME/repository/conf/log4j.properties

An example configuration addition is as below;

log4j.rootLogger=INFO, CONSOLE, FILE, CARBON_CONSOLE, CARBON_LOGFILE

# CARBON_CONSOLE is set to be a ConsoleAppender using a PatternLayout.
log4j.appender.CARBON_CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CARBON_CONSOLE.layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout
# ConversionPattern will be overridden by the configuration setting in the DB
log4j.appender.CARBON_CONSOLE.layout.ConversionPattern=[%d] %P%5p {%c} - %x %m%n
log4j.appender.CARBON_CONSOLE.layout.TenantPattern=%U%@%D[%T]
log4j.appender.CARBON_CONSOLE.threshold=DEBUG


# CARBON_LOGFILE is set to be a DailyRollingFileAppender using a PatternLayout.
log4j.appender.CARBON_LOGFILE=org.apache.log4j.DailyRollingFileAppender
# Log file will be overridden by the configuration setting in the DB
# This path should be relative to WSO2 Carbon Home
log4j.appender.CARBON_LOGFILE.File=${carbon.home}/repository/logs/${instance.log}/wso2carbon${instance.log}.log
log4j.appender.CARBON_LOGFILE.Append=true
log4j.appender.CARBON_LOGFILE.layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout
# ConversionPattern will be overridden by the configuration setting in the DB
log4j.appender.CARBON_LOGFILE.layout.ConversionPattern=TID: [%T] [%S] [%d] %P%5p {%c} - %x %m {%c}%n
log4j.appender.CARBON_LOGFILE.layout.TenantPattern=%U%@%D [%T] [%S]
log4j.appender.CARBON_LOGFILE.threshold=DEBUG




With above given minimal configurations you can test deploying Carbon bundles in Geronimo server. Please note that there are several bundle resolution issues which avoids complete startup of Carbon. 

For example some of the problems faced are: 

• When using Geronimo v3, some of our bundles have import package resolution issue for javax.activation package. Some of our bundles (eg: jaxb, abdera, org.wso2.carbon.core.common, org.wso2.carbon.user.mgt.common) require package javax.activation with version [0.0.0,1.0.0) but Geronimo Spec system bundle in Geronimo provides javax.activation;version=1.1.0.  
• We either have to override the export package version of javax.activation package to support our bundles import version range or change our bundle set not to bind to version range [0.0.0,1.0.0) for javax.activation package.
• The bundle org.eclipse.equinox.util_1.0.300.v20110502 in plugins doesn't get resolved as it's  Missing Constraint: Bundle-RequiredExecutionEnvironment: OSGi/Minimum-1.1 in the runtime.  To resolve thisn change the OSGi execution environment property to OSGi/Minimum-1.1 in $GERONIMO_HOME/etc/config.properties
• Carbon bundles require osgi.http.service to register servlets. Need a bundle which exposes osgi.http.service in the container for Carbon bundles to use. (eg: we can use org.eclipse.equinox.http bundle to provide osgi http service)
• Noticed that the org.eclipse.equinox.http.helper_1.0.0.qualifier.jar doesn't get picked up automatically from plugins, because of this org.wso2.carbon.ui bundle doesn't get started. Therefore install org.eclipse.equinox.http.helper_1.0.0.qualifier.jar manually from the Geronimo console and start org.wso2.carbon.ui bundle
• Class loading issues for org.wso2.carbon.core package

These issues require further investigations and experiments inorder to resolve completely.

Configuring a Carbon Datasource in Geronimo

In this post I will explain how create an embedded Derby database and configure a Carbon Datasource in Apache Geronimo. This configured datasource is useful if you are deploying Carbon bundles or running Carbon in web-app mode under restricted conditions in Geronimo. 

For this purpose I'm using the Geronimo J2EE distribution with Tomcat 7 integration for Linux platform downloaded from here. 

We can use the created database in Geronimo to configure the Carbon master datasource and use it to for UserManager and Registry in Carbon.


1. In Geronimo console go to "Datasources" and click the Using the Geronimo database pool wizard link, and complete the following steps:

Step 1: 

Select Name and Database, enter WSO2CarbonDB for the Name of Database Pool field. Select Derby embedded from the Database Type pull-down menu, and click Next

Step 2: 

Select Driver, JAR, Parameters, select org.apache.geronimo.configs/system-database/3.0.0/car from the Driver JAR pull-down menu. 

Configure the database with required configurations;

Sample configuration:

Database Name :  WSO2_CARBON_DB
Username : wso2carbon
Password : wso2carbon

Step 3: 

After creating the Database pool, run the derby.sql using the created WSO2CarbonDB pool as the datasource. The db scripts to create the Registry and User Manager Database can be found in any Carbon product at $CARBON_HOME/dbscripts directory.

Step 4: 

Now get the JNDI object name for the created database and use it to configure the WSO2_CARBON_DB datasource in $CARBON_HOME/repository/conf/datasources/master-datasource.xml
Also change the database url to point to the newly created Database location.

eg: 

            WSO2_CARBON_DB
            The datasource used for registry and user manager
            
                jca:/console.dbpool/WSO2CarbonDB/JCAConnectionManager/WSO2CarbonDB
            
             
                
   jdbc:derby:/home/test/geronimo-tomcat7-javaee6-web-3.0.0/var/derby/WSO2_CARBON_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000
                    wso2carbon
                    wso2carbon
      org.apache.derby.jdbc.EmbeddedDriver
                    50
                    60000
                    true
                    SELECT 1
                    30000
                
            

Now you can use this datasource in user-mgt.xml and registry.xml in $CARBON_HOME/repository/conf as below. This is how we configure Registry and User Manager to use a database referred by a Carbon datasource.

1. In user-mgt.xml

                admin
                
                     admin
                     admin
                
            everyone 
     jca:/console.dbpool/jdbc_WSO2CarbonDB/JCAConnectionManager/jdbc/WSO2CarbonDB
            org.wso2.carbon.user.core.config.multitenancy.SimpleRealmConfigBuilder

2. In registry.xml

 jca:/console.dbpool/jdbc_WSO2CarbonDB/JCAConnectionManager/jdbc/WSO2CarbonDB