TechnoGems Tech Notes

Making https connection from Android client if your CA provider is not trusted by Android

noreply@blogger.com (jean) — Tue, 16 Oct 2012 06:04:00 +0000

Android does not recognize all the SSL certificate issuers. If your certificate happens to be issues by one of them or you have a self signed certificate and you want your android application to talk to the server, the code can be a bit tricky. Android requires a special version of the keystore. Here are 5 steps you need to perform to make it work.

1) Download the correct version of bouncycastle. For clients running API level 8, version 1.46 works fine. It can be downloaded from http://ftp.uasw.edu/pub/security/bouncycastle/release1.46/bcmail-jdk13-146.jar

2) Extract the certificate from the server using the following command:

echo | openssl s_client -connect server.name.com:443 2>&1 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > mycert.pem

3) Run the following command to create the keystore

keytool -importcert -v -trustcacerts -file mycert.pem \
-alias server.name.com -keystore mykeystore.bks \
-provider org.bouncycastle.jce.provider.BouncyCastleProvider \
-providerpath /path/to/your/Download/bcprov-jdk15on-146.jar \
-storetype BKS -storepass mypass

4) Copy the file to the Android resource directory under the name res/raw/mykeystore.bks

5) Add the following code snippet in your application and have it executed before you make the https url connection.

    public void setDefaultSSL () {
        Context con = getApplicationContext() ;
        TrustManagerFactory tmf;
        try {
            tmf = TrustManagerFactory.getInstance("X509");
            KeyStore ks = KeyStore.getInstance("BKS");
            InputStream in = con.getResources().openRawResource(R.raw.mykeystore);
            ks.load(in, "mypass".toCharArray());
            in.close();
            tmf.init(ks);
            SSLContext context = SSLContext.getInstance("TLS");
            context.init(null, tmf.getTrustManagers(), null);
            HttpsURLConnection.setDefaultSSLSocketFactory(context.getSocketFactory());
            Log.d("HTTPS", "Setting custom trust store");
        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

Once you have invoked the above code segment, you can make url connection as you normally do and you should be fine. Here is a code snippet to do that. One point to note in this code below is that the keepAlive header. Without this, you may experience intermittent issues where the response from the call is empty.

    private String getServerContent(String url_str ) throws Exception {
        System.setProperty("http.keepAlive", "false");
        URL url = new URL(url_str);
        HttpsURLConnection urlConnection = (HttpsURLConnection) url
                .openConnection();
        urlConnection.setDoInput(true);
        InputStream ins = urlConnection.getInputStream();
        BufferedReader reader = new BufferedReader(new InputStreamReader(
                ins, "iso-8859-1"), 8);
        StringBuilder sb = new StringBuilder();
        String line = null;
        while ((line = reader.readLine()) != null) {
            sb.append(line + "\n");
        }
        ins.close();
        urlConnection.disconnect();

        return sb.toString();

    }

Weblogic disabling freemark debug messages

noreply@blogger.com (jean) — Tue, 04 Sep 2012 21:11:00 +0000

Recently we migrated our applications to Oracle weblogic server (OAS 11g) and we noticed that the application log file is filled with debug messages from freemark like the one below:
<Notice> <Stdout> <<BEA-000000> <DEBUG 13467 [freemark] ():
I searched for options to turn off freemark using the logger option and it did not work. The reason could be that the application is not using and logging configuration (no log4j or jdk logging) and oracle diagnostic logging is picking up the logs.

We also have stdout redirecting to the log file. We have the logger configuration set to debug level as it is the development environment. However, I do not want the freemarker logging to fill up the log file. The solution I ended up implementing was to create a log filter to disable the freemarker log messages. I am not sure if this is the best way or not, but this is the only way I could turn off the freemarker debug message and leave the rest of the debug messages on.

Here is the log filter configuration. The create a log filter click on the domain name on the weblogic console and go to Configuration->LogFilters

Here is the log file configuration that I used in the development environment. (For production environment, I would recommend setting the log levels to Warning so the log files are not clogged up with debug messages)

Cross Platform Mobile Application Development Framework Comparision Chart

noreply@blogger.com (jean) — Fri, 11 Nov 2011 03:14:00 +0000

Corona	Corona lets developers use integrated Lua, layered on top of Objective-C, to build graphically rich applications that are also lightweight in size and quick in development time.	Wikipedia Link Developer Link
Phone Gap (Adobe)	The mobile framework allows web developers to natively target all smartphone with a single codebase (JavaScript, HTML and CSS) by enabling a Foreign Function Interface (FFI) to an embedded WebView or Webkit on the device.	Wikipedia Link Developer Link
Rhomobile (Motorola Solutions company)	Rhodes is a framework for building native applications that can run on a variety of smartphones. Rhodes uses a Model View Controller pattern. Views are written in HTML	Wikipedia Link Developer Link
Titanium Mobile	Appcelerator Titanium Mobile is a web based application framework solutions allowing web developers to apply existing skills to create native applications for iPhone and Android using the familiar JavaScript syntax. Developers will also have to learn the extensive Titanium API. Wikipedia notes that the term cross-compiler is misleading as the titanium engine interprets the code during run time.	Wikipedia Link Developer Link

Using PHPMailer and Gmail to send email

noreply@blogger.com (jean) — Tue, 11 Oct 2011 11:33:00 +0000

If you are developing applications using PHP and you need to send email you can use the PHPMailer() class in PHP. Using a publicly available SMTP server to send the email is much easier than trying to setup your own email server. The following code snippet shows the various settings for the mailer.
The code assumes that you have PHP 5.x version and you have class.phpmailer.php file in the include directory.
Google uses ssl for the smtp connection. In order for this example to work with google smtp server, you need to enable ssl in your php.ini file by adding a line that says extension=php_openssl.dll

If you are not sure of the exact location of the php.ini file and you are using xampp, you can find the location of the php.ini file by navigating to http://localhost/xampp/phpinfo.php on your browser and look for the text "Loaded Configuration File". Once you find the file, edit it and look for the text "extension=php_openssl.dll". If the text is not found in your file, add a new line at the end of the file with the above text.

IsSMTP();
$mail->SMTPDebug = 1; // 1 tells it to display SMTP errors and messages, 0 turns off all errors and messages, 2 prints messages only.

$mail->Host = "ssl://smtp.gmail.com"; // specify main and backup server
$mail->Port = 465; // set the port to use
$mail->SMTPAuth = true; // turn on SMTP authentication

$mail->Username = 'user@gmail.com'; // replace this with your email acct
$mail->Password = 'userPassword'; // replace this with your password

$mail->From = 'jmeslie@gmail.com';
$mail->FromName = 'Jean Meslie';
$mail->AddAddress('receipient@yahoo.com', 'Receiver');
$mail->AddReplyTo('user@gmail.com'); // Adds a “Reply-to' address. Un-comment this to use it.
$mail->Subject = 'test message';
$mail->Body = 'message body goes here. This message was sent at '. time();

if ($mail->Send() == true) {
echo 'The message has been sent at '. time();
}
else {
echo 'The email message has NOT been sent for some reason. Please try again later.';
echo 'Mailer error: ' . $mail->ErrorInfo;
}
?>

Recreate GRANTS to user in oracle without dba privilege

noreply@blogger.com (jean) — Tue, 04 Oct 2011 02:00:00 +0000

If you want to get list of grants made to tables and you want to recreate the user permissions in a different environment, the easy way to generate a script is to use the dbms_metadata.get_granted_ddl function as described here. However, if you do not have DBA privilege on the database you will not be able to use that method. Here is a simple sql that you can use to generate a script that will work in most common situations. This just relies on the TABLE_PRIVILEGES table.

select replace('GRANT ' || decode(select_priv,'Y','SELECT','') || decode(insert_priv,'A',',INSERT','') || decode(delete_priv,'Y',',DELETE','') || decode(update_priv,'Y',',UPDATE','') || decode(references_priv,'Y',',REFERENCES','') || decode(alter_priv,'Y',',ALTER','') ||' ON '|| owner || '.' || table_name || ' TO ' || GRANTEE ||';','GRANT ,','GRANT ') from TABLE_PRIVILEGES where owner= 'OWNER' order by table_name, grantee;

window.onload being used in multiple places within the same application included

noreply@blogger.com (Anita Selwyn) — Tue, 27 Sep 2011 18:05:00 +0000

Recently one of my team members ran into an issue where we had used
the window.onload to dynamically set some values in a cookie object.
Our application handles multiple javascript files and we had used
window.onload in another jsp also for another reason.

So,when multiple jsp files are trying to add different functions to the window.onload event, only the last function was executing. We developed a workaround by adding the function that chains the onload event functions. Below snippet gives an example of how we solved it.

-
function addLoadEvent(functionName) {
var firstonload = window.onload;

if (typeof window.onload != 'function') {
window.onload = functionName;
}
else {
window.onload = function() {
if (firstonload ) {
firstonload();
}
functionName();
}
}
}

and the actual function definition is in the original jsp and this
function is called from there. So the first time window.onload may
not be a function so we are setting it, On subsequent assignments,
window.onload exists, so whatever it holds will be wrapped up with the
new function. the list grows like that...

Struts2 navigate away from error pages

noreply@blogger.com (jean) — Tue, 07 Jun 2011 05:00:00 +0000

Struts 2 provides a nice workflow interceptor that makes sure there are no validation errors before allowing the interceptor chain to continue.

This will also prevent the user from navigating away to a different method in the same action. Sometimes that is not the desired behavior we like. For example, if the user is editing an item and there is a validation error and the user does not want to fix the validation error, however they choose to navigate to a different page on the same action, the workflow interceptor will prevent the user and put them back to the edit page.

If the desired behavior is to let them proceed to to another page, you can exclude those methods in the interceptor configuration similar to the following example.

<interceptor-ref name="defaultLoginStack" >
         <param name="validation.excludeMethods">doInput,doList</param>
         <param name="workflow.excludeMethods">doInput,doList</param>
</interceptor-ref>

In the above example, if the user tries to access the doList or doInput methods the validation will be skipped and the workflow will allow to continue even if the previous page had errors.

Tomcat Resource Configuration for Oracle Database Connection

noreply@blogger.com (jean) — Fri, 13 May 2011 18:39:00 +0000

Tomcat has a Database Connection Pool mechanism that will work for most of the database including Oracle. The mechanism described in the tomcat web site will work for most situations. Using the above method will make the code not depend on a specific database.

However there are situations where the application code is dependent on Oracle database. One such example is when you use Oracle Stored procedures and you need to use Oracle Cursor as on output parameter. In such case you will have to cast the Statement to Oracle specific oracle.jdbc.OracleCallableStatement class to get the cursor.

However if you simply cast the java.sql.CallableStatement to oracle.jdbc.OracleCallableStatement you may notice that you get a class cast exception if you configured your DataSource using the above method.

To overcome the class cast exception, you need to configure the data source using a Oracle specific connection factory. The following example will server that purpose. The text in bold are the changes pertaining to the Oracle connection factory.

<Resource name="jdbc/OracleDS" auth="Container" type="oracle.jdbc.pool.OracleDataSource"
user="DBUser"
password="xxxxxx"
driverClassName="oracle.jdbc.OracleDriver"
factory="oracle.jdbc.pool.OracleDataSourceFactory"
url="jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=your.db.host)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=serviceName)))"
maxActive="?"
maxIdle="?"
maxWait="-1"/>

spring apache commons configuration

noreply@blogger.com (jean) — Tue, 05 Oct 2010 05:21:00 +0000

I was surprised to find out that integrating the apache commons configuration with the spring framework is not trivial. I was hoping that there will be one bean that we can use out of the box within the spring framework or the apache commons framework and we set that as a property in the bean and we can go on. Unfortunately it was not that easy. It takes a few steps to get this working. Here are the steps that you need to follow to get this integration working:

1) Define the properties beans in spring xml file (typically ApplicationConfiguration.xml file)
as follows:

<bean id="configurations"
        class="org.springmodules.commons.configuration.CommonsConfigurationFactoryBean">
        <property name="configurations">
            <list>
                <bean class="org.apache.commons.configuration.PropertiesConfiguration">
                    <constructor-arg type="java.net.URL"
                        value="classpath:myconfiguration.properties" />
                    <property name="reloadingStrategy">
                        <bean class="org.apache.commons.configuration.reloading.FileChangedReloadingStrategy"/>
                    </property>
                </bean>
                <bean class="org.apache.commons.configuration.SystemConfiguration"/>
            </list>
        </property>
        
    </bean>
    <bean id="configuration" class="org.apache.commons.configuration.Configuration" factory-bean="&configurations" factory-method="getConfigurations"/>

2) Define a bean with the getter and setter that returns/accepts an array of org.apache.commons.configuration.Configuration class. To make it easier for the rest of the code to get the configuration easy, you may want to add a utility method in there to return the combined Configutaion. Here is an example:

private Configuration configs[] ;
private ConfigurationBuilder cfgBuilder ;
    public Configuration[] getConfigs() {
        return configs;
    }

    public void setConfigs(Configuration[] configs) {
        this.configs = configs;
        cfgBuilder = new ConfigurationBuilder();
        for (Configuration cgf: configs) {
            cfgBuilder.addConfiguration(cgf);
        }
    }

    public Configuration getConfig() {
        return cfgBuilder.getConfiguration();
    }

3) Define the bean properties in the spring xml file. I suggest, that you may want to define the above method in a base class of all your beans and define it as an abstract bean. This will enable you to use the properties in all your beans without having to define the properties in every bean. Here is an example:

<bean id="baseActionBean" class="com.my.company.BaseAction" abstract="true" >
            <property name="configs" ref="configuration" />
        </bean>
         <bean id="logonClass" class="com.my.company.UsefulAction" parent="baseActionBean">
        </bean>

4) Add the necessary libraries if you do not have them already. Here is the list of jars you will need.

commons-lang-*.jar
commons-configuration*.jar
spring-modules-jakarta-commons*.jar

With these changes, you should be able to use the properties in your beans with a code as simple as getConfig().getString("propertyKey")

Deleting old log files based on size and time

noreply@blogger.com (jean) — Sat, 25 Sep 2010 02:47:00 +0000

There are times when the log files take up too much space in the system and bring the system to run out of space. Fortunately there are few steps you can take to keep the log files under control for development servers. Here I have listed a simple strategy:
1) Create a dedicated partition for the log files so if the log files grow out of space, that still does not crash the system by making no space left for your data and configuration files.
2) Keep all log files for all applications under a common directory dedicated for logs. Replace the location of the log files within your application with symbolic links to the common logs folder or change the log configuration files to directly point them to the log directory.
3) Create a simple script to delete files that are older than certain number of days.
4) For files that does not get rolled over by the application, have a script that trims the head of the file by certain percentage of their size when the file size grows over a certain size limit.

If you follow these simple steps, your log files and directories will stay tidy and you still have logs left when you need them to debug issues. Here is a sample script that you can use to cleanup old files and trim larger files.

logdir="/tmp/test"
trimAmt=50
tempFile=/tmp/trimedfile.$$
sizeLimit=+1G
fileAge= +30

cd $logdir

find . -mtime $fileAge -exec rm -f {} \;

for FILENAME in $(find . -size $sizeLimit -print )
do
    filesize=$(wc -l $FILENAME | awk '{print $1}')
    trimsize=$(( $filesize -($filesize / $trimAmt)))
    echo " $FILENAME $filesize $trimsize "
    tail -$trimsize $FILENAME > /tmp/trimedfile.$$
    cat $tempFile > $FILENAME
    rm $tempFile
done

Java URL connection timeout - default timeout might save you from hanging

noreply@blogger.com (jean) — Wed, 25 Aug 2010 17:27:00 +0000

Many times, when you make a url connection or any connection that works over tcp ip (ftp, http etc) protocol, it is possible that your client just hangs. It is frustrating to debug this kind of issue because, you do not get any exception from your application except it just hangs. This can be a problem especially in production environments where there are firewalls between every single component and it is hard to trace network traffic.

If you are using sun jdk.1.4 or above there is a way you can prevent your code from just hanging. Best of all, you do not even have to make a code change. The sun jvm has a way to specify default timeout values for the net client. All you have to do is just add the following java parameters to the command line that starts your application.

-Dsun.net.client.defaultConnectTimeout=TimeoutInMiliSec -Dsun.net.client.defaultReadTimeout=TimeoutInMiliSec 

This will force the client to timeout and hopefully your application logs the exception that gives enough clue to debug the issue. For more information about specifying network properties in java check out the Networking Properties guide from sun/oracle.

Exporting ClearQuest records to Excel

noreply@blogger.com (jean) — Mon, 23 Aug 2010 20:43:00 +0000

If you need to export clearquest records to excel, the easy option is to use CSV format. You can do that by using the "Save result set to file" option and select a delimiter of your choice. Hoqwever, if one of your exported column is a multi line text, this may not provide the results you expect when you open the file in excel.

In that case you may want to export the data as xml file. Use the "Rational ClearQuest Client" (not the windows client) and click on the "export query results" icon above the query results pane. This will open a window where you can select the export format. In this window select the xml format and you can get the results as an xml file. (You can find detailed instruction on how to import here) However, the xml file is not very friendly to read. You can create a simple xsl file and add a single line to the xml file to use the xsl file you just created to make the file more reader friendly.

Here is a sample xsl file you can use: (You can download a copy from here)

<?xml version="1.0" encoding="ISO-8859-1"?

> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:template match="/">

<html> <body> <h2>My ClearQuest Results </h2> <table> <xsl:for-each select="exportedResults/columnNames"> <tr bgcolor="gray"> <xsl:for-each select="columnName"> <th><xsl:value-of select="."/> </th> </xsl:for-each> </tr> </xsl:for-each> <xsl:for-each select="exportedResults/records/record"> <tr> <xsl:for-each select="field"> <td><xsl:value-of select="."/></td> </xsl:for-each> </tr> </xsl:for-each> </table> </body> </html> </xsl:template> </xsl:stylesheet>

You can add the following line to the generated xml so you tell the browser to use the xsl. Make sure that the xsl and xml fiels are in the same directory.

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="cqTransform.xsl"?>

java urlconnection difference between jdk 1.4 vs 1.5

noreply@blogger.com (jean) — Fri, 25 Jun 2010 02:53:00 +0000

Recently, I ran in to an issue when trying to do some maintenance on an application that was last compiled using java 1.4 compiler. After compiling with java 1.5 or 1.6 compiler, the application stopped working. The part that was failing was when the application is making a URLConnection and posting a bytestream to the URL. It appears that from the client's perspective it wrote the entire bytestream. However, when the server is attempting to read the bytestream, it would get a EndOfFile exception. After scratching my head for a day, I noticed that one of my colleague had the same application working on his machine. The only difference is the URL he is using has a backslash at the end and mine did not. I tried changing the url by adding a backslash at the end, and the application starts working.

I have not seen any official documentation regarding this difference in the behavior between the same code compiled by jdk 1.4 and the newer ones. But, it seem to be the case as few others also experienced similar issue.

I will update this blog entry if I ever find more details on this issue. Meanwhile, if you run in to this issue, you may want to try adding a backslash to the end of your url.

Pagination using sql

noreply@blogger.com (jean) — Tue, 20 Apr 2010 00:49:00 +0000

If you have a complex query that returns lot of rows from the database and you are displaying the results in a table, you may want to add some pagination logic to your application. However, most people add pagination to the application layer, but retrieve all the rows back from the database. Java programming frameworks/API such as IBatis, JPA etc provides a mechanism to limit the number of rows retrieved or skip a number of rows. However, the implementation of those framework is left behind the jdbc driver and often results in the data still retrieved to the application server and then discarded. You could avoid this resource wastage by changing your query to limit the rows. The following is an example of how you can do just that.

    select * from ( select rownum rnum, a.* from (
   select columns from your_complex_table order by someField ) a
      where rownum < #UPPER_LIMIT )     where rnum >= #LOWER_LIMIT

In this example, you can change the values of #UPPER_LIMIT and #LOWER_LIMIT as parameters to your query limit the results to just the rows you want to deal with. If you are using a iBATIS you can make your actual query (the one does fetches all the records) in to an sql fragment and reuse it for the paginated query and get row counts. It also helps to get all the records in case of exporting all the data to some external format. The following example shows how you can reuse the sql fragment within your sqlMapping.

<sql id="selectItems">
select columns from your_complex_table order by someField
</sql>
<select id="selectItemCount" resultClass="int">
SELECT COUNT(*) AS total
<include refid="selectItems"/>
</select>
<select id="selectPaginated" resultClass="Item" parameterClass="map">
select * from ( select rownum rnum, a.* from (
<include refid="selectItems"/>
) a where rownum < #UPPER_LIMIT )
where rnum >= #LOWER_LIMIT
</select>

How to get UTM/MGRS from Latitude/Longitude using Oracle

noreply@blogger.com (jean) — Thu, 11 Feb 2010 21:12:00 +0000

Oracle Spatial queries can be used to derive the UTM/MGRS from a given Latitude and Longitude.
The following sql provides an example on how to get the UTM/MGRS

Select SDO_CS.to_USNG( SDO_GEOMETRY(2001,8307,SDO_POINT_TYPE(41.98,33.23,NULL),NULL,NULL), 1) UTM_MGRS from dual;

For more information on oracle spatial features, refer to http://www.oracle.com/technology/products/spatial/pdf/10gr2_collateral/locator_twp_10gr2.pdf

Oracle Locator vs Spatial

noreply@blogger.com (jean) — Thu, 11 Feb 2010 20:48:00 +0000

Oracle Locator	Oracle Spatial
Free included in oracle 10G onwards What Can Oracle Locator Do for You? Query nearest neighbor and other spatial relationships between geometries Perform location queries on relational information not stored in Oracle Spatial geometry Support long transactions Store and index vector geometries in the database Enhance application performance http://www.oracle.com/technology/products/spatial/pdf/10gr2_collateral/locator_twp_10gr2.pd	Additional option on top of oracle database What Can Oracle Spatial Do for You? Everything that Oracle Locator does, and: Perform length and area calculations on geometries Generate new geometries such as buffers and unions Perform coordinate systems transformations, for individual geometries or entire layers Store linear measure information http://www.oracle.com/technology/products/spatial/pdf/10gr2_collateral/spatial_twp_10gr2.pdf

Oracle Locator

Oracle Spatial

Free included in oracle 10G onwards
What Can Oracle Locator Do for You?
Query nearest neighbor and other spatial relationships between geometries
Perform location queries on relational information not stored in Oracle Spatial geometry
Support long transactions
Store and index vector geometries in the database
Enhance application performance
http://www.oracle.com/technology/products/spatial/pdf/10gr2_collateral/locator_twp_10gr2.pd

Additional option on top of oracle database

What Can Oracle Spatial Do for You?
Everything that Oracle Locator does, and:
Perform length and area calculations on geometries
Generate new geometries such as buffers and unions
Perform coordinate systems transformations, for individual geometries or entire layers
Store linear measure information
http://www.oracle.com/technology/products/spatial/pdf/10gr2_collateral/spatial_twp_10gr2.pdf

Using Oracle ESB as a web service client to pull data periodically

noreply@blogger.com (jean) — Wed, 10 Feb 2010 15:47:00 +0000

In this article we will cover the detailed steps to create an ESB service that invokes a web service periodically and write the response to a file. The web service in this case takes an input parameter and returns some response based on the input. The client is expected to parse the response and take an element from the response and supply that as the input for the next call.

1.    Create a service group. Lets call it EsbWebServiceClient
2.    Create a xsd file to represent the configuration information that we will use to pass to the webservice.
3.    Create a xml file that complies with the schema and the initial parameters for invoking the web service. The ESB adaptor will update this file after each invocation.
4.    Create a file adaptor to read a file. Don’t check the mark that says delete file after reading. Specify the schema that you created in step 2 for the schema of the file. When you are done with step you should end up with boxes 1 & 2 shown in the diagram with the line connecting them.
5.    Create a ESB Service for Soap service (Right click on design pane-> Create ESB Service->Soap Service) On the pop up window, select the WSDL file representing the service. For this demo, the web service is a simple service that takes a string as input and returns a string. This creates Box 3.with no lines connected.
6.    Now create a schema that represents the input elements to the webservice and the output of the webservice. We will use this as the holding area to have both the request and response so we can write the output to a file and update the config file with the updated information. In the example the schema is called serviceParameterAndResponse.
7.    Create a ESB routing Service and name it ConfigMapper.(Right click on design pane-> Create ESB Service->Routing Service) On the pop up window, select the serviceParameterAndResponse schema for both request and response formats.. This creates Box 4.with no lines connected. This routing service provides the framework for us to merge the input parameters and the webservice output.
8.    Now create a file adaptor that can write an output file and name it WsResponseWriter. To keep it simple, specify the schema for the file as the same as the response from the webservice This should create Box 5 with no lines connected.
9.    Now create a file adaptor that can write the config file and name that ConfigFileWriter.. Specify the schema and the location for the file as the same as the schema created in step 2. This should create Box 6 with no connector lines
10.    Double click box 2 and add a routing rule to invoke the method on the webservice. This should create a reply line. Now point the reply to the execute method on the routing service created on step 7. This should create lines 8 & 9 on the esb diagram..
11.    Double click box 4 and add two routing rules. One pointing to WsResponseWriter and the second one pointing to ConfigFileWriter. This should create the lines 10 & 11 in the diagram.
12.    Now the diagram is almost complete. You will see the mappings are still gray. We will add the mappings which will make them yellow.
13.    Double click on the top "X" and create a new mapping. When the designed is displayed, drag a line from the config parameter to the input parameter of the web service.
14.    Double click of the bottom "X" of box 2. Create a new mapping. You will notice there is an option to "include the request in reply play load". Make sure you select this option. When the designer panel is displayed you should see a new node called “ESBREQUEST” on the left hand side. To learn more about the $ESBREQUEST parameter refer to this article (http://www.soastation.org/2007/05/esbrequest-in-oracle-esb-routing.html)
15.    Now select the source view of the xslt and add a function to parse the required fields from the request to map to the output. In the example below, I have mapped the string after the pipe symbol to be stored as the parameter for the next call. And for the response, I have concatenated the input and the output to be written out to the file. Please note that you will not be able to drat lines to or from the $ESBREQUEST variable on the designer view.

<xsl:param name="ESBREQUEST"/>
<xsl:template match="/">
   <out1:serviceParameterAndResponse>
     <out1:param1>
       <xsl:value-of select='substring-after(/tns:getElapsedTimeSinceResponse/tns:return,"|")'/>
     </out1:param1>
     <out1:response>
       <xsl:value-of select='concat(substring-after(/tns:getElapsedTimeSinceResponse/tns:return,"|"),substring($ESBREQUEST/tns:lastTime,1.0,2.0))'/>
     </out1:response>
   </out1:serviceParameterAndResponse>
</xsl:template>

16. Now double click on the "X" on box 4 and create new mappings. These mappings should be straightforward as all the manipulation is done on the previous step. Just draw a straight line from the corresponding field on the left to right. Repeat this for both the ‘X” on the box.
17. Now your diagram should look similar to the one shown above. You can register the service to your ESB and test it.

If you like see a copy of the project that I used to create the demo, feel free to drop me an email. My email address is jmeslie at gmail dot com.

How to set Vista to allow inbound web traffic?

noreply@blogger.com (jean) — Thu, 14 Jan 2010 16:12:00 +0000

If your organization uses Group Policy, use the Windows Firewall Group Policy settings to configure Windows Firewall. If you have set exceptions in the windows firewall too to allow trafic and you are still not able to access your computer from a remote machine, most likely cause is the group policy is set on your computer. The Group Policy Object Editor (type Gpedit.msc on the windows search bar) provides access to the Windows Firewall settings. The settings are stored within the Group Policy Object Editor at Computer Configuration/Administrative Templates/Network/Network Connections/Windows Firewall.

Once you are in the group policy editor, navigate to Local Computer Policy -> Windows Setting -> Security Seting -> Windows Firewall with advanced security -> Inbound rules

How to add version info to subversion files automatically

noreply@blogger.com (jean) — Thu, 22 Oct 2009 14:15:00 +0000

Adding version info to subversion files
You can configure Subversion and TortoiseSVN to update the version information on the files checked in to subversion by adding subversion keywords to your file and setting the subversion properties. To automatically add the version number, the user who changed the file and the date in which the file was changed, follow these steps:

Add the subversion keywords to the file. These properties and keywords are case sensitive. To ensure that you are using the correct value of the keyword, I suggest copy the following text to your file and make the changes necessary to make it a valid comment:

/**
**  Last Changed by    : $LastChangedBy$
**  Last Changed Rev   : $Rev$
**  Last Changed Date  : $LastChangedDate$
**/

The next step is to add the subversion properties to the file. It can be done by one file at a time or at a folder level to add the properties to all the files in the folder and its sub folders. To do this, right click on the file/folder, select TortoiseSVN->Properties. Click “New” and in the properties window select “svn:keywords” and enter “LastChangedBy Rev LastChangedDate” as the value as shown below:

To avoid making errors, I would suggest that you use the import/export functionality to add these properties. Select a file that already has this property and you know it is working and click on export as shown below. Save the property to a file and close the properties window. Now go to the TortoiseSVN properties window of the new file and select import and select the properties file you saved before. This will ensure that you have set the properties correctly for that file.

Subversion automatic property setting
You can configure Subversion and TortoiseSVN to set properties automatically on files and folders when they are added to the repository. There are two ways of doing this.
You can edit the subversion configuration file to enable this feature on your client. The General page of TortoiseSVN's settings dialog has an edit button to take you there directly. (Right click anywhere within the right pane of windows explorer, Highlight the TortoiseSVN menu and click on the settings tab to bring up the General settings page of Tortoise SVN )
The config file is a simple text file which controls some of subversion's workings. You need to change two things: First, in the section headed miscellany uncomment the line enable-auto-props = yes. Secondly you need to edit the section below to define which properties you want added to which file types. This method is a standard subversion feature and works with any subversion client. However it has to be defined on each client individually - there is no way to propagate these settings from the repository. Table 1 shows a sample snippet of the config file with the changes that adds the properties for the file types .txt, .java, .jsp, .htm*, *.properties, *.xml and *.xml. You can add more lines as needed to add more file types.
An alternative method is to set the tsvn:autoprops property on folders. I recommend that we use this method as all of us are using TortoiseSVN client. This method only works for TortoiseSVN clients, but it does get propagated to all working copies on update. The advantage is if even if one user did not have the correct settings on their PC, these properties will get propagated as long as the top folder was created from a pc with the right setting. To do this right click on a top level folder for the project, select tortisSVN properties and select add property and select the tsvn:autoprops property name and add the values as shown below:

Whichever method you choose, you should note that auto-props are only applied to files at the time they are added to the repository. Auto-props will never change the properties of files which are already versioned. If you want to add the properties to files that are already committed to subversion, you may want to select the top level folder and add the subversion property.

Table 1: Partial Text of the Subversion config file

### Section for configuring miscelleneous Subversion options.
[miscellany]
.
.
.
### Automatic properties are defined in the section 'auto-props'.
enable-auto-props = yes
### Set interactive-conflicts to 'no' to disable interactive
### conflict resolution prompting.  It defaults to 'yes'.
# interactive-conflicts = no

### Section for configuring automatic properties.
[auto-props]
### The format of the entries is:
###   file-name-pattern = propname[=value][;propname[=value]...]
### The file-name-pattern can contain wildcards (such as '*' and
### '?').  All entries which match (case-insensitively) will be
### applied to the file.  Note that auto-props functionality
### must be enabled, which is typically done by setting the
### 'enable-auto-props' option.
*.txt = svn:eol-style=native;svn:keywords=LastChangedBy Rev LastChangedDate
*.java = svn:eol-style=native;svn:keywords=LastChangedBy Rev LastChangedDate
*.jsp = svn:eol-style=native;svn:keywords=LastChangedBy Rev LastChangedDate
*.htm* = svn:eol-style=native;svn:keywords=LastChangedBy Rev LastChangedDate
*.properties* = svn:eol-style=native;svn:keywords=LastChangedBy Rev LastChangedDate
*.xml = svn:eol-style=native;svn:keywords=LastChangedBy Rev LastChangedDate
*.sql = svn:eol-style=native;svn:keywords=LastChangedBy Rev LastChangedDate
# *.png = svn:mime-type=image/png
# *.jpg = svn:mime-type=image/jpeg
# Makefile = svn:eol-style=native

Oracle - Generate a list of dates with time

noreply@blogger.com (jean) — Mon, 19 Oct 2009 20:40:00 +0000

I had to create a report that had to have a line for each date/time within the given range, even if the actual data did not have value for that date. Say, web trafic count for each 5 minutes for a given day. The simple full join of the date will not work for this because, there might be periods during the day there was no user activity, and it is expected to have zeroes in that case.

A simple approach would be to use the hirerical query used in this blog if you only need one row per date. But if you need date entries for multiple times per day, say a row for every 5 minutes, the following function will come in handy. It uses a Pipelined Table Function.


CREATE OR REPLACE TYPE date_array AS TABLE OF DATE;
/
CREATE OR REPLACE FUNCTION MPI2.date_table(sdate DATE, edate DATE, mi_interval integer   )

RETURN date_array PIPELINED is
 fDate date ;
BEGIN
  fDate := sdate;
  while fDate < edate LOOP
    PIPE ROW(fDate );
    fDate := fDate +  mi_interval / 1440;  
  END LOOP;
  RETURN;
END date_table;

Now you may query this function just like you query a table, as follows


SELECT to_char(column_value,'DD-MON-YYYY HH24:MI')  FROM TABLE(CAST(MPI2.DATE_TABLE(trunc(sysdate-1),sysdate,5) AS date_array));

The above query will return a row for every 5 minutes. You can do a full join of this table with your actual tables to get the result you wanted.

How to create your own Windows Service Monitor? - A scheduled task

noreply@blogger.com (jean) — Fri, 09 Oct 2009 17:27:00 +0000

There are times when a windows service just stops running without any error. The following script provides a brute force script that will help monitor the service and restart it if the service is down.

To create this service, copy the following code to a cmd file and setup a schedule task. (Replace YourServiceName with the actual name of your service)


@ECHO OFF
set logFile=c:\temp\svcmonitor.log
set tempFile=c:\temp\tmp.txt
set serviceName=YourServiceName
sc query %serviceName% >  %tempFile%
for /F "skip=3 tokens=1,4" %%i in (%tempFile%) do if %%i equ STATE  (
if  %%j equ STOPPED (
echo %date% %time%  %serviceName% service is stopped. Trying to restart. >> %logFile%
sc start %serviceName% >>  %logFile%
echo %date% %time%  %serviceName% restarted >> %logFile%
) else (
echo %date% %time%  %serviceName% service is %%j >> %logFile%
)
)

Now update the scheduled task in to run every day. Then go to the advanced scheduling options menu and set to repeat the task every 10 minutes(or what ever frequency deemed appropriate for your task) as shown below. Now you have a poor man's version of a windows service monitor.

Debugging Webservice clients - Log HTTP transport content

noreply@blogger.com (jean) — Thu, 08 Oct 2009 14:36:00 +0000

Many times it is useful to see the actual content that is send and received through the wire in order to troubleshoot webservice clients. Unfortunately, there does not seem to be a standard way to enable logging across all the webservice clients. Depending on the client you use for accessing the webservice, the debugging method varies. Here are a few tips that will help.

If you are using Axis client to access a webservice and using log4j for logging, you can enable printing of the http request and response contents by setting the "httpclient.wire.content" property to "debug". For example if using the log4j.properties file set the following property:


log4j.category.httpclient.wire.content=debug,stdout

If the webservice is connecting to a secure url using https protocol, setting the java system property "javax.net.debug" to "plaintext" will provide a hex dump of the content send and received.

If you are using the web service proxy provided by oc4j within jdeveloper or the oc4j container, you can enable the java logging property "oracle.webservices.client" in logging.properties file to FINE


oracle.webservices.client.level=FINE

Setting this option will allow a standalone client to log the request and response content to the log file specified by the logging.properties file.

If the client is running within the oc4j container, you can enable the oracle diagnostics logging for "oracle.webservices.client". This can be done by navigating to the <oc4j container> -> Administration --> Logger Configuration page and setting the log level as "FINE" for "oracle.webservices.client" property.
You can view the request and response contents in the log.xml file.

Failed to set the internal configuration of the OC4J JMS Server with: XMLJMSServerConfig

noreply@blogger.com (jean) — Wed, 07 Oct 2009 13:27:00 +0000

After setting up a few jms queues, if the oc4j did not shutdown properly, (ie) shutting down the machine without issuing "opmnctl stopall", the above error is very likely to occur.

To fix this just rename the <ORACLE_HOME>/j2ee/<soa_instance>/persistence folder and create an empty folder in its place.

Now if you start the server, it should startup fine.

Asynchronous Web Service Calls over HTTP using Oracle ESB and JMS

noreply@blogger.com (jean) — Thu, 17 Sep 2009 15:49:00 +0000

How to create an asynchronous webservice that supports failover using Oracle ESB?

This article describes a general design for invoking web services in an asynchronous fashion even if the web service provider only supports synchronous invocation. I have used JDeveloper to create this service however the same principles can be used for any development tool.

Use case:

Generally there are 3 different scenarios of using a web services.

Request/Response Real Time: .Here the response from the web service is critical for the application to proceed to the next step. If the web service is not available, user needs to know right away. Example: An application depends on a web service to display a list of accounts they can manage. In this case invoking the web service in an asynchronous fashion is not going to help much. A simple synchronous web service call will do the work without adding any complexity.
Request only – (delivery critical): Here, the client application needs to send a message to a web service, but it is not expecting any response from the service. The client wants to make sure that the message will be delivered to the service even if the service was down at the time of invocation.
Request/Response time (not critical, delivery critical): Here, the client application needs to send a message and receive response from the web service. The client wants to make sure that the message will be delivered to the service even if the service was down at the time of invocation. The client is not waiting for the response from the service. Instead, the client has provided a callback mechanism to receive the response.

The design described here suites for the third scenario and the second scenario. For the second scenario, only part of the design is needed.

The approach:

The idea here is to use JMS queues to receive the request and response messages to and from the web service. Clients will access the web service through a JMS-WebService adaptor. JMS listeners will invoke the web service and the callback service when messages are received in the request or response queues.

The following diagram describes the approach.

The web service is created and deployed in a remote server. In many cases this might be an external web service provided by a third party hosted outside your firewall. The client provides its callback mechanism as a web service and it is deployed locally. JMS queues are created in the local server for receiving the request and response messages. A JMS adaptor is created that exposes the request queue as a web service. Using JMS adaptor provides the following benefits:

The client can communicate with the service using SOAP/HTTP(S) which is well supported by most tools than the SOAP/JMS.
It hides the dirty details of the JMS details from the web service client developer.
The web service client developer is using the same protocol to access the service if he has to call the service directly. (except for the return value)

A JMS listener is created to monitor the request queue and invoke the actual web service using a SOAP adaptor. The response from the web service will be written to the response queue. The listener and the adaptor can be configured to retry the service invocation if the service is down.

Another JMS listener is created to monitor the response queue and invoke the callback web service using a SOAP adaptor. The listener and the adaptor can be configured to retry the service invocation if the callback service is down when the actual web service responded.

Benefits:

This approach allows you to develop an asynchronous callback mechanism to your web service, even if the original service did not provide one. Most web services that are request/response provides only synchronous calling mechanism only.

It provides a way to decouple the dependency on the external web service without adding any custom client code. These days most application server vendors provide JMS and SOAP adaptor widgets that let you create these without having to write them yourself.

If you are using a web service that does not return and response back (one way request only), this approach can still work just be not creating the response queue and the related components (step 5 and beyond in the above diagram).

Limitation:

This approach assumes that you have enterprise grade tools such as JMS/SOAP Adaptors in your production environment. If such tools are not available, you may have to create these adaptors yourself. If you are developing them yourself, the time spent creating these generic services might be higher than adding failover logic to the application itself.

Creating the service

The instructions assume you are using Oracle application server 10.1.3.x and jDeveloper 10. The exact steps will be different for other application server and IDEs. Please note that these instructions assume that you are familiar with the above tools and does not provide step by step details of each widget.

Create and deploy the ‘actual WebService. If the service is provided to you by a third party you can skip this step.
Create and deploy the web service response receiver web service to your local server. If you are creating only a one way web service call you can skip this step.
Create JMS queues for request and response queues. This can be done through the Enterprise Manager application by navigating to à Administration à Enterprise Messaging Service à Create/delete/edit JMS destinations. Since we are going through this exercise to create a failover mechanism, I would suggest create these queues with File/Database persistence.
Create separate JMS Connection factories for each of these queues so we don’t get in to any resource contentions.
Create a new jDeveloper ESB project
Copy the WSDL for step 1 and 2 in to the project so we can reuse the schema elements.
Create JMS adaptor for writing to the request queue (produce message). Check the box that says “Invocable from external service”.

If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Produce Message as the operation type and specify a meaningful name (similar to the original service’s method name) for the operation. The WSDL exposed by this adaptor will be later used by the web service client. Select the request queue and the appropriate connection factory. Specify a large time out so if the external service is down, the message will not expire before the ESB exhausts all retry attempts. On the schema selection screen, select the message element that corresponds to the input message to the external web service from the WSDL.

Create an ESB Soap service (Create ESB Service à SOAP Service) and specify the WSDL for the actual web service.
Create JMS adaptor for reading from the request queue (consume message). If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Consume Message as the operation type. Select the request queue and the appropriate connection factory. On the schema selection screen, select the message element that corresponds to the input message to the external web service from the WSDL (schema and queue names same as previous step). JDeveloper should have created the adaptor service icon and the routing service.
Edit the routing service by double clicking and add a routing rule. For the consume message operation. Wire it to point to the appropriate operation of the SOAP adaptor from step 8. Create a transformation map to map the elements from the request to response. Also, you may want to make this execution asynchronous.
Create JMS adaptor for writing to the response queue (produce message).

If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Produce Message as the operation type. Select the response queue and the appropriate connection factory. Specify a large time out so if the callback service is down, the message will not expire before the ESB exhausts all retry attempts. On the schema selection screen, select the message element that corresponds to the input message to the callback web service from the WSDL (or the output message from the external web service).

Create an ESB Soap service (Create ESB Service à SOAP Service) and specify the WSDL for the callback web service.
Create JMS adaptor for reading from the response queue (consume message). If you selected Oracle Enterprise Messaging Service in the adaptor configuration widget, select Consume Message as the operation type. Select the response queue and the appropriate connection factory. On the schema selection screen, select the message element that corresponds to the input message to the callback web service from the WSDL. JDeveloper should have created the adaptor service icon and the routing service.
Edit the routing service by double clicking and add a routing rule. For the consume message operation. Wire it to point to the appropriate operation of the SOAP adaptor from step 12. Create a transformation map to map the elements from the request to response. Also, you may want to make this execution asynchronous.
Edit the routing service created in step 10, by double clicking and expand the routing rule for the consume message operation. Wire the response message from the service to point to the produce message operation of the JMS adaptor for the response queue. Create a transformation map to map the elements from the request to response. Also, you may want to make this execution asynchronous. The following diagram represents the ESB diagram for a project with these settings.
Deploy/Register the ESB project to the OC4J container.
Create a web service proxy for the WSDL exposed in step 7. Invoke the service using the generated proxy. If everything went well you should notice the response came back through your callback service.

How to Change Oracle ORAWSM Database Password

noreply@blogger.com (jean) — Tue, 21 Apr 2009 18:33:00 +0000

If you are reading this you already know that the WSM uses the ORAWSM user in the infra database. If you need to change the Oracle database password for the ORAWSM user, you would guess that you will may have to change the password in some configuration files for the webservices manager to use the new password. However that process is not that simple. Here are the steps you need to follow.

Step 1: Change the password directly in the database.

Login as sysdba and change the password of orawsm.

Step 2: Change the password in WSM configuration files
Reffer to steps described in the following url. (Content copied here for your reference.)
http://download.oracle.com/docs/cd/B31017_01/integrate.1013/b31007.pdf (appendix C - Changing Passwords)

The following procedure describes how to change a database password for a single instance of an Oracle WSM Database.
To change a password for a single database instance
1. Edit the attribute db.password in the ORACLE_HOME/owsm/bin/coresv.properties file.
2. From the command prompt, execute the following command:
wsmadmin encodePasswords coresv.properties dataload.messagelog.db.password
3. From the command prompt, execute the wsmadmin copyDBConfig command:
wsmadmin copyDBConfig
4. From the command prompt, execute the wsmadmin deploy command:
wsmadmin deploy application_server_password component
See Appendix B, "Oracle Web Services Manager WSMADMIN Commands" for more information.

The following procedure describes how to change your database password when you have multiple instances of Oracle WSM Database.

To update and reconcile a password for multiple database instances
1. Update the relevant attribute in the application properties file for each application.
These are located under ORACLE_HOME/owsm/config/application_name.
2. Update and reconcile the passwords for all the database instances.
3. From the command prompt, execute the wsmadmin encodePassword command
for each affected file:
wsmadmin encodePasswords path_to_file properties
4. From the command prompt, execute the wsmadmin deploy command:
wsmadmin deploy application_server_password component
See Appendix B, "Oracle Web Services Manager WSMADMIN Commands" for more
information.

Step 3: Change the password in the policy configuration for each gateway defined.

For each gateway and agent in your system, there is a property - cfluent.messagelog.db.password - which has to be manually changed to the correct password value for your OWSM schema, as it is left unaffected by the redeployment of the OWSM applications.
This is handled through the OWSM console by accessing the properties screen for each gateway and agent (Policy Management > Manage Policies > Edit Component Properties).
The password that you enter should be the actual password with no encryption. The value will be encrypted before it is stored in to the orawsm.policy_manager_objects table.