Technology highlights

How to extract HTTP header fields in a REST API

2011-12-30T14:35:00.000-08:00

To be able to extract HTTP headers fields from incoming requests accessing a REST API is particularly useful when trying to log the transactions that are coming through the REST based service.

In this example, I use Java based JBoss RESTEasy together with @Context java annotation to extract the header information.

One of the REST API operation is /version that returns the current version of the API in a JSON format.The interface operation is defined as follow:


     import javax.ws.rs.core.HttpHeaders;

     /**
     * Get the current version of the REST API.
     */
     @GET
     @Path("/version")
     @Produces("application/json")
     @GZIP
     public Response getVersion(@Context HttpHeaders headers);

The @Context annotation allows you to map request HTTP headers to the method invocation.

One way to safely extract all available header parameters is to loop through the headers.The method getRequestHeaders from javax.ws.rs.core.HttpHeaders returns the values of HTTP request headers. The returned map is case-insensitive and is read-only.

   if (headers != null) {
       for (String header : headers.getRequestHeaders().keySet()) {
          System.out.println("Header:"+header+
                             "Value:"+headers.getRequestHeader(header));
       }
   }

When querying the REST API via a browser:

  http://www.acme.com/api/version

you obtain a minimum set of headers fields:


   Header:cache-control Value:[max-age=0]
   Header:connection Value:[keep-alive]
   Header:accept-language Value:[en-us,en;q=0.5]
   Header:host Value:[www.acme.com]
   Header:accept Value:[text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8]
   Header:user-agent Value:[Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0)    
   Gecko/20100101 Firefox/8.0]
   Header:accept-encoding Value:[gzip, deflate]
   Header:session-id Value:[d636369c]
   Header:accept-charset Value:[ISO-8859-1,utf-8;q=0.7,*;q=0.7]

To be able to extract a specific header field (e.g. Host), you can use the getRequestHeader function:

 public Response getVersion(@Context HttpHeaders headers) {
     if (headers != null) {
        List<String> hostHeader = headers.getRequestHeader("host");
        if (hostHeader != null) {
           for (String host : hostHeader) {
              LOG.debug("Host:"+host);
           }
        }
     } 
  
     // Get the version
     final Version current_version = new Version();
     final ObjectMapper mapper = new ObjectMapper();
     final JsonNode rootNode = mapper.createObjectNode();
     ((ObjectNode) rootNode).putPOJO(Version.XML_ROOT_ELEMENT, current_version);
     
     final ResponseBuilder builder = Response.ok(rootNode);
     return builder.build();
   }

If you are using a test harness tools like cURL, you can also easily add additional HTTP header fields
(e.g. the email address of the user making the request or the referer) and test the logging functionality of your REST API:

curl -H "From: user@example.com" http://www.acme.com/api/version

curl -H "Referer: http://consumer.service.acme.com" http://www.acme.com/api/version

The headers can also be obtained through the HttpServletRequest object.
This can be done using the @Context HttpServletRequest annotation and can provide additional information about the incoming request:

public Response getVersion(@Context HttpServletRequest request) {
   LOG.debug("Host:"+request.getHeader("host"));
   LOG.debug("Request-URL:"+request.getRequestURL());
   ...
}

Since HttpServletRequest extends ServletRequest you also getting useful methods providing information on the Internet Protocol (IP) address, host and port of the client or last proxy that initiated the request.

public Response getVersion(@Context HttpServletRequest request) {
   LOG.debug("Remote-IP:"+request.getRemoteAddr());
   LOG.debug("Remote-Host:"+request.getRemoteHost());
   LOG.debug("Remote-Port:"+request.getRemotePort());
   ...
}

cURL setup and OpenSSL

2011-11-28T15:29:00.000-08:00

In a previous post I was explaining how to use cURL to quickly test REST web services. If your web service has to be secure, you are probably using Transport Layer Security (TLS) - the new Secure Sockets Layer (SSL) cryptographic protocol.

To be able to use TLS/SSL with cURL, you will need to have some SSL libraries/DLL such as OpenSSL installed. If you did not install OpenSSL, you will most likely get the following error when trying to use cURL:

"The program can't start because LIBEAY32.dll is missing from your computer. Try to reinstall the program to fix this problem."

To fix this, you can either install the OpenSSL on your machine, or if you just want to install the minimum, you can just copy the following DLLs from OpenSSL (I am using OpenSSL 1.0.0c) to your cURL folder:

libeay32.dll
ssleay32.dll

By the way, you can download cURL for most of the OS and platforms and usually SSL and non SSL versions are available for each OS (see current windows versions below):

Combining IHE transactions : Orchestration or Choreography?

2011-10-28T14:33:00.000-07:00

Integrating the Healthcare Enterprise (IHE) has gained tremendous momentum in the past few years. Started as a Healthcare Information and Management Systems Society (HIMSS) and Radiological Society of North America (RSNA) workshop in October 1998 with only 15 participants including AGFA, Cerner, Fuji, GE, HP, Philips, Siemens, the IHE initiative has more than 400 members worldwide. It is composed of healthcare professional associations, government agencies, Health Information Exchanges (HIE), healthcare providers, IT and consulting companies, trade, educational, standard and research organizations. IHE provides a standards based-interoperable framework to share and exchange information between healthcare organizations across networks.

Combined with the latest technology and well established standards (HL7, DICOM, IDC9/10, LOINC, W3C), clinical data can then be securely and privately accessed and transmitted locally between network endpoints (e.g. within the same hospital between the practices and a lab). IHE profiles can also be used across Health Information Exchanges (HIE) of Regional Health Information Organization (RHIO), or a state level (e.g. an individual state in the US, Canada or Europe), or at the federal level (e.g. the US Nationwide Health Information Network or NwHIN). As a result, there is a strong need to integrate and combine individual IHE profiles end-points to form “hub of hubs” or “network of networks” to support health information exchange between the participating nodes entities.

Because IHE transactions are most likely to be offered as web services, combining those transactions can be done following Service Oriented Architecture (SOA) and Service Oriented Computing (SOC) principles.

Conventional middleware distributed system infrastructures (e.g. JMS) are generally not sufficient or flexible enough to mediate, transform, federate and route messages from and to web services.

Enterprise Application Integration (EAI) goes one step ahead, trying to separate the applications from the web services end-points. EAI usually employs a centralize service broker for this, a set of connectors and an independent data model. Services can then send and subscribe to receive messages to and from the broker. However, this very centralized approach requires a large amount of up front development and business process design for the connectors, as well as high cost of maintenance in general. Enterprise Service Buses (ESB) is an infrastructure that leverages EAI principles.

Orchestration

Like EAI, orchestration uses a centralized approach. Web services orchestration is realized through Business Process Execution Language (BPEL) that describe the collaboration and interaction between the web service participants.

Business workflows, states, actions, events, control flows and exception handling can be specified. Messages can be received and sent directly from and to WSDL ports. Results received asynchronously from web services can be combined to create new messages. Usually BPEL workflows are created and updated with visual design tools.

There is often an overlap between orchestration engines and Enterprise Services Buses (ESB). Vendors now also offer BPEL design on top of more generic EAI mechanisms enabling true orchestration for ESBs.

Choreography

Choreography is more distributed and collaborative in nature and uses the Web Service Choreography Interface (WSCI) specification and the WSDL description files to represent the flow of messages exchanged between the Web services involved.

Choreography seems more flexible than orchestration since it does not rely on a central element that could become a bottle neck and seems to offer more complex interaction potential between web services. However, choreography has some drawbacks including the necessity for all web services to be aware of overall business process workflow. WSCI itself does not specify the definition and the implementation of the mechanism to implement the message exchange.

In addition to this, performance can be an issue if high volume message transactions between the end-points peers are not handled properly.

Moreover, there is no clear responsibility for the overall workflow leading to legal issues related to monitoring and maintenance.

Orchestration vs Choreography

Orchestration also has the advantage to be a much more mature integration technology than choreography. For these reasons, most of the state-wide health information exchange networks in the US employ a Service-Oriented Architectural (SOA) model that is implemented through an Enterprise Service Bus (ESB) orchestration.

In addition to this, web service orchestration offers much more than just technical benefits:

Organizational: standardization, narrow gap between business analysts and developers;
Managerial: risk reduction, lower costs, more flexibility;
Strategic: IT resilience, delivery time reduction, less technology lock-in;
Technical: portability, reuse, interoperability of tools, less complex code, better maintainability;
Operational: efficiency, automation, higher level tasks management.

When deployed on high performance platforms such as SOA software appliances, orchestration solutions are easy to test, extend and maintain.

More on this topic to be published as part of the following paper: Andry F., Wan L., HEALTH INFORMATION EXCHANGE NETWORK INTEROPERABILITY THROUGH IHE TRANSACTIONS ORCHESTRATION, 5th International Conference on Health Informatics (HEALTHINF 2012).

cURL tests harness and TLS

2011-09-23T10:18:00.000-07:00

In a previous post, I have explained how to use cURL to test harness REST based Web services. One thing I did not described was how to add Transport Layer Security (TLS) in your tests. In other words, how to successfully test REST Web services over HTTPS?

The cURL manual describes a certain number of options that can be used. One of the most convenient option is -k or --insecure. It allows curl to perform "insecure" SSL connections and transfers. All SSL connections are attempted to be made secure by using the CA certificate bundle installed by default. So if you SSL connection does not require client side authentication it is a very quick way to test your web service over SSL:

curl -k https://www.acme.com/api/version

Another useful option can be used if in conjunction to SSL, you need to compress your payload via GZIP for example to optimize the transfer of large messages. In this case, you will use the option -H or --header that will help you specify custom headers to your request:

curl -H "Accept-Encoding: gzip,deflate" -k https://www.acme.com/api/users/list

Notice in that case that the -k or --insecure option is always placed just in front of URL.

Of course, other headers such as the one described previously can be combined:

curl -c ./cookies.txt --data-binary @login_password.json -H "Content-Type: application/json" -H "Accept-Encoding: gzip,deflate" -k https://www.acme.com/api/users/token

Medication Adherence - How technology can help?

2011-08-31T15:10:00.000-07:00

Looking for ideas to reduce the rise of health care cost? Easy! Ask patients to swallow their medication!

According to recent studies, the lack of prescription medication adherence cost between $250 and $300 billion annually, including $100 billion in hospitalization. For example, it is estimated that in the US 89,000 deaths annually are due to non adherence to anti-hypertensive treatments. This problem is particularly acute for patients with coexisting conditions who take a variety of medications sometimes prescribed by different physicians.

Obviously doctors and pharmacists have a critical role to encourage patients and caregivers to administrate medication correctly and rigorously.

Cost is not necessarily the main factor: even when drugs are free, adherence rate is 60% in the US and only %50 in developed countries. In other words, improvements in co-payments structures will only partially improve adherence. Another area of possible improvement will be the use of fee-for-service model.

Another way to improve medication adherence is the proper use of specific technologies:

Here are some suggestions:

the integration of practice EMRs systems and pharmacies via HIE solutions (a lot of practices are still using paper).

accurate and Electronic Health records (EMRs, and PHRs) that help verify drug, herbs, supplements interactions and countraindications. The meaningful use guidelines go in this direction.

wider use of medication home delivery (recognized as a way to save cost by introducing better competition)

electronic monitoring systems coupled with personalized counseling has shown improvement in medication adherence rates to patients

online incentive programs (e.g. provided by the employers) to promote prevention, quality of life and best practices in therapy. It is recognized that financial incentives and other rewards when well designed and targeted can improve adherence (smaller and higher reward frequency is usually more efficient than big and sporadic rewards).

advances in personalized medicine can help taylored medication intake and increase medication adherence

screening and assessment tools (e.g. patients with chronic conditions could be screened for depression leading to poor medication adherence).

new high-tech integrated devices including:

Home devices such as the well known high-tech Japanese toilets which provide urine analysis, blood pressure, weight and body temperature
Radio Frequency Identification (RFID) based wireless sensor device can help track drug taking compliance
Wireless pills organizers designed to simplify medication management and improve adherence
Personalized wearable sensors such as the latest electronic temporary tattoo

With the amount of money at stake, I am sure there is a large number of companies, including start-ups, who are working hard on this problem. So stay tune!

Finally, we have to keep in mind, that other factors including lifestyle, psychological issues and health literacy play an important role as well. And these are not negligible!

How to test secure web services with soapUI - part #2

2011-07-29T11:59:00.000-07:00

In a previous article I described how to specify signature and encryption for outgoing Secured Web Services request.

1. SAML

In this second article I will talk about SAML and incoming Secure Web Service responses.

If you are using SAML 1.X then you just need to add a SAML 1.X assertion in the corresponding window.
You may have to add a timestamp as follow:

In this example, we specify 10,000 milliseconds (10 seconds) for the life time of the Timestamp:

SAML 1.X assertions are copied and pasted in the SAML tab:

Of course, if you combine signature and encryption with SAML 1.X. You can those to the configuration tabs as well.

For SAML 2.0, the only option you have for this version of soapUI is to add it manually to the WSSE section of your SOAP request:

2. Incoming Secure Web Service
Responses

Setting up incoming secure Web services responses encrypted and/or signed is easier than for outgoing request:

1) Make sure that you have Keystores / Certificates tab set - You probably have done that earlier as described in part #1 of this article.
You verify the signature with public key contained in the server's sender certificate/store and your private key to decrypt what the other server sends you.

2) Create an incoming WSS configuration (e.g. my_config_from_server_A) that will decrypt the incoming SOAP requests coming from server A:

You specify that you want to use your keystore to decrypt the message and the server's certificate/store to verify the signature of server A
(theses should appear in the drop box for each field):

3) Last, you want to specify that you are using the incoming configuration for each request:

Enjoy!

Problem initializing the class javax.crypto.SunJCE_b with soapUI ?

2011-06-27T11:18:00.000-07:00

Last week, I have decided to download and install the lastest soapUI PRO version 4.0 from Eviware.
I have been using soapUI for some time to test secure SOAP Web services using SSL/TLS as well as encrypting and signing the SOAP payload.

However when I tried to use my existing projects I end-up with the following errors in my soapUI log:

Wed Jun 22 11:43:05 PDT 2011:ERROR:java.lang.NoClassDefFoundError: Could not initialize class javax.crypto.SunJCE_b
   java.lang.NoClassDefFoundError: Could not initialize class javax.crypto.SunJCE_b
    at javax.crypto.KeyGenerator.a(DashoA13*..)
    at javax.crypto.KeyGenerator.(DashoA13*..)
    at javax.crypto.KeyGenerator.getInstance(DashoA13*..)
    at org.apache.ws.security.message.WSSecEncrypt.getKeyGenerator(WSSecEncrypt.java:701)
    at org.apache.ws.security.message.WSSecEncrypt.prepare(WSSecEncrypt.java:228)
    at org.apache.ws.security.message.WSSecEncrypt.build(WSSecEncrypt.java:291)
    at com.eviware.soapui.impl.wsdl.support.wss.entries.AddEncryptionEntry.process(AddEncryptionEntry.java:311)
    at com.eviware.soapui.impl.wsdl.support.wss.OutgoingWss.processOutgoing(OutgoingWss.java:157)
    at com.eviware.soapui.impl.wsdl.submit.filters.WssRequestFilter.filterWsdlRequest(WssRequestFilter.java:58)
    at com.eviware.soapui.impl.wsdl.submit.filters.AbstractRequestFilter.filterAbstractHttpRequest(AbstractRequestFilter.java:37)
    at com.eviware.soapui.impl.wsdl.submit.filters.AbstractRequestFilter.filterRequest(AbstractRequestFilter.java:31)
    at com.eviware.soapui.impl.wsdl.submit.transports.http.HttpClientRequestTransport.sendRequest(HttpClientRequestTransport.java:133)
    at com.eviware.soapui.impl.wsdl.WsdlSubmit.run(WsdlSubmit.java:123)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

A quick search on the web told me to check if my some of my security setup in my JRE was properly configured.
For this it was recommended to check the security directory under the lib folder (.\Java\jre6\lib\security):

I also checked the content of the content of the jar file ./Java/jre6/lib/jce.jar which should contain the missing class in question: javax.crypto.SunJCE_b

However everything looked fine. Even though my JRE was almost up-to-date, I finally managed to fix this issue by upgrading my java JRE to the latest version!

How to test secure web services with soapUI - part #1

2011-05-31T22:59:00.000-07:00

Recently I have been involved in a SOA project which goal is to orchestrate IHE integration profiles SOAP web services (XCPD & XCA) using an enterprise service bus (ESB).

The security characteristics of these web services include digital certificates for transport level security two-way transport layer security (TLS), message level security (encryption and digital signature) as well as single sign-on (SSO) based authentication using SAML 2.0 assertions.

One of the challenges was to find a tool we could use to create test harness for these web services, but also to simulate and mock some of these same services during development and testing.

For this I used soapUI. Overall it was not too complicated, except the support for SAML 2.0 since the current version of soapUI (4.0) only supports SAML 1.1 out-of-the-box.

Initially security features on the XCPD web service were turned off so we could test the basic SOAP web service functionalities. For this a new soapUI project was created by introspecting the XCPD service Web Services Description Language (WSDL) file:

To make the initial project simple, no test suite has been created and the initial operation has been re-labelled "RespondingGateway_XCPD" and its sample request has been modified to query a specific test patient (Joan Hunter) and renamed XCPD_PATIENT_JOAN_HUNTER:

The test is done by submitting the request to the XCPD end-point:

This returns in the response window the XCPD response.

In the next steps, we will setup all the security features for querying our XCPD service :

digital certificates for transport level security two-way transport layer security (TLS)
message level security (encryption and digital signature) as well as single sign-on (SSO) based authentication using SAML 2.0 assertions.

The SAML 2.0 assertion will be described in the part two of this post.

1. TLS setup

First we need to indicate that we are using SSL on top of HTTP for the transport layer. For this we need to setup the client keystore via file/preference/SSL settings (from the machine where soapUI will run):

2. Message level security

Probably the first thing you want is to specify your keystores and certificates if you decide to have mutal secure communication. You will use your private key from your key store (my_key_store.jks) to sign messages you send and decrypt the payloads you receive, and you will use specific server public key/certificate (a_another_server_keystore.jks) to encrypt the SOAP messages you send and verify signatures you receive. The server with whom you have secure mutual communication will do the reverse.

The soapUI message level security configuration for SOAP (WS-security) can be setup by selecting our soapUI project XCPD_Tests, right-click and select Show Project View and then add using the sign + in the out-going WS-Security configuration :

You then specify your configurations (e.g. my_config_to_server_A) for outgoing messages and similarly to ingoing messages.

Here how you would define a configuration in the Outgoing WS-security Configurations tab:

add a signature tab a
add an encryption tab (is you also use SAML 1.1 you would also add a SAML tab and a timestamp tab before adding the signature and the encryption).

2a. Signature

In the Signature tab, you specify:

the keystore that contain the private key to sign the messages you are going to send (usually your private key - expect if you want to mock another server)
the alias for the key to use for signature
the certificate password
the key identifier type (none, binary security token, X509 certificate or subject key identifier)
the signature algorithm (e.g SHA256)
the signature canonicalization
the number of certificates used to sign
the parts you want to sign (e.g. the content of the body of namespace http://www.w3.org/2003/05/soap-envelope)

2b. Encryption

In the Encryption tab, you specify:

the public key of the server you are sending your SOAP messages to
the alias for the key to use for encryption
the certificate password
the key identifier type (none, binary security token, Issuer Name and Serial, X509 certificate, subject key identifier, Embedded KeyInfo, Embedded SecurityToken Reference, Thumbprint SHA1 Identifier)
an embedded key name (if any)
an embedded key password (if any)
a symmetric encoding algorithm
a key encryption algorithm
the encryption canonicalization
the number of certificates used to sign
the parts you want to encrypt (e.g. the content of the body of namespace http://www.w3.org/2003/05/soap-envelope)

And finally, last but not least, you need to specify for the request which outgoing security configuration you want to choose. In our case we have defined only one configuration: my_config_to_server_A at the project level. To select the configuration, click on the Auth Tab at the bottom of the request window, select in the drop-down Outgoing WSS field your configuration. Create and configuring incoming Secure Web Services will be very similar.

You are now ready to test your SOAP Web Service with soapUI!

In part #2 we will look how SAML can be added to the mix.

How to test harness REST web services with cURL

2011-04-28T09:37:00.000-07:00

cURL is a very convenient command line tool to send and retrieve data using the URL syntax. It supports a large number of protocols: HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, LDAP, LDAPS, DICT, TELNET, FILE . IMAP, POP3, SMTP and RTSP.

I have been using cURL for quickly and conveniently testing RESTFul APIs.

Since I am doing most of my development on Windows platforms, I am using cURL in conjunction with Console, a Windows console window enhancement tool.

First you need to download and install cURL for your platform (windows, Linux ect). Then you can do a quick test by accessible the home page of your favorite web site:

This should display the HTML content of the home page. At this point I would suggest to look at the documentation to see what you can do with cURL including the FAQ.

I usually use simple test harness such as 'curl http://www.acme.com/api/versions' if you REST API exposes available versions.

You can then start to do more sophisticated tests such as authentication credentials from files (in my case I use a JSON data structure to automatically authenticate to my web service), saving cookies on files (using the the -c option):

curl -c ./cookies.txt --data-binary @login_password.json -H "Content-Type: application/json" http://www.acme.com/api/users/token

With my JSON file as follow: {"login":{"username":"user1","password":"StrongPassword1"}}

The cookie in my case contains a session token that I can reuse between each cURL calls. In the next call I read the cookie (via the -b option):

curl -b ./cookies.txt http://www.acme.com/api/users/userid1234567890/orders

The output of the command can be saved in a file using the option -o or redirecting our output:

curl -o google.html www.google.com

curl www.google.com > google.html

Enjoy!

In my next post on this topic, I will explain how to use SSL/TLS and specify GZIP headers to your cURL requests.

JUnit based integration testing with Simple JNDI

2011-03-31T10:38:00.000-07:00

I regularly use TJWS a light weight Java Web Server build as a servlet container to provide standard Web Server capabilities at building and testing time.

The main advantage is that I don't have to deploy my war file to my production server (e.g. JBoss) to test my Java Web application. My maven build runs complex integration JUnit tests immediately after building the war file with maven.

In my configuration however, I have to use JNDI (Java Naming and Directory Interface) to connect to a specific datasource (DB2 via JDBC) for some of my tests.

I used Simple JNDI for this which also offer an easy and elegant way to configure data source access.

Here are the steps I follow to setup my JUnit tests using Simple JNDI:

Add dependencies in Maven 2 Pom file for Simple JNDI, java persistence and DB2 JNDI/JDBC:

    
    <dependency>
        <groupId>simple-jndi</groupId>
        <artifactId>simple-jndi</artifactId>
        <version>0.11.4.1</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>javax.persistence</groupId>
        <artifactId>persistence-api</artifactId>
        <version>1.0</version>
        <scope>test</scope>
    </dependency>
  
    <dependency>
        <groupId>com.ibm.db2</groupId>
        <artifactId>db2jcc4</artifactId>
        <version>9.7.0.2</version>
        <scope>test</scope>
    </dependency>

Create a small Java class for the JNDI setup

     import javax.naming.InitialContext;
     import javax.sql.DataSource;

     public class JndiSetup { 
         /**
          * Setup the Data Source
          */
         public static void doSetup(String ds_name) {
             try {
                 InitialContext ctxt = new InitialContext();
                 DataSource ds = (DataSource) ctxt.lookup("jdbc."+ds_name);
                 // rebind for alias if needed
                 ctxt.rebind("jdbc/"+ds_name, ds);
             } catch (Exception ex) {
                 ex.printStackTrace();
             }
         }
     }

In more complex situations, you may have also to create an EntityManager and use an EntityManagerFactory.

In the JUnit java script code, setup your JNDI connection before running your tests:

    @BeforeClass
     public static void setUpClass() throws Exception {
          JndiSetup.doSetup("");
     }

Create a jndi.properties file in the "\src\test\resources" path in your project

 
     java.naming.factory.initial=org.osjava.sj.SimpleContextFactory
     org.osjava.sj.root=target/test-classes/config
     org.osjava.jndi.delimiter=/
     org.osjava.sj.jndi.shared=true

Create a jdbc.properties file in the "\src\test\resources\config" path in your project (I am using an IBM DB2 data source). Create the config directory if it doesn't exist. The "config" name comes from org.osjava.sj.root parameter in jndi.properties file. If you want a different name, "foo", for the folder, make sure to update the "org.osjava.sj.root" property and create a "foo" folder in "\src\test\resources" path. Make sure the directory /src/test/resources is in the Java build path, and the output folder is set to target/test-classes


     <your-ds>.type=javax.sql.DataSource
     <your-ds>.driver=com.ibm.db2.jcc.DB2Driver
     <your-ds>.url=jdbc:db2://<your-database-host>:50000/<your-ds-or-ds-alias>
     <your-ds>.user=<your-db-login>
     <your-ds>.password=<your-db-password>

With all of this you should be ready to test your integration using >mvn clean install.

You may have also to do a >mvn eclipse:eclipse -DdownloadJavadocs=true as well if you are using Eclipse and your new dependencies and imports do not work properly.

REST-Style Architecture and the Development of Mobile Health Care Applications

2011-02-28T15:23:00.000-08:00

Mobile devices offer new ways for users to access health care data and services in a secure and user-friendly environment. These new applications must be easy to create, deploy, test and maintain, and they must rely on a scalable and easily integrated infrastructure.

In the ambulatory health care environment, providers spend the majority of their time in an examination room with patients. Although some clinics have installed personal computers in the exam room for use at the point of care, many physician practices have yet to do so or have no such intention. Reasons for not installing PCs in the exam room include (among others) lack of space, security concerns, and cost. Often, clinics have PCs installed outside of the exam room to be used for encounter documentation or health history research (i.e., reviewing the patient's health records). This physical setup is often satisfactory for providers to complete their documentation needs. Providers often scratch rough notes on paper during an encounter, then dictate or type their notes after the visit has ended. The absence of computers in the exam room, however, is a disadvantage for research activities. Frequently, after listening to the patient's verbal health history, a provider wishes to read past records. If those records are in an electronic format, it is optimal to access those records at the point of care (i.e., in the exam room).

Thus, computer devices that are smaller and more mobile than a PC (e.g., smart phones, PDAs, tablets) would be the optimal hardware choice to access these electronic records. Given that many physicians carry smart phones, such mobile devices would be the ultimate tools to look up patient records.

Since the development of client applications on different mobile platforms requires more time than creating web applications for a handful of browsers, it is important to minimize the complexity of the integration with the back-end services and legacy systems and to try to decouple the development and maintenance of the client- and server-side components.

The Representational State Transfer (REST) architecture is an alternative to SOAP and offers clear advantages over SOAP including lightweight architecture, extensibility, scalability, easy of development, testing, deployment and maintenance.

REST API prototypes can be created in a matter of days and a full functioning set of sophisticated clinical based web services accessible by mobile client applications within few weeks.

In addition to this, REST APIs are particularly suitable for fast and loosely-coupled solution integration such as mobile applications, but can also be used in health care for portal and mash-up applications as well.

Reference:
Andry F., Wan L., Nicholson D., A mobile application accessing patients' health records through a REST API, 4th International Conference on Health Informatics (HEALTHINF 2011), pp 27-32, Rome 2011.

Increase your productivity on Windows platforms with Console

2011-01-29T10:46:00.000-08:00

A couple of years ago, one of my colleagues showed me Console, a very useful and nice Windows console window enhancement tool. Since then I have been using it and increased my productivity when it comes to command line tasks on Windows platforms. This is an open source software available on Source Forge.

With Console, you have all your Windows consoles within a single application - you also get:

multiple tabs
text editor-like text selection
different background types
alpha and color-key transparency
configurable font, different window styles

As a result, you can customize each console that appear in different tabs.

In the example below, I have created a Tab called "MY-MAVEN-BASED-PROJECT" that opens at a specific windows path with a particular prompt: "PROJECT-ROOT:" to build your project.

To configure your prompt, you need to do the following:

In the console settings tab, go to the shell field and enter:

cmd.exe /k "prompt <your-prompt>"

and for the startup directory, just specify the directory where you want to open your customized tab.

You can select your background color and style in the background customization tab:

Enjoy!

LotusScript Connectors for DB2

2010-12-28T12:55:00.000-08:00

In my previous post I was explaining how to install the IBM Data Server Runtime Client, including some ODBC DB2 drivers to access a DB2 Database remotely. In this new post I explain the bare minimum to install in order to access a DB2 Data Source via LotusScript Extension for Lotus Connectors (LS LSX-LC).

By the way, the ODBC DataDirect Lotus-branded drivers might be available only to paying customers ...

A. Installing the DB2 ODBC CLI drivers

First you need to install the DB2 Client Drivers on the System (which are different from the those that come with the IBM Data Server Runtime Client). I am using a Windows server, so after download I just unzip either v9.7fp3a_nt32_odbc_cli.zip (23 bits) or v9.7fp3a_ntx64_odbc_cli.zip (64 bits) in a folder (e.g. C:\clidriver).

Then I open a command prompt and navigate to my folder (e.g. C:\clidriver\bin) and type:

db2oreg1.exe -i -setup

Immediately following this you can set up the data sources in ODBC. Open Control Panel -> Administrative Tools -> Data Sources (ODBC).

You will see a screen that looks like the following, click on the "System DSN" tab:

Click "Add..." button to get the following, and select "IBM Data Server Driver for ODBC - C:/clidriver:

Enter a Data Source name, this is used directly in the Lotus Script( use the same name as the Database itself):

Enter the DB2 User ID and password that the agent uses to connect to DB2:

Select the "Save Password" option and click OK on the warning popup for saving the password in db2cli.ini file:

Click on the "Advanced Settings" tab:

Click "Add" and select the "Database" CLI Parameter:

Enter the Database name in the prompt and click OK:

Continue to do this for these parameters:

Database: The database name
Hostname: The DB2 server/host name (IP address is not recommended)
Port: 50000 (The default value for DB2 TCPIP accepting port)

B. Accessing the DB2 Database from LotusScript

1. Get access to the Lotus Connector Extensions (this is always installed)

      Option Public
      Option Explicit

      UseLSX "*lsxlc"

2. Create the LCSession object at the top of all functions or subroutines

      Dim session As New LCSession

3. Enable Connection pooling

      session.ConnectionPooling = true

4. Create the Connection, using the LCConnection class's constructor that takes a single argument (the name of the connector type). We're using ODBC, which has the Lotus Connector name of "odbc2".
      Dim conn As New LCConnection ("odbc2")
      conn.Server = "RLS" 'Using the ODBC DATA SOURCE name created previously.
      conn.Connect

   5. When done with the connection, disconnect - This will not actually disconnect if connection pooling is enabled

      conn.Disconnect

Queries

Querying DB2 from a LCConnection object takes a couple of variables for holding the field names. There are multiple ways to issue a query:

LCConnection Execute

The execute command takes a full SQL statement, which is useful to capture complex queries. Unfortunately LSX LC (like LS:DO) does not support any kind of parameterized query syntax or method calls. This means that the parameter values sent to the database need to be encoded specifically for DB2. This kind of encoding may be difficult from LotusScript, and therefore it is recommended that for complex queries we use stored procedures in either SQL or Java. For simple select queries involving one table (or potentially view) and "ANDed" WHERE clause predicates, one can use the Select method against the LCConnection class.

Execute example code:

Dim fldLst As New LCFieldList

conn.Execute "SELECT * from TEST.CUSTOMER", fldLst ' fldLst is only used for result set purposes

Set fld = fldLst.Lookup ("CUST_NAME")
While (conn.Fetch(fldLst) > 0)
 Dim sName As string
 sName = fld.text(0) '' Do something with this column value
Wend

LCConnection Select

The Select command is best described in the LC LSX Manual, as there are many options. In the example code it shows the user accessing a "count" of returned records, this is not accurate for the DB2 and ODBC setup we are using. Instead, like the Execute method, the count can only be determined by the amount of times we loop in Fetching each row.

When using Select you must set the Metadata property to the schema and table name you're selecting from. Always use the form "schemaname.tablename" to avoid runtime errors later.

Select example code:

Dim result As New LCFieldList

conn.Metadata = "TEST.CUSTOMER"

conn.Select Nothing, 1, result ' Is like SELECT * FROM TEST.CUSTOMER

Set fld = result.Lookup ("CUST_NAME")

While (conn.Fetch(result) > 0)
 MessageBox fld.text(0) ' display the result

Wend

For more on LotusScript see the IBM documentation on Lotus Domino

I would also like to thank you my colleague Ravi L. for his walk through step by step on this topic!

Database Alias and DB2 ODBC Drivers

2010-11-24T11:07:00.000-08:00

One of my recent project required to use ODBC to access DB2 databases located on remote VMWare LabManager images. I am using a Windows PC (Vista) laptop to develop and test my project (LotusScript Data Object code) - a Lotus Notes/Domino DB2 integration using ODBC. The first step for me was to install the ODBC DB2 drivers since I did not have DB2 installed on my laptop.

Several installations options were offered to me for DB2 9.7:

IBM Data Server Driver for ODBC and CLI (CLI Driver) : the smallest footprint - a 13MG zip file (installing the drivers manually might be a little bit tricky - see my next post on this)
IBM DB2 (server + client tools) : this would work but this much more than I need since DB2 is already installed on my remote LabManager image
IBM Data Server Runtime Client : the best solution for me (34MB zip file), especially if I have installed other tools such as IBM Data Studio for managing my DB2 databases instances remotely.

The installation of the IBM Data Server Runtime Client is very fast and straightforward. It installs the ODBC/CLI drivers and a small set of useful command line setup tools:

After this, we can create the Database Aliases using the Windows ODBC Data Source Administrator.
When you look at the Drivers tab, you should now see your DB2 ODBC drivers

To add a DB2 Data Source Name (DSN):

select User or System and click on the Add... button.
select the DB2 ODBC/CLI driver
enter a Data source name and add an Alias if needed (click on the Add button next to your existing aliases if needed)
enter Data Source parameters (Description, user ID, password) - click "Save password" checkbox to save your login and password locally in your db2cli.ini file.
enter your TCP/IP connection (port number is 50000 by default for me for DB2), the host name is the IP address of my DB2 server VMWare image.
I did not have to change anything in the defaults of Security options and Advanced Settings.

From there you are ready to use your ODBC DSN ready to connect to your DB2 Database.

One issue you will encounter though will be how to delete an existing Database Alias from the DB2 ODBC tab either to modify an existing one or to remove an old one.
These appear in the drop down of the ODBC IBM DB2 Driver - Add popup window.

The truth is that even though you are accessing a remote DB2 server machine, these aliases are stored locally on your DB2 installation.
To remove the DB2 Database Aliases ODBC drivers, just start the IBM DB2 Command Line Processor and use the following command:

UNCATALOG DATABASE <database_alias>

In certain cases, you also need to refresh the directory cache. For this, just stop and restart the DB2 Management Service on your local Windows machine.

Healthcare REST APIs - JSON or XML?

2010-10-29T13:39:00.000-07:00

I have been working recently on a REST API which produces subsets of Continuity of Care Documents (CCD). This REST API is used by an iPhone application which is targeted to physicians and nurses. Since I wanted to minimize the amount of data exchange between the server and the client, I originally used JSON as my data exchange format. The motivation to use JSON was to have a compact format that offers better performance than a more complex XML representation.

For example, the request to obtain lab-results from a CCD is as following:

GET /users/<user-id>/patients/<patient-id>/lab-results?hl7v3=true&max=<max>offset=&<offset>

The resulting of this request to the API is a JSON object containing a list of lab results:

{"lab-results":{
    "list":[{"lab-result":{"entry":"...",
                           "facility":"...",
                           "normalcy":"...",
                           "orderedBy":"...",
                           "status":"...",
                           "subject":"...",
                           "urgency":"..."}},
            {"lab-result":{...}},...],
    "count":"...",
    "offset":"...",
    "remain":"..."}}

A lab result HL7 V3 entry is returned as the following JSON object:

{"entry":{
    "organizer":{
        "code":{"displayName":"..."}},
        "components":[
            {"component":{...}},
            {"component":{...}},...],
        "notes":[...]}}}

A lab-result component itself:

{"component":{
    "observation":{
        "code":{"displayName":"..."},
        "effectiveTime":{"value":""},
        "value":...,
        "interpretationCode":{"code":"..."},
        "referenceRange":{
            "observationRange":{...}},
        "notes":[...]}}}

An observation value is returned as a JSON object containing either a string value, a unit and a type, or just some text.

{"value":{"unit":"...","value":"...",type:"..."}}

{"value":"..."}

An observationRange is returned as a JSON value object containing a low and high value, or just some text.

{"observationRange":{
    "value":{
        "low":{"value":"..."},
        "high":{"value":"..."}}}}

{"observationRange":{"text":"..."}

All these JSON objects are marshalled from annotated Java POJOs using JBOSS RestEasy framework and Jackson:

XmlRootElement(name = "high")
public class HighValue {

 private String value = "";

 /**
  * Construct a new instance.
  */
 public HighValue() { }    // Empty constructor

 /**
  * Create a new {@code HighValue} during JAXB unmarshalling.
  * @param value
  *            String as value for the high value.
  */
 public HighValue(final String value) {
  if (value != null)
   this.value = value.trim();
 }

 /**
  * Get the {@code value} attribute.
  * @return {@code value} attribute value (may be {@code null}).
  */
 @XmlElement
 public String getValue() {
  return value;
 }

 /**
  * Set the {@code value} attribute.
  * @param value
  *            value to set.
  * @see #getValue()
  */
 public void setValue(final String value) {
  if (value != null)
   this.value = value.trim();
 }
}

This was fine initially since I was focusing on just lab results and I was using a specific back-end API that providing values to populate my POJOs. This solution started to become more complex when I was asked to generated a large set of CCD data types. As a result, the number of Java objects became quickly larger.

The other option I had was to use another internal API I could use which was already generated full or subset of CCD. However the resulting CCD format provided was in XML:

<component>
  <observation classCode="OBS" moodCode="EVN">
    <templateId root="2.16.840.1.113883.10.20.1.31"/>
    <templateId root="1.3.6.1.4.1.19376.1.5.3.1.4.13"/>
    <templateId root="2.16.840.1.113883.3.88.11.83.15"/>
    <id root="1"/>
      <code code="Remark" codeSystemName="L" displayName="Remark"/>
      <text>
        <reference value="#Observation_504ccbaf5ecea7b1096720"/>
      </text>
      <statusCode code="completed"/>
      <effectiveTime value="20091223231100"/>
        <value xsi:type="ST">Spec #106641063: 23 Dec 09  2311</value>
      <interpretationCode code="N" codeSystem="2.16.840.1.113883.5.83" codeSystemName="ObservationInterpretation" displayName="Normal"/>
  </observation>
</component>

I could of course just used it as it is and have my REST API return XML CCD subsets in XML:

GET /users/<user-id>/patients/<patient-id>/CCD&section=<section>

They are several issues with this:

As you can see XML is much more complex to understand, parse and debug than JSON
XML increases bandwidth consumption
Browsers and client application (e.g. mobile devices) can consume JSON much more efficiently than XML

For me, the best solution was to have the internal API marshalling the CCD in both XML and JSON so I will not have to unmarshall the CCDs again into POJOS.

The good news for all of us is that you can use java tools such as JAXB which has adapters to support other formats than XML such as JSON. With Java annotations, this is very easy to implement.

Spring Dependency Injection with JBOSS : the CLASSPATH issue

2010-09-10T11:31:00.000-07:00

When facing the problem of deploying web archives (war) to be configured through Spring dependency injection, you probably want to have generic applications that do not have to be recompile every time you deploy them on new configurations.

In my current project I need to configure a REST API with various parameters (host name, database paths, maximum of records per request). For this I use Spring dependency injection where the parameters are injected at run-time via a resource file located outside the war file, in a folder specified by the Windows CLASSPATH variable (my testing and production platforms are windows machine).

First I need to add a windows CLASSPATH system variable (in your system properties/environment variables) if this variable does not exist. Then I add the resources.xml file directly in the folder specified by CLASSPATH. You can also use a sub-folder but you will need to hard-code the name of the folder in your spring config file - in my case applicationContext.xml located in ./src/main/webapp/WEB-INF/

<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xmlns:context="http://www.springframework.org/schema/context"
 xsi:schemaLocation="
        http://www.springframework.org/schema/context 
        http://www.springframework.org/schema/context/spring-context-2.5.xsd
        http://www.springframework.org/schema/beans 
        http://www.springframework.org/schema/beans/spring-beans.xsd">
    <import resource="classpath:/resources.xml" />
</beans>

My resources.xml looks like this:

<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xmlns:context="http://www.springframework.org/schema/context"
 xsi:schemaLocation="
        http://www.springframework.org/schema/context 
        http://www.springframework.org/schema/context/spring-context-3.0.xsd
        http://www.springframework.org/schema/beans 
        http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
  <bean id="custService" 
                      scope="prototype"
                      class=".....">
   <property name="hostName" value="121.122.123.124"/>
   <property name="databasePath" value="..."/>
   <property name="maxRecordsPerRequest" value="1000"/>
  </bean>

</beans>

One issue you might encounter when you try to deploy your application on JBoss is that the web application server does not take into account the CLASSPATH out-of-the-box (I am using redhat EAP 5.0.X - production setting), but this might be also the case with JBOSS community edition.

Your war file will probably fail to deploy and you will find a bunch of errors in your log file ./jboss-as/server/<setting>/log/server.log including:

org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: 
Failed to import bean definitions from URL location [classpath:/resources.xml]
Offending resource: ServletContext resource [/WEB-INF/applicationContext.xml]; nested exception is org.springframework.beans.factory.BeanDefinitionStoreException:
IOException parsing XML document from class path resource [resources.xml]; 
nested exception is java.io.FileNotFoundException: 
class path resource [resources.xml] cannot be opened because it does not exist

What is missing is that you need to tell JBoss about your CLASSPATH variable.
Just edit ./jboss-as/bin/run.bat and add the CLASSPATH variable and you will be up and running in no time.

:RESTART
"%JAVA%" %JAVA_OPTS% ^
   -Djava.endorsed.dirs="%JBOSS_ENDORSED_DIRS%" ^
   -classpath "%JBOSS_CLASSPATH%;%CLASSPATH%" ^
   org.jboss.Main -b 0.0.0.0 -c production %*

RESTeasy JAX-RS embeddable server and SpringBeanProcessor

2010-08-24T13:09:00.000-07:00

TJWS (Tiny Java Web Server and Servlet Container) is a very convenient miniature Java Web Server build as a servlet container with HTTPD servlet providing standard Web server functionality.

I have been using TJWS for testing a REST API to be deployed on JBoss application server. The advantage is that JUnit tests can run without the need to deploy a war file on JBoss. Since I have implemented the REST API with RESTEasy, I am using the embedded TJWS server part of the org.jboss.resteasy.plugins.server.tjws.TJWSEmbeddedJaxrsServer package.

The RESTEasy documentation (chapter 23) describes how to use the embedded container.

@Path("/")public class MyResource {

   @GET
   public String get() { return "hello world"; }
 
   public static void main(String[] args) throws Exception 
   {
      TJWSEmbeddedJaxrsServer tjws = new TJWSEmbeddedJaxrsServer();
      tjws.setPort(8081);
      tjws.getRegistry().addPerRequestResource(MyResource.class);
      tjws.start();
   }
}

As you can see, TJWS is very simple to use. You create an instance of the server, setup the port (this is very useful when for example certain ports are already used - I had to set a specific port for our Hudson continuous builds). Then you specify the class to test and you start the server.

In my JUnit tests, I start the server before each tests and stop it after the tests are completed:

private TJWSEmbeddedJaxrsServer server; 

@Before
    public void start() {
      
     server = new TJWSEmbeddedJaxrsServer();
     server.setPort(SERVER_PORT);
     server.getDeployment().getActualResourceClasses().add(MyResource.class);
     server.start();
    }

@After
    public void stop() {
     server.stop();
    }

Since I am using Spring I was interested to leverage the framework for dependency injection in order to configure certain server settings. However the RESTeasy documentation provides only some pseudo-code example:

public static void main(String[] args) throws Exception 
   {
      final TJWSEmbeddedJaxrsServer tjws = new TJWSEmbeddedJaxrsServer();
      tjws.setPort(8081);

      org.resteasy.plugins.server.servlet.SpringBeanProcessor processor = new SpringBeanProcessor(tjws.getRegistry(), tjws.getFactory();
      ConfigurableBeanFactory factory = new XmlBeanFactory(...);
      factory.addBeanPostProcessor(processor);

      tjws.start();
   }

I had to make some modifications to the code provided as follow:

@Before
    public void start() {
      
     server = new TJWSEmbeddedJaxrsServer();
     server.setPort(SERVER_PORT);
     server.getDeployment().getActualResourceClasses().add(MyResource.class);
     server.start();
     
     Resource resource = new FileSystemResource("src/test/resources/resources.xml");
     ConfigurableListableBeanFactory factory = new XmlBeanFactory(resource);
     SpringBeanProcessor processor = new SpringBeanProcessor(
             server.getDeployment().getDispatcher(),
             server.getDeployment().getRegistry(), 
             server.getDeployment().getProviderFactory());
     processor.postProcessBeanFactory(factory);
    }

Alternatively you can define your Spring resource file in a static string directly in your JUnit test class:

Resource resource = new ByteArrayResource(SPRING_BEAN_CONFIG_FILE.getBytes());

How to secure the JBoss JMX and Web Consoles?

2010-07-28T17:18:00.000-07:00

Lately I have been using JBoss more and more as my deployment platform of choice. I am currently using the latest JBoss Enterprise Middleware solution (EAP version 5.0.1). This is a commercial version, but you can also use the community edition as well which offers most of the same features.

One of the issue I have encountered recently was how to secure the web based administration consoles?

As a matter of fact, the default installation offers a login and password for the admin console which is typically accessible on http://localhost:8080/admin-console if your web application runs locally, or more generally on http://<host>:<port>/admin-console.
However if you want to protect the JMX console (http://localhost:8080/jmx-console) and the JBoss Web console (http://localhost:8080/web-console) you have to make sure that certain files in your installation are setup correctly.

Generally, the JBoss community and RedHat are quite good at documenting the features of their products, but I was disappointed to find incomplete information in the main page on this subject.

This page explains that "the jmx-console and web-console are standard servlet 2.3 deployments that can
be secured using J2EE role based security. Both also have a skeleton setup to allow one to easily enable security using username/password/role mappings found in the jmx-console.war and web-console.war deployments in the corresponding WEB-INF/classes users.properties and roles.properties files".

Until this point, it is quite clear. The difficulty starts with a vague description where to find the files in questions :

To secure the JMX Console using a username/password file -

Locate the directory. This will normally be in directory..

The author probably assumes that the various locations are obvious to everyone. Let me be more precise and generous in details:

First you will need to know which profile/configuration you are running. JBoss EAP has six configurations based on your needs:

all (everything, including clustering support and other enterprise extensions)
default (for application developers)
minimal (the strict minimum)
production (everything but optimized for production environments)
standard (tested for Java EE compliance)
web (experimental lightweight configuration)

If you did not specify your profile explicitly at starting time (run.sh -c )you most likely use the default profile.

You can check which profile is running by looking at your JBoss EAP Admin Console. The name of the profile/configuration is indicated at the top of the server hierarchy.

By the way, the various applications that you will likely developed (war, ear, rar or jar files) will be deployed under the corresponding folders under the Applications node.

First you might want to change the login and password of the admin console itself before adding those for the JMX and Web consoles?

One important "meta" file is login-config.xml which is located under \jboss-as\server\

This file specify for a specific profile (e.g. default) the security-domain values for the consoles:

<application-policy name = "jmx-console">
    <authentication>
      <login-module code="org.jboss.security.auth.spi.UsersRolesLoginModule"
        flag="required">
        <module-option name="usersProperties">props/jmx-console-users.properties</module-option>
        <module-option name="rolesProperties">props/jmx-console-roles.properties</module-option>
      </login-module>
    </authentication>
  </application-policy>

<application-policy name = "web-console">
    <authentication>
      <login-module code="org.jboss.security.auth.spi.UsersRolesLoginModule"
        flag="required">
        <module-option name="usersProperties">web-console-users.properties</module-option>
        <module-option name="rolesProperties">web-console-roles.properties</module-option>
      </login-module>
    </authentication>
  </application-policy>

In other words, this file can help you locate and map how authentication (login and password in users.properties file) and authorization (access control in roles.properties file) is specified.

Since these consoles are themselves web applications, you will need to look at exploded war files under your profile, under the deploy folder.

Securing the JMX Console:

Locate the folder jmx-console.war under ./server/<config>/deploy
Open the file ./server/<config>/deploy/mx-console.war/WEB-INF/web.xml
Verify that the <security-constraint> section is not commented (in this section, you should see specified the roles for authorization (see below)

<security-constraint>
     <web-resource-collection>
       <web-resource-name>HtmlAdaptor</web-resource-name>
       <description>An example security config that only allows users with the
         role JBossAdmin to access the HTML JMX console web application
       </description>
       <url-pattern>/*</url-pattern>
     </web-resource-collection>
     <auth-constraint>
       <role-name>JBossAdmin</role-name>
     </auth-constraint>
   </security-constraint>

Locate the file: .\server\<config>\conf\props\jmx-console-users.properties (if the file name has not been changed in login-config.xml)
change admin=admin to your new <new_login>=<new_password>

The authentication method is specified in the following section:

 <login-config>
      <auth-method>BASIC</auth-method>
      <realm-name>JBoss JMX Console</realm-name>
   </login-config>

   <security-role>
      <role-name>JBossAdmin</role-name>
   </security-role>

Securing the Web Console:

Login credentials are the same as used for the JMX console - in : .\server\<config>\conf\props\jmx-console-users.properties
change admin=admin to your new <new_login>=<new_password>

For more information and this topic you can also look at Securing the JMX Console and Web Console (HTTP).

Response objects and the use of GenericEntity class with RESTEasy

2010-06-10T14:50:00.000-07:00

Recently during the implementation of a REST API, I wanted to return a complex response containing a list of objects (Patients). The issue was that the RESTEasy build-in JAXB MessageBodyWriter could not directly handle lists of JAXB objects (Java has trouble obtaining generic type information at runtime).

I was recently in a situation where I had to create a complex response to a HTTP POST for my REST API. I am using JAXB /JSON support from RESTEasy.

I found some element of answer in the book "RESTFul Java with JAX-RS" from Bill Burke (pp 102). However the code snippet had a couple of errors:

• the GenericEntity object cannot be passed to the Response.ok() method directly (a ResponseBuilder is required).

• references to GenericEntity needs to be parameterized.

My use case is a little more complex than in the book. I am receiving a user-name and password from a POST (e.g. a form submit). I then perform the authentication and returns a list of Patient objects in a JSON/GZIP compressed format (instead a list of Customer objects) together with an authentication token.

The resulting code looks like this:


   @POST
   @Path("/token")
   @Consumes("application/x-www-form-urlencoded")
   @Produces("application/json")
   @GZIP
   public Response getPatientsWithToken(@FormParam("username") String username, @FormParam("password") String password) {
  
        Login login = new Login(username, password);
        // ... perform authentication here ....
    
        // Build the returning patient list
        List<Patient> returnList = new ArrayList<Patient>();
        returnList.addAll(patients.values());
        Collections.sort(returnList);
      
        GenericEntity<List<Patient>> entity = new GenericEntity<List<Patient>>(returnList){};
      
        // Create the response
        ResponseBuilder builder = Response.ok(entity);
        return builder.build();
   }

Of course you will have to import the following classes as well:

import javax.ws.rs.core.GenericEntity;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.Response.ResponseBuilder;

Open APIs: State of the Market, May 2010

2010-06-09T13:25:00.001-07:00

Today, I was looking at the presentation from John Musser related to Open APIs (see below). Even though these statistics comes mainly from mashup and consumer applications, I was surprised by the fact that REST APIs are gaining market shares over SOAP APIs so rapidly.

In B2B and in the enterprise world in general SOAP is often the top choice. The advantages for SOAP often mentioned are:

Type checking (via the WSDL files)
Availability of development tools

On the other hand, REST offers the following:

Lightweight and easy to build
Human Readable Results
Extensibility
Scalability

In Health Care, SOAP is still widespread and prevalent. However there are some interesting projects such as NHIN Direct Health Information Exchange where the relevance of REST vs other API protocols are discussed.

It will be interesting to see what will be the outcome of such discussions.

Open APIs: State of the Market, May 2010

View more presentations from jmusser.

JAXB-JSON Rest API using RESTEasy on JBoss EAP

2010-06-01T17:32:00.000-07:00

In this second part of my evaluation of JBoss RESTEasy, I focus on adapting the JAXB-JSON samples provided by RESTEasy for JBoss Enterprise Application Platform 5.0.1.

Earlier I find myself to adapt the maven POM file to have the proper dependencies for the Twitter RESTEasy client.

Initially this simple JAXB-JSON sample had been designed to run on Jetty Web Server which run fine out-of-the box. However I had to make some modifications to the original project structure to have the code running as a simple eclipse project that can be deploy on JBoss EAP 5.0.X from RedHat (this will also work on Jboss community edition).

The new project (eclipse) structure looks as below:

Notice that I have also moved the code for both packages: org.jboss.resteasy.annotations.providers.jaxb.json
org.jboss.resteasy.plugins.providers.jaxb.json
at the root of my project, since I am not using the remaining code of the example.

I also made some additional adaptations for JBoss to some of the project files including:

pom.xml file
web.xml

I also added a small test suite to test the REST API operations using JUnit which will work after the first deployment (I run JBoss locally on http://localhost:8080).

By the way, make sure you have src/main/resources/META-INF/services/javax.ws.rs.ext.Providers included in your project.

Here is the content of my new pom.xml file for JBoss:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.jboss.resteasy.examples</groupId>
    <artifactId>jaxb-json</artifactId>
    <version>0.1.0</version>
    <packaging>war</packaging>
    <name/>
    <description/>

    <repositories>
        <repository>
            <id>java.net</id>
            <url>http://download.java.net/maven/1</url>
            <layout>legacy</layout>
        </repository>
        <repository>
            <id>maven repo</id>
            <name>maven repo</name>
            <url>http://repo1.maven.org/maven2/</url>
        </repository>
        <!-- For resteasy -->
        <repository>
            <id>jboss</id>
            <name>jboss repo</name>
            <url>http://repository.jboss.org/maven2</url>
        </repository>
    </repositories>
    <dependencies>
    
        <!-- core library -->
        
        <dependency>
            <groupId>org.jboss.resteasy</groupId>
            <artifactId>resteasy-jaxrs</artifactId>
            <version>1.2.1.GA</version>
            <!-- filter out unwanted jars -->
            <exclusions>
                <exclusion>
                    <groupId>commons-httpclient</groupId>
                    <artifactId>commons-httpclient</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>tjws</groupId>
                    <artifactId>webserver</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>javax.servlet</groupId>
                    <artifactId>servlet-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        
        <!-- optional modules -->
        
      <dependency>
            <groupId>org.jboss.resteasy</groupId>
            <artifactId>resteasy-jettison-provider</artifactId>
            <version>1.2.1.GA</version>
        </dependency>
        
        <!-- modules already provided by Java 6.0 -->
         
   <dependency>
       <groupId>javax.xml.bind</groupId>  
       <artifactId>jaxb-api</artifactId>
       <version>2.1</version>
       <scope>provided</scope>
   </dependency>
   
    <dependency>
         <groupId>junit</groupId>
          <artifactId>junit</artifactId>
          <version>4.1</version>
                 <scope>test</scope>
             </dependency>
     
    </dependencies>

    <build>
        <finalName>jaxb-json</finalName>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>jboss-maven-plugin</artifactId>
                <version>1.4</version>
                <configuration>
                    <jbossHome>C:\JBoss\EnterprisePlatform-5.0.0.GA\jboss-as</jbossHome>
             <contextPath>/</contextPath>
             <serverName>default</serverName>
             <fileName>target/jaxb-json.war</fileName>
          </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.5</source>
                    <target>1.5</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

I modified the web.xml for the URL looks more simple by removing the mapping to reasteasy:

<?xml version="1.0"?>
<!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
        "http://java.sun.com/dtd/web-app_2_3.dtd">

<web-app>
   <display-name>Archetype Created Web Application</display-name>

   <context-param>
      <param-name>javax.ws.rs.Application</param-name>
      <param-value>org.jboss.resteasy.examples.service.LibraryApplication</param-value>
   </context-param>

   <context-param>
      <param-name>resteasy.servlet.mapping.prefix</param-name>
      <param-value>/</param-value>
   </context-param>

   <listener>
      <listener-class>
         org.jboss.resteasy.plugins.server.servlet.ResteasyBootstrap
      </listener-class>
   </listener>

   <servlet>
      <servlet-name>Resteasy</servlet-name>
      <servlet-class>
         org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher
      </servlet-class>
   </servlet>

   <servlet-mapping>
      <servlet-name>Resteasy</servlet-name>
      <url-pattern>/</url-pattern>
   </servlet-mapping>

</web-app>

The JUnit code looks like this:

package org.jboss.resteasy.examples.test;
 
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

import junit.framework.Assert;
import junit.framework.Test;
import junit.framework.TestCase;
import junit.framework.TestSuite;
  
public class LibraryTest extends TestCase
{
    /**
     * Create the test case
     *
     * @param testName name of the test case
     */
    public LibraryTest( String testName )
    {
        super( testName );
    }

    /**
     * @return the suite of tests being tested
     */
    public static Test suite()
    {
        return new TestSuite( LibraryTest.class );
    }

      
    /**
     * Testing the Library REST API
     */
    
    public void testGetMapped()  
    {
     validateRESTCall("GET", "http://localhost:8080/jaxb-json/library/books/mapped");
     assertTrue( true );
    }
    

    public void testGetBadger()  
    {
     validateRESTCall("GET", "http://localhost:8080/jaxb-json/library/books/badger");
     assertTrue( true );
    }
    
    private void validateRESTCall(String method, String url) {
     
     try {
         System.out.println("*** "+method);
         URL resURL = new URL(url);
         System.out.println("URL: " + url.toString());
         HttpURLConnection connection = (HttpURLConnection) resURL.openConnection(); 
         connection.setRequestMethod(method);
         System.out.println("Content-Type: " + connection.getContentType());
         
         BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
 
         String line = reader.readLine();
         while (line != null)
         {
            System.out.println(line);
            line = reader.readLine();
         }
         Assert.assertEquals(HttpURLConnection.HTTP_OK, connection.getResponseCode());
         connection.disconnect();
     } catch (Exception err) { 
         System.out.print("Error in VHRResourceTest.validateRESTCall : " + err); 
        };
    }
}

To build I am using Maven (I recommend to install the maven eclipse plugin) with the goals mvn clean install compile package. Make sure also that before that you do a mvn eclipse:eclipse to update the dependencies in your project.

To deploy jaxb-json.war file from eclipse (so I don't have to manually copy the war file from the target folder), I have installed the JBoss eclipse plugin. As a result I can make it deployable (accessible by a right-click) and it appear in the Eclipse JBoss server view:

The REST API JSON Library resources are then accessible directly on a browser via http://localhost:8080/jaxb-json/library/books/mapped or http://localhost:8080/jaxb-json/library/books/badger.

If you want to compress your response, RESTEasy provides GZIP Compression/Decompression support using a very simple @GZIP annotation:

   @GET
   @Path("books/mapped")
   @Produces("application/json")
   @GZIP
   public BookListing getBooksMapped()
   {
      return getListing();
   }

Just import the following class:

import org.jboss.resteasy.annotations.GZIP;

Overall the adaptation from Jetty to JBoss was easy and the documentation very clear.

Additional discussions, recommendations and information can be found on the JBoss Community.

For an example of using REST architecture for Mobile Applications (HealthCare) see this post.

Enhanced POM for JBoss RESTEasy Twitter API Client Sample

2010-05-18T11:10:00.000-07:00

I recently looked at JBOSS RESTEasy as a way to create and test RESTful APIs. The platform looks very promising with a lot of praise from developers. Also the documentation seems very extensive and precise.

I started by downloading RESTEasy 1.2.1 GA and tried the sample code. I started with a java client to access existing RESTful Web Services and APIs. Among the api-clients, there is a Twitter small client that works out-of-the box (located under /RESTEASY_1_2_1_GA/examples/api-clients/src/main/java/org/jboss/resteasy/examples/twitter).

However when I started to extract the code and wanted to create a Maven 2 based stand-alone project, I encountered some issues related to JAR dependency conflicts, including the following error message also described here.

java.lang.NoClassDefFoundError: Could not initialize class com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl

The project (eclipse) structure looks as below:

I managed to fix these issues by modifying the POM file as follow:

<?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
 
    <groupId>org.jboss.resteasy.examples</groupId>
 <artifactId>api-clients</artifactId>
 <version>1.2.1.GA</version>
  
  <dependencies>
    <!-- Resteasy Core -->
    <dependency>
      <groupId>org.jboss.resteasy</groupId>
      <artifactId>resteasy-jaxrs</artifactId>
    </dependency>
    <!-- JAXB support -->
   <dependency>
      <groupId>org.jboss.resteasy</groupId>
      <artifactId>resteasy-jaxb-provider</artifactId>
   </dependency>
    
  </dependencies>
  <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.jboss.resteasy</groupId>
                <artifactId>resteasy-bom</artifactId>
                <version>1.2.1.GA</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
   </dependencyManagement>
   
   <!-- Build Settings --> 
   <build>
    <plugins>  
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
    </plugins>
   </build>
  
  <!-- Environment Settings -->
  <repositories>
    <repository>
      <id>jboss</id>
      <name>jboss repo</name>
      <url>http://repository.jboss.org/maven2</url>
     </repository>
   </repositories>
  
</project>

The most important piece, beside the cleaning of the POM file, was to include a pom that can be imported so the versions of the individual modules do not have to be specified (see RESTEasy documentation - Chapter 43. Maven and RESTEasy).

I also made sure to have correct dependencies for resteasy-jaxrs and resteasy-jaxb-provider.

As a result, I was able to compile the whole project without any errors (mvn clean compile) and run it to access the Twitter REST API

mvn exec:java -Dexec.mainClass="org.jboss.resteasy.examples.twitter.TwitterClient" -Dexec.args="<userid> <password>"

(Replace last parameters by your twitter user and password).

The small client in question leverages JAX-RS annotations to read and write the Twitter API resources:

package org.jboss.resteasy.examples.twitter;

import java.util.Date;
import java.util.List;

import javax.ws.rs.FormParam;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;

import org.apache.commons.httpclient.Credentials;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.UsernamePasswordCredentials;
import org.apache.commons.httpclient.auth.AuthScope;
import org.jboss.resteasy.client.ProxyFactory;
import org.jboss.resteasy.client.ClientExecutor;
import org.jboss.resteasy.client.core.executors.ApacheHttpClientExecutor;
import org.jboss.resteasy.plugins.providers.RegisterBuiltin;
import org.jboss.resteasy.spi.ResteasyProviderFactory;

public class TwitterClient
{
   static final String friendTimeline = "http://twitter.com/statuses/friends_timeline.xml";

   public static void main(String[] args) throws Exception
   {
      RegisterBuiltin.register(ResteasyProviderFactory.getInstance());
      final ClientExecutor clientExecutor = new ApacheHttpClientExecutor(createClient(args[0], args[1]));
      TwitterResource twitter = ProxyFactory.create(TwitterResource.class,
            "http://twitter.com", clientExecutor);
      System.out.println("===> first run");
      printStatuses(twitter.getFriendsTimelines());
      
      twitter
      .updateStatus("I programmatically tweeted with the RESTEasy Client at "
            + new Date());
      
      System.out.println("===> second run");
      printStatuses(twitter.getFriendsTimelines());
   }

   public static interface TwitterResource
   {
      @Path("/statuses/friends_timeline.xml")
      @GET
      Statuses getFriendsTimelines();

      @Path("/statuses/update.xml")
      @POST
      Status updateStatus(@FormParam("status") String status);
   }

   private static void printStatuses(Statuses statuses)
   {
      for (Status status : statuses.status)
         System.out.println(status);
   }

   private static HttpClient createClient(String userId, String password)
   {
      Credentials credentials = new UsernamePasswordCredentials(userId,
            password);
      HttpClient httpClient = new HttpClient();
      httpClient.getState().setCredentials(AuthScope.ANY, credentials);
      httpClient.getParams().setAuthenticationPreemptive(true);
      return httpClient;
   }

   @XmlRootElement
   public static class Statuses
   {
      public List<Status> status;
   }

   @XmlRootElement
   public static class Status
   {
      public String text;
      public User user;

      @XmlElement(name = "created_at")
      @XmlJavaTypeAdapter(value = DateAdapter.class)
      public Date created;

      public String toString()
      {
         return String.format("== %s: %s (%s)", user.name, text, created);
      }
   }

   public static class User
   {
      public String name;
   }

}

The small DateAdapter class is a utility class for date formatting:

package org.jboss.resteasy.examples.twitter;

import java.util.Date;
import java.util.List;
package org.jboss.resteasy.examples.twitter;
 
import java.util.Date;
import javax.xml.bind.annotation.adapters.XmlAdapter;
import org.jboss.resteasy.util.DateUtil;

public class DateAdapter extends XmlAdapter<String, Date> {

   @Override
   public String marshal(Date date) throws Exception {
       return DateUtil.formatDate(date, "EEE MMM dd HH:mm:ss Z yyyy");
   }

   @Override
   public Date unmarshal(String string) throws Exception {
       try {
           return DateUtil.parseDate(string);
       } catch (IllegalArgumentException e) {
           System.err.println(String.format(
                   "Could not parse date string '%s'", string));
           return null;
       }
   }
}

SOA and Health Care Meaningful Use requirements of the Recovery Act

2010-04-16T16:36:00.000-07:00

The Interim Final Rule of the Health Information Technology for Economic and Clinical Health (HITECH) Act was passed by Congress in February of 2009. Under this act, eligible providers will be given financial rewards if they demonstrate "meaningful use" of "certified" Electronic Health Record (EHR) technologies.

Therefore there is a big incentive for health care vendors to offer solutions that meet the criteria described in the law. More precisely, the associated regulation provided by the Department of Health and Human Services describes the set of standards, implementation, specifications and certification for Electronic Health Record (EHR) technology.

As a Software Architect, I was curious to see whether Service Oriented Architecture (SOA) or Web Services in general were mentioned in these documents.

The definition of an EHR Module includes an open list of services such as electronic health information exchange, clinical decision support, public health and health authorities information queries, quality measure reporting etc.

In the transport standards section, both SOAP and RESTful Web services protocols are described. However Service Oriented Architecture (SOA) is never explicitly described or cited. No reference how these services might be discovered and orchestrated in a "meaningful way". I would assume that the reason is that the law makers and regulators wanted to be as vague as possible on the underlying technologies for an EHR and its components.

The technical aspect of "meaningful use" is specified more precisely when associated with interoperability, functionality, utility, data confidentiality and integrity of the data, security of the health information system in general.

These characteristics are not necessarily specific to SOA, but to any good health care software and solution design.

Still, the following paragraph seems to describe a solution that could be best implemented using a Service Oriented Architecture: "As another example, a subscription to an application service provider (ASP) for electronic prescribing could be an EHR Module" where software is offered as a service (SaaS). This looks more like the description of an emerging SOA rather than a full grid enabled SOA.

It will be up to the solutions providers to come up with relevant products and tools to maximize the return on investment (ROI) of the tax payer's money and the professionals and organizations eligible for ARRA/HITECH.

SOA will definitively be part of the mix since it gives the ability create, offer and maintain large numbers of complex EHR Software solutions (SaaS) that have a high level of modularization and interoperability.

Further developments toward a complete SOA stack such as offering a Platform as a Service (PaaS) and even the underlying Infrastructure as a Service (IaaS) in the cloud will face more resistance in a domain known for a lot of legacy systems and concerns about privacy and security.

The Object Management Group (OMG) is organizing a conference this summer on the topic of "SOA in Healthcare: Improving Health through Technology: The role of SOA on the path to meaningful use". It will be interesting to see what healthcare providers, payers, public health organizations and solution providers from both the public and private sector will have to say on this topic.

Cloud Computing and Health Care Applications: a change in opinions?

2010-03-31T12:25:00.000-07:00

I have designed and implemented Health Care Applications for more than 3 years and I have experienced a dramatic change of opinions toward the use of Cloud Computing for Health IT.

Several years ago, the idea of having on demand resources offered as a service, used to process or store Health Care related data, was out of the question. The main concerns were the security, privacy and confidentiality of the data; the reliability and ease of use of the underlying systems and platforms.

Health Care solution providers did not hesitate to require a minimum of tens of thousands of dollars of hardware to deploy a minimum configuration for a multi-tier EHR or PHR web based application. In fact, some players were even barely starting to virtualize their platforms.

One of the requirement to comply with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulation is that the transmission of patients protected health information (PHI) over open networks must be encrypted.

These issues have been recently addressed and companies offering virtual infrastructure as a service such as Amazon EC2 offer 256 bit AES encryption algorithms for files containing PHI, as well as token or key-based authentication and sophisticated firewall configurations for their virtual servers. Encryption is also available when storing the data on Amazon S3. The access from the internet or EC2 to Amazon S3 is done via encrypted SSL endpoints which ensures that PHI information stays protected. AWS indeed describes several Cloud based Healthcare related applications in their case study, including MedCommons (a health records services provider that give the ability to the end users to store among other medical information CCR and DICOM documents).

Cloud infrastructure providers such as Amazon Web Services (AWS) ensure that their administrators or third-party partners cannot have access to the underlying PHI data. Strong security policies, access consent processes, as well as monitoring and audit capabilities are available to reduce dramatically the risks of unauthorized access. In addition to this, these providers offer highly available solutions for automated back-ups and disaster recovery which make them more attractive that traditional solutions. Some providers also ensure that the data in question stay within the borders of specific regions, states or countries to comply with regulations in place.

In fact it is very interesting to see these days Health Care becoming a show case of the benefits of Cloud computing. Last month, at the San Francisco Bay Area ACM chapter presentation on cloud computing, I was surprised to see that the first Cloud Application example mentioned was TC3. The numbers were indeed very convincing: When facing with sudden increase of insurance claims processing (from 1 to 100 millions per day in a very short time), TC3 had the option of a traditional solution consisting of $750K of new hardware and $30K of maintenance and hosting per month, or use an Amazon Web Service Cloud solution for $600 per month. The decision was easy I suppose!

MapReduce an opportunity for Health and BioSciences Applications?

2010-02-26T08:49:00.000-08:00

HealthCare and BioScience software products and solutions have embraced Database Management System (DBMS) for their back-end storage and processing for years like most other domains where performance, scalability, security, extensibility, auditing capabilities and maintenance are critical.

In the past few years with alternative or complement technologies such as MapReduce and Hive originally created from the need of extremely high volume web applications such as Google, Facebook or LinkedIn. A lot of people, especially engineers are now wondering if these technologies could be used in HealthCare and BioSciences.

More and more job openings outside the Social Networks or SEO sphere now mention MapReduce and Hadoop in their required or "nice to have" skills, including HealthCare and BioScience companies. In fact, recently at a talk from Bay Area Chapter of the ACM on Hadoop and Hive, even though the talk was quite technical, there were few venture capitalists in the crowd who were checking if this the topic was only hype or would potentially bring big ROI. Healthcare and biotechnologies were definitively in their mind.

Why then would the MapReduce paradigm be a good candidate to provide the "next quantum leap" for HealthCare and BioSciences?

In HealthCare, as more and more users, patients and professionals upload data to applications such as PHRs and EMRs, there is a need to parse, clean and reconcile extremely large amount of data that might be initially stored in log files. Medical observations from patients with chronic diseases such as blood pressure or blood glucose might be good candidates for this, especially when they are uploaded automatically from medical devices. Also the aggregation of data coming from potentially large numbers of sources makes it more suitable to a Map and Reduce processing paradigm than DBMS based data mining tasks.

HealthCare decision makers might be hesitant to use these new technologies as long as they have some concerns related to security, confidentiality and certification to standards such as HL7 (see CCIT and HITSP). However with the overall reforms in progress in HealthCare it will be interesting to see if MapReduce will be part of the technical package for the benefits of not only the patient and care givers, but all healthcare actors including payers and various service providers.

BioSciences (drug discovery, meta-genomics, bioassay activities ...) is also a good candidate for MapReduce. In addition to the fact that BioScience applications deal also with large amount of data (e.g. biological sequences such as DNA, RNA, proteins) a lot of the data is semi-structured data that is semantically rich and most likely best represented as a RDF data model than a Database set of tables (e.g. see "Storage and Retrieval of Large RDF Graph Using Hadoop and MapReduce") . Even though database has made progress to store and process XML, MapReduce is more suitable to very fast processing and aggregation of large amount of key-value elements.

Another element is price and return on investment (ROI), especially for startups is the fact that the implementation of MapReduce over a cloud based infrastructure using an open source framework such as Hadoop and Hive can be an attractive economic proposition for a CTO.

Also both fields can also take advantage of other applications of MapReduce in areas other than hard-core technology but related to brand management, sales and supply chain optimizations used with success in other domains.