middleware&security technical blog

CRRTC3505E: The following fetch destination cannot be deleted - Rational Build Engine

2013-05-21T10:59:00.005+02:00

If you happen to have problem running your build in RTC and your builds end up with:

2013-05-21 10:29:15 [Jazz build engine] Deleting fetch destination "/db2data/BuildWorkspace" before fetching ...
com.ibm.team.build.common.TeamBuildException: CRRTC3505E: The following fetch destination cannot be deleted: "/db2data/BuildWorkspace". For more details, open the help system and search for CRRTC3505E.
at com.ibm.team.build.internal.engine.JazzScmPreBuildParticipant.preBuild(JazzScmPreBuildParticipant.java:218)
at com.ibm.team.build.internal.engine.BuildLoop.invokePreBuildParticipants(BuildLoop.java:881)
at com.ibm.team.build.internal.engine.BuildLoop$2.run(BuildLoop.java:685)
at java.lang.Thread.run(Thread.java:738)

try to kill existing engine process and restart it anew:

ps -ef | grep jbe

kill -9 <pid from above>

Good luck, leave a comment!

libXm.so.3: cannot open shared object file & libXm.so.3: wrong ELF class: ELFCLASS64 (RRDI 2.0.1 installation)

2013-05-15T15:44:00.000+02:00

Just today I wanted to install Rational Reporting for Development Intelligence (RRDI) 2.0.1 on 64bit SuSE 11. I got my repository, I got Installation Manager 1.6 (it's a must!) uznipped and started the ride.

The first error IM thrown at me was "Installation failed with code status =127" and investigation in the IM log files (/var/ibm/InstallationManager/logs/native/...) shown:

/opt/IBM/RRDI/install/cognos_bi/linuxi38664h/issetup: error while loading shared libraries: libXm.so.3: cannot open shared object file: No such file or directory

So, I looked google up and found that I'm missing some libraries for my system, namely:

Open Motif 2.2.4 Libraries (openmotif22-libs-2.2.4-189.1.i586.rpm)

After getting and installing it (for 64bit) I rerun the installation....and failed again. I found then some tip to link libraries from /usr/lib64 into /usr/lib and during next installation attapmt I ran into:

/opt/IBM/RRDI/install/cognos_bi/linuxi38664h/issetup: error while loading shared libraries: libXm.so.3: wrong ELF class: ELFCLASS64

So, fine, some progress because a different error :) Search again, and found that this means wrong bitness of the libraries, so I went back to RPM search and got 32bit libraries unlinked those 64bit from /usr/lib, installed the package and...this time success! I'm no Linux expert, but that occured a bit strange to me, that I eventually solved it by installing 32bit libs on 64bit system. Anyway, should you run into the same problem, perhaps this helps you.

Good luck, comments welcome!

Google, don't take Reader away from us!

2013-03-15T10:28:00.001+01:00

As you may know, Google wants to close Google Reader service, which is widely used and highly praised.

You may want to contribute to possible prolonging of this tool of choice of many internauts here.

WAS 8.5 profile creation sample response file

2013-03-05T13:46:00.001+01:00

I looked over the net for sample response files for profile creation in WAS 8.5, but I found none. Well, I didn't expect much of a revolution in comparison to V7 or V8, but still, better be safe than sorry :)

So I ended up converting possible listed manageprofiles flags into response file entries, in a traditional manner. The result was as follows and you can safely use it as a starting point for your profile management automation.

#profile name and capabilities
profileName=P01Dev
profilePath=/opt/IBM/WebSphere/AppServer85/profiles/P01Dev
templatePath=/opt/IBM/WebSphere/AppServer85/profileTemplates/default
#location and names
hostName=chronos.warszawa.pl.ibm.com
nodeName=P01DevNode
cellName=P01DevCell
serverName=P01DevS01
#starting port
startingPort=4000
#certificates
personalCertDN=cn=P01DevS01\\,ou=Root=Certificate\\,ou=P01DevNode\\,ou=P01DevCell\\,o=IBM\\,c=PL
signingCertDN=cn=P01DevRoot\\,ou=Root=Certificate\\,ou=P01DevNode\\,ou=P01DevCell\\,o=IBM\\,c=PL
#standard WAS keystore/truststore password
keyStorePassword=WebAS
#admin security
enableAdminSecurity=true
adminUserName=devadmin
adminPassword=<yourpassword>

good luck, and leave a comment if you made some use of this content. and don't forget to like this page on facebook :)

http status 500: exception on request for [/web] : null - Rational Jazz/CLM 4.0.1

2013-02-18T10:12:00.002+01:00

If you happen to deploy CLM 4.0.1 (formerly known as Jazz Team Server/Requirements Composer/Requirements Management etc.) and when you further try to customize it (in my case customization was attempt to deploy BuildForge Connect Adapter for CLM) you may run inot some CLM dashboard pages unresponsive. Particularly, admin.war application stops responding and seems to have some sort of authentication issues. In my case, when I tried to go to Home->Collaborative Lifecycle Project Management page, I received the follwing error in the browser:

Http status 500: exception on request for [/web] : null

and SystemOut.log contained a bunch of exceptions:

00000044 zazl E org.dojotoolkit.zazl.internal.RhinoDTLHandler runDTLScript Exception on request for [/projects/new] java.lang.NullPointerException

00000044 SystemOut O 15:39:08,820 [WebContainer : 14] ERROR com.ibm.team.lpa.config - An uncaught Exception has been thrown com.ibm.team.jfs.app.http.HttpInternalServerErrorException: Exception on request for [/projects/new]

00000044 zazl E org.dojotoolkit.zazl.internal.RhinoDTLHandler runDTLScript Exception on request for [processTemplate]

java.lang.NullPointerException

0000004f zazl E org.dojotoolkit.zazl.internal.RhinoDTLHandler runDTLScript Exception on request for [/web]

etc.

I wondered what to do about it, and finally I saw that all troublesome pages are in /admin context. As my CLM was deployed to WAS 8 server, I simply redeployed (by Update) admin.war application from the console. It worked as a wonder!

to do this go to Websphere applications -> tick admin.war, click Update and find original file in

<CLMInstallDir>/server/webapps, proceed through Nexts, and save configuration afterwards. Then restart application and it should be working.

This trick also works with jts.war application, if you happen to have any trouble with it (with similar errors). Of course, I assume that your previous configuration was in general ok, because war Update does not fix any database content issues and so on. (for db/config problems you should use repotools but that's another story)

Hope this helps, good luck and leave a comment!

TAMeSSO 8.2 configuration SQL0613N SQLSTATE=54008

2013-01-22T16:09:00.000+01:00

I have just ran into this problem today, when configuring my test instance of TAMeSSO (lately known as ISAM - IBM Security Access Manager for Enterprise Single Sign-on).

After installing WAS7 and IMS server package, I tried to configure my IMS server. But it failed with error SQL06013N on WAS side, when attempting to create DB tables (on the screen you can see below, procedure dropped with error at ~6%)

After a quick search and a conversation with a colleague I found solution: to increase pagesize in your database to at least 8K. Actually, it is listed as requirement here:

but there is a slight chance you might omit that :) (I did...)

So, be sure that your database have been created with pagesize of 8K. If not, drop it and recreate it with a proper setting. The reason for it being necessary is a requirement for an index tablespace to be large enough to house an indexes for a colums of a specified size. Here it was 1024 VARCHAR + 128 bit for index, which gives over 1024, and hence requires 8K Pagesize. Details can be found here

Good luck and leave a comment if it solved your problem.

TAM 6.1 - HPDAC0457E The protected object name is invalid. (status 0x1005b1c9)

2013-01-02T16:06:00.001+01:00

A colleague of mine run today into a problem while trying to attach ACL to a protected object seen in the objectspace tree. Although being "seen" by both WPM console and pdadmin client via

object show /xxx/yyy/subcat

command, trying to use

acl attach /xxx/yyy/subcat ACL_name

resulted in getting below error:

Error: HPDAC0457E The protected object name is invalid. (status 0x1005b1c9)

At the same time attaching ACLs to parent and child objects was just fine.

We checked initial loading scripts, and we found that there were only entries for super- and sub- items, like:

/xxx/yyy/ and then for /xxx/yyy/subcat/subsubcat/ but not for the middle one

So what was happening was that /xxx/yyy/subcat wasn't really created before, but only was visible as a tree level, because TAM had to show something in that place. Then the fix was easy, we just created the missing object with

object create /xxx/yyy/subcat "" 10 ispolicyattachabe yes

and then we could successfuly attach ACL to this object.

Hope this helps, enjoy and leave a comment!

commonDBUtility.ant Source file does not exist! during WebSphere ESB DMgr creation

2012-09-20T12:18:00.001+02:00

When creating ESB DMgr profile you may encounter the following error in the log:

<date>Sep 20, 2012 10:47:53 AM</date>
<millis>1348130873432</millis>
<sequence>10111</sequence>
<logger>com.ibm.ws.install.configmanager.actionengine.ant.utils.ANTLogToCmtLogAdapter</logger>
<level>WARNING</level>
<class>com.ibm.ws.install.configmanager.logging.LogUtils</class>
<method>logException</method>
<thread>0</thread>
<message>/opt/websph/ESB/util/dbUtils/profileHelpers/commonDBUtility.ant:985: Source file does not exist!
at org.apache.tools.ant.taskdefs.SQLExec.execute(SQLExec.java:324)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275)
at org.apache.tools.ant.Task.perform(Task.java:364)
at org.apache.tools.ant.Target.execute(Target.java:341)

(...)

and resulting:

<record>
<date>Sep 20, 2012 10:47:53 AM</date>
<millis>1348130873556</millis>
<sequence>10164</sequence>
<logger>com.ibm.wsspi.profile.WSProfileCLI</logger>
<level>INFO</level>
<class>com.ibm.wsspi.profile.WSProfileCLI</class>
<method>invokeWSProfile</method>
<thread>0</thread>
<message>Returning with return code: INSTCONFFAILED</message>
</record>
</log>

This happened to my colleague, who was configuring ESB DMgr against Derby network server. We spent a while figuring out what's wrong. Closer look into logs couple of lines before first error showed

<message>Target stopped for: init.db - SUCCESS</message>

so the Derby DB was initialized fine. Right afterwards we found:

<message>Execute SQL script with parameters: JDBCDriver='org.apache.derby.jdbc.ClientDriver' DB_URL='jdbc:derby://localhost:1527//opt/websph/ESB/profiles/Dmgr2/databases/WPRCSDB4' dbUserId='admin' sqlScriptPath='/opt/websph/ESB/profiles/Dmgr2/dbscripts/CommonDB/Derby/WPRCSDB4/createTable_CommonDB.sql' JDBC_DRIVER_FILE_STATIC='/opt/websph/ESB/derby/lib/derbyclient.jar'</message>

following this trail I asked my buddy to check dbScriptOutputDir content. Bingo! it contained the following structure:

/opt/websph/ESB/profiles/Dmgr2/dbscripts/CommonDB/Derby/WPRCSDB4/CommonDB/Derby/WPRCSDB4/pliki.sql

with clearly abundant subtree portion. The reason for this was that in the manageprofiles dbScriptOutputDir parameter contained unnecessary tail that is created automatically. Hence, ant script failed not finding sql files where expected. Changing

./manageprofiles.sh -create (...) -dbOutputScriptDir '/opt/websph/ESB/profiles/Dmgr2/dbscripts/CommonDB/Derby/WPRCSDB2'

./manageprofiles.sh -create (...) -dbOutputScriptDir '/opt/websph/ESB/profiles/Dmgr2/dbscripts'

did the trick. Please note that InfoCenter says you need to do it wrongly! Look here (Table 2, -dbOutputScriptDir description. Thank you Rafał!)

Good luck!

HPDAZ0602E Corrupted file: Insufficient information to contact a Policy Server (SECJ0391E)

2012-07-30T12:44:00.003+02:00

Now, a few words about possible problems when configuring external JACC authorization provider for WebSphere App Server, namely - Tivoli Access Manager. The key issue here always lies in obtaining Policy Server's certificate for JRE to authenticate later on. This can be done in various ways (sslsrvcfg, pdconfig etc.) and obviously, the results may vary depending on the way used - at least it seems so.
Anyway, recently I was asked what may be the reason for the following error appearing in the logs:

[7/25/12 10:07:44:950 CEST] 00000013 AMWASConfigMe I   com.tivoli.pd.as.jacc.cfg.TAMConfigController execute() AWXJC0048E   An error occurred during the configuration. The details are: com.tivoli.pd.as.jacc.cfg.ConfigActionFailedException:
[java.lang.IllegalStateException: HPDAZ0602E   Corrupted file: Insufficient information to contact a Policy Server.
]

Wrappered Exception:
java.lang.IllegalStateException: HPDAZ0602E   Corrupted file: Insufficient information to contact a Policy Server..

This one happened during JACC configuration attempt from the WAS admin console, when console reported that action cannot be successfully completed. "Insufficient information" could only mean that JRE is trying to register with Policy Server, but fails to trust/present valid certificate. Unfortunately, we were stubborn, and hoped that action will complete during server restart. But after saving configuration and restarting server, it fails to start! In the logs we could see:

[7/25/12 10:19:49:404 CEST] 00000000 distSecurityC E   SECJ0391E: Error when setting the Policy object to the provider's policy implementation com.tivoli.pd.as.jacc.TAMPolicy. The exception is com.tivoli.pd.as.jacc.util.JACCException: AWXJR0006E   The file, /opt/ibm/WebSphere/AppServer/profiles/PSS1UPRB01DMgr/etc/tam/amwas.PSS1UPRB01DMgr_dmgr.amjacc.properties, was not found.
    at com.tivoli.pd.as.jacc.TAMPolicy.init(TAMPolicy.java:680)
    at com.tivoli.pd.as.jacc.TAMPolicy.<init>(TAMPolicy.java:97)
    at java.lang.J9VMInternals.newInstanceImpl(Native Method)
    at java.lang.Class.newInstance(Class.java:1345)

which meant that WAS has got JACC enabled, but in fact had failed to provide proper configuration file for amwas/jacc. We gave it some thinking and first, we had to recover from this fatal condition: by editing security.xml we disabled security and disabled using JACC in section:

<security:Security xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:orb.securityprotocol="http://www.ibm.com/websphere/appserver/schemas/5.0/orb.securityprotocol.xmi" xmlns:security="http://www.ibm.com/websphere/appserver/schemas/5.0/security.xmi" xmi:id="Security_1" useLocalSecurityServer="true" useDomainQualifiedUserNames="false" enabled="true" cacheTimeout="600" issuePermissionWarning="false" activeProtocol="BOTH" enforceJava2Security="false" enforceFineGrainedJCASecurity="false" appEnabled="true" dynamicallyUpdateSSLConfig="true" allowBasicAuth="true" activeAuthMechanism="LTPA_1" activeUserRegistry="WIMUserRegistry_1" defaultSSLSettings="SSLConfig_1"> <--- change bold to FALSE

<authConfig xmi:id="AuthorizationConfig_1" useJACCProvider="true"> <-- change bold to FALSE

After that server went up again and we gave a bit thinking to the problem. It turned out to be awfully simple and...well, strange ? We had to add hostname of the WAS machine to policy server's /etc/hosts. Possibly this can be also solved by adding WAS hostname to DNS, anyway, pdmgrd must be able to resolve WAS's IP based on presented hostname. Then, we were able to complete JACC configuration successfully, turn security back on for the whole cell, save and restart and all worked like a wonder!

Why do I say it was strange? Because my suspicion is that different PD.jar packages (or more precisely: pd.* classes) responsible for connecting to Policy Server do this in a different way. I'm too weak a programmer to dig this up and resolve it 100%, but I just know that you may get different results when a) configuring JRE from pdconfig b) using sslsrvcfg and c) configuring JACC from WAS (WebSphere's embedded TAM) - eventually, you just need to see what works best for you.

Good luck, leave a comment if it helped!

SSL/HTTPS - problems with kdb files

2012-07-23T12:14:00.002+02:00

I'm active Experts Exchange contributor, and there's this SSL/kdb problem. I indulge myself into publishing my comment to one of the questions (http://www.experts-exchange.com/Networking/Protocols/Application_Protocols/SSL/Q_27794894.html) here (with some edits):

It's on CMS format (IBM Key Database file) and problems with opening it with your ikeyman tool (with WebSphere java):

for CMS it is IBM proprietary format (like LTPA) and is not available in non-IBM JRE/JDKs, BUT I also had this issue that WAS's JDK could not open CMS (kdb) files - can't really say why as I did not troubleshoot it. but the workaround that worked for me was to run ikeyman not from /opt/IBM/WebSphere/AppServ

er/java but from different WAS package JRE - like UpdateInstaller or InstallationManager - I'm sure you have either installed on your machine so try them.
I just now checked how it looks like when running ikeyman from: C:\Program Files (x86)\IBM\WebSphere\AppServer\java\jre\bin I can operate on CMS files but when running from: C:\Program Files (x86)\IBM\Java60\jre\bin I can't, so it might be something with your java paths. If you can't figure it out, try the workaround I suggested above (UI or IM java)

difference between kdb and p12 is - at least this is "emiprical" difference experienced by me - that kdb usually houses many certificates (signer&personal) for use by applications, whereas p12 is usually used to carry one certificate from an issuer to the owner (for instance I get my corporate certificate in p12 from supplier). just "any" java's keytool or any gsk7 won't be able to open kdb file, it must me somewhere near ;) WebSphere

if you use kdb file for your IHS, don't forget to indicate your certificate as "default" in the kdb file. I was looking for the way to set cert alias to use from within httpd.conf file, but it seems to be impossible

I thought that IHS uses ONLY kdb database to get certifcates from but I just found that you may simply supply crt file - PEM encoded (example: http://rimuhosting.com/howto/modssl.jsp)

some reference - different product, but usage of gsk7 commands is given:
http://pic.dhe.ibm.com/inf

ocenter/tivihelp/v3r1/index.jsp?topic=%2Fcom.ibm.tivoli.itws.doc_8.5.1%2Ffipsensurenetwork.htm

WebSphere Community Blog: WebSphere Application Server V8.5 is Now Available...

2012-07-10T12:30:00.001+02:00

Now watch this... this is really cool!

WebSphere Community Blog: WebSphere Application Server V8.5 is Now Available...: I first talked about WAS V8.5 back in October 2011 when we started the early program. And in June 2012 we shipped it...Walt Noffsinger, WAS...

Windows 7 XP Mode - Virtual PC

2012-07-09T12:08:00.001+02:00

Did you know that windows 7 users have the option to create easily virtual machines with Windows XP? It is called "XP mode" and works really great?

A colleague of mine has recommended it a long time ago, but until recently I had no real need to use it, because all my critical work tools ran smoothly on win7. But a few days ago I wanted to set up a test environment for some IBM software and I didn't want to infest my usual Win7 machine with additional software, which I will probably want to discard soon. So, having downloaded XP mode package I configured it, wondering will it be useful at all. It turned out pretty good!

Installation is quite quick - I guess VPC shares some libraries with the host system and does not really create VMWare-like virtual machine - rather slices out some system resources and presents it as a standalone VM. Anyhow it makes it, it does it really well. XP machine starts literally a few minutes after you begin installation of XP Mode and it runs pretty quickly, letting your work on the VM to be quite nice - no noticable lags, hard disk scratching and so on. And it is all on the encrypted hard disk.

You can use XP Mode twa ways: either as a standalone vm, or as a "xp native" runtime for applications hosted in fact on your Win7 system, with whom for some reasons xp compatibility set in program properties is not enough. I did no try the latter, but I assume the experience would be good as well.

The XP VPC is istantly connected to the internet (provided of course your host system is) by the means of NAT. However, if you want to connect over tcp ip to some software running on the VM, like some specific port, you need to do some tweaking. It'd be a nice topic for another post, but I did not solve this issue myself, so I just post a link to a forum page with solution by "judios".
So anyway - if you are a licensed Win7 user and you need a simple WinXP virtal machine VPC is definately a thing for you! Good luck, have fun and don't bother to leave a comment!

PDJRTE for TAI++ config error: java.lang.NullPointerException at com.tivoli.pd.jutil.jb.getCACert(jb.java:129)

2012-05-30T13:17:00.000+02:00

If you work with complex TAM&J2EE deployments, you will most probably come to the point where you need to use TAI++ trust association scheme to tie your J2EE server (either WebSphere or some other) with TAM&WebSeal system. In particular, this is useful when you want to authenticate users in WebSeal to "let them in" to your backend server but leave authorization for J2EE application to it's internal mechanisms (based on LDAP, for example). If you want to read more on TAI/TAI++ you can do it here or here.
However, in TAI++ scenario you will most probably come to the point when you will need to configure your Java Runtime for Policy Director (usually done in pdconfig or with pdjrtecfg directly). What WAS really needs for TAI++ is essentialy address of Policy Server, it's certificate to be trusted (downloaded during PDJRTEconfig) and registration with TAM pdmgrd as a member of security domain. These information is stored in (not strictly set, but reasonable to do it so) .conf and .key files producedafter invoking java com.tivoli.pd.jcfg.SvrSslCfg but first you need to have your PDJRTE configured.

I tried this with WebSphere App Server's (WAS 7 and TAM 6.1.0.5) java first and usually you do it by sourcing WAS environment first and then using pdconfig. However, in WAS 7 there's a class conflicts of some kind and when you go to pdconfig and choose to configure WAS java (normally /opt/ibm/WebSphere/AppServer/java/jre) to be the runtime for Policy Director in picks proper java, but fails to finish the configuration with nasty error:

Configuration of Access Manager Runtime for Java is in progress.
This might take several minutes.
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at com.tivoli.pd.jcfg.PDJrteCfg.config(PDJrteCfg.java:245)
        at com.tivoli.pd.jcfg.PDJrteCfg.interactCfg(PDJrteCfg.java:1307)
        at com.tivoli.pd.jcfg.PDJrteCfg.invoke(PDJrteCfg.java:1460)
        at com.tivoli.pd.jcfg.PDJrteCfg.main(PDJrteCfg.java:350)
Caused by:
[java.lang.NullPointerException
]

Wrappered Exception:
java.lang.NullPointerException
        at com.tivoli.pd.jutil.jb.getCACert(jb.java:129)
        ... 8 more
Caused by: java.lang.NullPointerException
        at org.apache.harmony.security.fortress.Services$NormalServices.createDefaultProviderInstance(Services.java:286)
        at org.apache.harmony.security.fortress.Services$NormalServices.getService(Services.java:423)
        at org.apache.harmony.security.fortress.Services$NormalServices.access$2100(Services.java:141)
        at org.apache.harmony.security.fortress.Services.getService(Services.java:824)
        at org.apache.harmony.security.fortress.Engine.getInstance(Engine.java:133)
        at java.security.KeyFactory.getInstance(KeyFactory.java:81)
        at com.ibm.security.x509.X509Key.buildX509Key(X509Key.java:275)
        at com.ibm.security.x509.X509Key.parse(X509Key.java:189)
        at com.ibm.security.x509.X509Key.parse(X509Key.java:215)
        at com.ibm.security.x509.CertificateX509Key.<init>(CertificateX509Key.java:112)
        at com.ibm.security.x509.X509CertInfo.parse(X509CertInfo.java:966)
        at com.ibm.security.x509.X509CertInfo.<init>(X509CertInfo.java:236)
        at com.ibm.security.x509.X509CertInfo.<init>(X509CertInfo.java:222)
        at com.ibm.security.x509.X509CertImpl.parse(X509CertImpl.java:2285)
        at com.ibm.security.x509.X509CertImpl.<init>(X509CertImpl.java:227)
        at com.ibm.security.x509.X509CertImpl.<init>(X509CertImpl.java:213)
        at com.tivoli.pd.jutil.jb.getCACert(jb.java:51)
        ... 8 more

[java.lang.reflect.InvocationTargetException
]

Wrappered Exception:
java.lang.reflect.InvocationTargetException
        at com.tivoli.pd.jcfg.PDJrteCfg.config(PDJrteCfg.java:51)
        at com.tivoli.pd.jcfg.PDJrteCfg.interactCfg(PDJrteCfg.java:1307)
        at com.tivoli.pd.jcfg.PDJrteCfg.invoke(PDJrteCfg.java:1460)
        at com.tivoli.pd.jcfg.PDJrteCfg.main(PDJrteCfg.java:350)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at com.tivoli.pd.jcfg.PDJrteCfg.config(PDJrteCfg.java:245)
        ... 3 more
Caused by:
[java.lang.NullPointerException
]

Wrappered Exception:
java.lang.NullPointerException
        at com.tivoli.pd.jutil.jb.getCACert(jb.java:129)
        ... 8 more
Caused by: java.lang.NullPointerException
        at org.apache.harmony.security.fortress.Services$NormalServices.createDefaultProviderInstance(Services.java:286)
        at org.apache.harmony.security.fortress.Services$NormalServices.getService(Services.java:423)
        at org.apache.harmony.security.fortress.Services$NormalServices.access$2100(Services.java:141)
        at org.apache.harmony.security.fortress.Services.getService(Services.java:824)
        at org.apache.harmony.security.fortress.Engine.getInstance(Engine.java:133)
        at java.security.KeyFactory.getInstance(KeyFactory.java:81)
        at com.ibm.security.x509.X509Key.buildX509Key(X509Key.java:275)
        at com.ibm.security.x509.X509Key.parse(X509Key.java:189)
        at com.ibm.security.x509.X509Key.parse(X509Key.java:215)
        at com.ibm.security.x509.CertificateX509Key.<init>(CertificateX509Key.java:112)
        at com.ibm.security.x509.X509CertInfo.parse(X509CertInfo.java:966)
        at com.ibm.security.x509.X509CertInfo.<init>(X509CertInfo.java:236)
        at com.ibm.security.x509.X509CertInfo.<init>(X509CertInfo.java:222)
        at com.ibm.security.x509.X509CertImpl.parse(X509CertImpl.java:2285)
        at com.ibm.security.x509.X509CertImpl.<init>(X509CertImpl.java:227)
        at com.ibm.security.x509.X509CertImpl.<init>(X509CertImpl.java:213)
        at com.tivoli.pd.jutil.jb.getCACert(jb.java:51)
        ... 8 more
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at com.tivoli.pd.jcfg.PDJrteCfg.config(PDJrteCfg.java:245)
        at com.tivoli.pd.jcfg.PDJrteCfg.interactCfg(PDJrteCfg.java:1307)
        at com.tivoli.pd.jcfg.PDJrteCfg.invoke(PDJrteCfg.java:1460)
        at com.tivoli.pd.jcfg.PDJrteCfg.main(PDJrteCfg.java:350)
Caused by:
[java.lang.NullPointerException
]

Wrappered Exception:
java.lang.NullPointerException
        at com.tivoli.pd.jutil.jb.getCACert(jb.java:129)
        ... 8 more
Caused by: java.lang.NullPointerException
        at org.apache.harmony.security.fortress.Services$NormalServices.createDefaultProviderInstance(Services.java:286)
        at org.apache.harmony.security.fortress.Services$NormalServices.getService(Services.java:423)
        at org.apache.harmony.security.fortress.Services$NormalServices.access$2100(Services.java:141)
        at org.apache.harmony.security.fortress.Services.getService(Services.java:824)
        at org.apache.harmony.security.fortress.Engine.getInstance(Engine.java:133)
        at java.security.KeyFactory.getInstance(KeyFactory.java:81)
        at com.ibm.security.x509.X509Key.buildX509Key(X509Key.java:275)
        at com.ibm.security.x509.X509Key.parse(X509Key.java:189)
        at com.ibm.security.x509.X509Key.parse(X509Key.java:215)
        at com.ibm.security.x509.CertificateX509Key.<init>(CertificateX509Key.java:112)
        at com.ibm.security.x509.X509CertInfo.parse(X509CertInfo.java:966)
        at com.ibm.security.x509.X509CertInfo.<init>(X509CertInfo.java:236)
        at com.ibm.security.x509.X509CertInfo.<init>(X509CertInfo.java:222)
        at com.ibm.security.x509.X509CertImpl.parse(X509CertImpl.java:2285)
        at com.ibm.security.x509.X509CertImpl.<init>(X509CertImpl.java:227)
        at com.ibm.security.x509.X509CertImpl.<init>(X509CertImpl.java:213)
        at com.tivoli.pd.jutil.jb.getCACert(jb.java:51)
        ... 8 more

The configuration failed.

Press Enter to continue.

I suppose it is because WAS 7 has it's own PD.jar file which may even be newer than the one supplied with TAM 6.1.0.5 <-- that's the version we're talking here about. Or it is because WAS 7 uses java 6, whereas tam works fine with java 5 - I can't tell exactly.

Anyway, what to do about it? Simply point pdconfig to a different java. For example, bundled with TAM base package is ibm java 5. Install it (it is in /opt/ibm/java2-i386-50/ directory), export it's path:

export PATH=$PATH:/opt/ibm/java2-i386-50/java/jre

and try pdconfig to configure pdjrte again. It should succeed now.

To obtain information for using with TAI++, run now SvrSslCfg with the java you just configured eg.:

/opt/ibm/java2-i386-50/jre/bin/java com.tivoli.pd.jcfg.SvrSslCfg -action config -admin_id sec_master -admin_pwd ***** -appsvr_id ******-host ***** -mode remote -port 8925 -policysvr tamsec-p2-1:7135:1 -authzsvr tamsec-p2-1:7136:1 -cfg_file domainname.cfg -key_file domainname.key -cfg_action create -domain domainname

and later supply it to WAS as TAI++ inteceptor config item.

Good Luck!

Generation of deployment environment fails - Reason: CWLDB9014E and Reason: CWWBZ0058E

2012-04-24T11:20:00.001+02:00

First of all, this is the task I failed numerous times on, until finally I managed to get past, getting the desired result at the same time. This is because I found some sort of workaround before, but the results were more or less doubtful. Well, this workaround is a bit messy as well, but at least you can benefit from creation of Deployment environment with the wizard, instead of putting DE parts together step by step (and probably missing some parts, which later cause you some, or a lot of, pain).

So, we're talking WPS 6.2.0.0 (a bit obsolete) and 7.0.0.4, deployed vs. DB2 database hosted on z/OS (which probably is the most troublesome circumstance, as you cannot create any objects automatically like in LUW edition). What we try to do is to generate Deployment environment (in any topology - single, distributed or "golden"), while defining databases on z/OS DB2. What we get is failure in this particular part:, where instead of:

you keep getting:

CWLDB9014E: The configuration of component WBI_BPCEventCollector failed.

To make it short, before, I could not find the reason. But recently, I think I found it by looking into FFDC error report, and saw there errors related to failing connection to DERBY database. I wondered why it tries to use Derby when I explicitly requested all databases to be hosted on z/OS, but I will never know (perhaps some buggy ant script, or other config xml issue). Nevertheless, what we need to do is to:

create "dummy" Derby database, just for sake of generation of DE:
- go to /opt/ibm/WebSphere/AppServer/derby/bin/networkServer
- run ./ij.sh
- issue command: connect 'jdbc:derby:/opt/ibm/WebSphere/AppServer/profiles/PPD2UZLB01DMgr/databases/event/PD2UZL.Support/DerbyEventDB/BPD2CEDB;create=true';
your database has been created and is ready to use, but first:
add custom user to connect to Derby server
- edit /opt/ibm/WebSphere/AppServer/derby/derby.properties by adding line
```
derby.user."yourUser"=yourpwd
```
start Derby network server (/opt/ibm/WebSphere/AppServer/derby/bin/networkServer/startNetworkServer.sh)
Note: default TCP/IP port for derby is 1527
now, get back to DE generation wizard and run it again, but now type in Derby database for your EventDB - server is localhost:1527, user is "yourUser" and password "yourpwd"
finish generation and you should get to successful end of operation
assuming you have already created all necessary database in your target DB (either DB2 on z/OS or other supplier), you now need to redefine jdbc datasource for EventDB to point to the datastore of your choice.

You may of course run into numerous problems before getting your DE working 100% fine but above instruction should at least get you through DE generation wizard.

Good luck, leave a comment or vote in the poll please!

WPM config - HPDCO1364E The specified domain does not exist. (0x1354a554)

2012-03-23T16:16:00.002+01:00

While trying to configure WPM (TAM Web Portal Management), you may encounter this particular error:

HPDCO1364E The specified domain does not exist. (0x1354a554)

You are most probably using non-default (not "Default") domain. Apparently, WPM has a problem with specifying different domain name than default, but this occurs in version 6.1.0.0. What you need to do is to upgrade to fp5 (or lower fp, but I tested it with fp5)

get 6.1.0-TIV-TAM-FP0005-LIN.tar.Z file (for Linux x86)
gunzip and tar -xf it
install with: rpm -Uhv PDWPM-PD-6.1.0-5.i386.rpm
try pdconfig

after fixpacking it should suggest you (get from configured Java Runtime) correct domain name.

Good luck, leave a comment.

a new EEv10 is here!

2012-03-13T09:12:00.003+01:00

Aside from this blog activity, I'm also active Experts Exchange contributor. They launched new version of their site recently, and are very proud of it. So proud, that they give iPads away :) Why should I not get one?

javax.management.JMRuntimeException: ADMN0022E

2012-03-07T10:08:00.000+01:00

If you stumble upon the following error:

See nested exception; nested exception is: javax.management.JMRuntimeException: ADMN0022E: Access is denied for the getChain operation on TransportChannelService MBean because of insufficient or empty credentials.

that means you need to revise your application permissions settings. That is, your application most probably has a RunAs role defined, which determines identity that is being mapped to each appliacation call - WebSphere then "thinks" that each call to this EJB is being made by some specified user (RunAs).

To check this, go to:

Applications > Enterprise Applications > application_name

and find Map RunAs roles to users under Additional Properties section. There you will find RunAs roles defined for this particular EJB. What you need to do is to type in valid username and password (having necessary permission level if you are using any authorization system - either role-based or external, like calling PDContext for TAM authorization), tick roles you want to assign them to and use Aplly, then save, synchronize and restart server.

Detailed information can be found on IBM InfoCenter.

Good luck, leave a comment or vote in the poll, please :)

humble anouncement

2012-03-02T14:48:00.001+01:00

Dear Readers, just want to humbly (ok, not so :) anounce that this blog has just reached 1000 pageviews, mostly happened withinin the last two months.

Thank you all for visiting, I do hope you will return to my posting spot. Please leave comments and feedback in the poll.

The new Google's privacy policy - take notice!

2012-03-01T10:11:00.000+01:00

Google is going to make it's new privacy policy effective soon. Among other things, one is particularly alarming - information given to google in all their services (60+) will be merged to create your unified online history. That includes your goolge search history, which is a clear footprint of your age, gender, preferences, medical concerns etc.

You may want to read more and before all - disable you Online history recording, as described in this EE article.

IBM HTTP Server not starting - http_plugin.log excess size

2012-02-27T12:00:00.001+01:00

I just happened to come across the following error:

I'm using IBM HTTP Server 7 with WAS Plugin configured. the following behavior was observed - after attempting to start IHS with

<IHS_INSTALL_ROOT>/bin/apachectl start

control returns to shell, as it should. But when I checked if IHS is up and running with

nestat -an | grep <portnumber>

it didn't show desired port listening, however when looking for httpd processes with:

ps -ef | grep httpd

showed all necessary httpd jobs (namely, 4 of them). That is a problem symptom.

This was the second time I came across that, so I knew what to do. The clue is http_plugin.log file size: when it grows over 2147483647 bytes (on Linux), OS is no longer able to write to it, and despite seemingly successful startup of the server. So, you just need to remove (or move to other place if you need it) this file, and try to start IHS again, it will recreate it and work fine. The log file is located in <PLUGIN_ROOT>/logs/<Servername>/http_plugin.log

If you happen to see this malfunction, review your WAS plugin LogLevel settings. Remeber that on Trace or Detail level it generates fairly lot of entries and log grows very quickly. If your system is working fine and you don't have any sophisticated file log monitoring implemented, just change it to Info or Warn - it will be sufficient and your log file won't clog up.

Good luck, thanks for comments and feedback!

Installation Manager java.lang.UnsatisfiedLinkError: Could not load SWT library

2012-02-13T14:14:00.001+01:00

IBM Installation Manager is more and more widely used to deploy number of IBM products, so it may be useful to know a little about possible problems when running this tool.
My today's accomplishment is that I overcame following issue with installation on a lightweight SuSE 11 distribution (eg. stripped from almost every non-necessary package).
First, when trying to install IM itself I was knocked by:

JVMDUMP010I Snap dump written to /tmp/was/IM/Snap.20120213.121315.11358.0003.trc
libgcc_s.so.1 must be installed for pthread_cancel to work

This one was tackled by adding libgcc43-32bit package to the system. After successful silent installation, I tried to actually run IM to install WAS 7. Shell showed nothing after issuing:

./install or ./IBMIM

It was all I got:

prep2def:/opt/ibm/InstallationManager/eclipse # ./launcher
prep2def:/opt/ibm/InstallationManager/eclipse # cd /tmp/
Looking into configuration/datestamp.log files I found this error:

!ENTRY org.eclipse.osgi 4 0 2012-02-13 13:32:15.756
!MESSAGE Application error
!STACK 1
java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons:
        /opt/ibm/InstallationManager/eclipse/configuration/org.eclipse.osgi/bundles/454/1/.cp/libswt-pi-gtk-3659.so (libgthread-2.0.so.0: cannot open shared object file: No such file or directory)
        swt-pi-gtk (Not found in java.library.path)
        /tmp/swtlib-32/libswt-pi-gtk-3659.so (libgthread-2.0.so.0: cannot open shared object file: No such file or directory)
        /tmp/swtlib-32/libswt-pi-gtk.so (/tmp/swtlib-32/liblibswt-pi-gtk.so.so: cannot open shared object file: No such file or directory)

And finally, installing:

DejaVu Truetype Fonts

and

libgthread-2_0-0-32bit

helped, and I was able to successfully run Installation Manager. Hope it helps. Good luck.

SECJ0053E and AWXJR0044E: PolicyConfiguration exists = false

2012-02-07T15:09:00.000+01:00

If you are working on WAS with JACC configured against Tivoli Access Manager (or other external authorization provider, but I assume TAM is the most popular) in your application development cycle you may stumble upon the following error sequence:
[2/7/12 11:24:45:304 CET] 000000c8 SecurityColla A   SECJ0053E: Authorization failed for defaultRealm/username while invoking (Bean)XyEAR-2.2#XyzBean.jar#soapAction:5 JACC Authorization failed for bean: xyzvBean
[2/7/12 11:24:45:660 CET] 0000003d AMWASJACCMess I   com.tivoli.pd.as.jacc.TAMPolicy implies(ProtectionDomain, Permission): permission = perm.toString()
0x864297004
AWXJR0044E   The access decision for Permission, (javax.security.jacc.EJBMethodPermission xyz), was denied because either the PolicyConfiguration or RoleConfiguration objects did not get created successfully at application installation time. RoleConfiguration exists = true, PolicyConfiguration exists = false.

It is more probable to occur right after application update (new version deployed, for instance).

Here are the things you may do to steer out of this trouble:

Ensure all necessary protected object exist in TAM policy database (use pdadmin or WPM - TAM console deployed on WAS)
Update role definition in your authorization provider with:
Global security > External authorization providers

choosing "Update with application names listed" and typing in your application name (as appears in Enterprise Applications tab), then Apply
If point 2. fails, follow this sequence: stop application, remove application from server, stop application server, clean temp directories for this server, start server and redeploy application. It should bind with TAM properly this time. After that, restart the server again.

Good luck.

Java exception: org.omg.CORBA.BAD_INV_ORDER

2012-01-25T12:11:00.000+01:00

It may happen that all of a sudden you will see the following exception in your SystemOut.log of your managed server (that is the one managed from deployment manager console via nodeagent process). That may occur during either server run or (more often) during server startup attempted from shell level.

[1/25/12 11:25:36:870 CET] 0000002c WSSecurityCon A   Error creating client_auth_token in initSecContext, reason: Major Code[15] Minor Code[0] Message[ create_gss_initial_context_token: Java exception: org.omg.CORBA.BAD_INV_ORDER: ORB has been shut down vmcid: OMG minor code: 4 completed: No]
[1/25/12 11:25:36:884 CET] 0000002c IPCConnectorC E   ADMC0076E: Failed to get security token due to Major Code[15] Minor Code[0] Message[ create_gss_initial_context_token: Java exception: org.omg.CORBA.BAD_INV_ORDER: ORB has been shut down vmcid: OMG minor code: 4 completed: No]
        at com.ibm.ISecurityUtilityImpl.CSIUtil.create_gss_initial_context_token(CSIUtil.java:979)
        at com.ibm.ISecurityLocalObjectTokenBaseImpl.WSSecurityContextLTPAImpl.initSecContext(WSSecurityContextLTPAImpl.java:179)
        at com.ibm.ISecurityLocalObjectTokenBaseImpl.WSSecurityContextLTPAImpl.initSecContext(WSSecurityContextLTPAImpl.java:133)
        at com.ibm.ws.management.connector.ipc.IPCConnectorClient.fillInCredential(IPCConnectorClient.java:307)
        at com.ibm.ws.management.connector.ipc.IPCConnectorClient.queryNames(IPCConnectorClient.java:792)
        at com.ibm.ws.management.AdminClientImpl.queryNames(AdminClientImpl.java:108)
        at com.ibm.ws.management.discovery.transport.MBeanDiscoveryHelper.discovery(MBeanDiscoveryHelper.java:507)
        at com.ibm.ws.management.discovery.transport.MulticastServer.rediscover(MulticastServer.java:406)
        at com.ibm.ws.management.discovery.transport.MulticastServer.run(MulticastServer.java:279)
        at java.lang.Thread.run(Thread.java:736)

It usually means that your server has lost communication with it's nodeagent, and the reasons for that may be numerous, but the most common are:

nodeagent process crashed or was killed any other way - that is the easiest: just bring up back the nodeagent
something went wrong with network interfaces (that's more applicable to startup problem) - in that case you need to try to bring up/down lo interface (read here) or check if in the meantime your hostname or ip address hadn't been changed (little chance, but you never know)
if above fail, check if nodeagent NDS (node discovery service) port, among with other ports (soap etc.) are locally available with telnet. if not, you need to diagnose further system's ip sockets setup.

Good luck!

WebSphere JSSL0080E: javax.net.ssl.SSLHandshakeException - The client and server could not negotiate the desired level of security.

2012-01-19T10:33:00.004+01:00

If you see the following exception while or after starting your server (nodeagent or app server):

JSSL0080E: javax.net.ssl.SSLHandshakeException - The client and server could not negotiate the desired level of security.

then big chance is that there's something wrong with internal certificates between WAS instances, either:
1. Hostname has been changed and hostname verification is turned on in SSL setup. you may check this here:
http://www.ibm.com/developerworks/websphere/techjournal/0612_birk/0612_birk.html
in paragraph:

The default trust manager, IbmX509, performs fundamental certificate validation, including the certificate signature validation (ensuring it has not been modified) and certificate expiration validation (ensuring it has not expired). This trust manager does not perform hostname verification by default, although you can set the com.ibm.ssl.performURLHostNameVerification=true property in the security custom properties to enable this function for URL connections only. If this is done, the trust manager will ensure that for URL connections, the hostname specified on the connection matches the SubjectDN in the certificate returned by the server (just as Web browsers do).
(recommend reading the whole article btw.)

2. Something went horribly wrong in terms of cell synchronization, so I'd suggest stopping servers and nodeagent on remote machine and running ./syncNode.sh from profile's bin directory. not sure it will work as probably wsadmin script client won't be able to connect to SOAP port due to root cause. hence, we're back at point 1. and I guess only manual copying of trust/keystores from dmgr machine to remote machine might help.

hope this helps, or at least points to the right direction.

WebSphere SOAP message trace (XML, JAX-WS)

2012-01-16T14:04:00.000+01:00

Okay, there are numerous articles on how to trace XML SOAP messages sent to websphere server, but it seems all of them address some different scenario I am in. Eg. they involve custom development or code-changing in order to add handlers and so on.

What I needed was to simply dump content of SOAP messages incoming to my http inbound channel. However, anything less than wssecurity.*=all given no SOAP content, so I started to drill down to the very one I needed. Finally I found some least fraction of WAS framework to check if you want to get incoming XML content, out of WSSecurityHandler class. As XML parsing involves security processing, you simply need to use the following trace:

com.ibm.ws.wssecurity.handler.*=finest 

in addition to the one you already have set.

Good Luck, I wish you happy tracing!

middleware&security technical blog

CRRTC3505E: The following fetch destination cannot be deleted - Rational Build Engine

libXm.so.3: cannot open shared object file & libXm.so.3: wrong ELF class: ELFCLASS64 (RRDI 2.0.1 installation)

Google, don't take Reader away from us!

WAS 8.5 profile creation sample response file

http status 500: exception on request for [/web] : null - Rational Jazz/CLM 4.0.1

TAMeSSO 8.2 configuration SQL0613N SQLSTATE=54008

TAM 6.1 - HPDAC0457E The protected object name is invalid. (status 0x1005b1c9)

commonDBUtility.ant Source file does not exist! during WebSphere ESB DMgr creation

HPDAZ0602E Corrupted file: Insufficient information to contact a Policy Server (SECJ0391E)

SSL/HTTPS - problems with kdb files

WebSphere Community Blog: WebSphere Application Server V8.5 is Now Available...

Windows 7 XP Mode - Virtual PC

PDJRTE for TAI++ config error: java.lang.NullPointerException at com.tivoli.pd.jutil.jb.getCACert(jb.java:129)

Generation of deployment environment fails - Reason: CWLDB9014E and Reason: CWWBZ0058E

WPM config - HPDCO1364E The specified domain does not exist. (0x1354a554)

a new EEv10 is here!

javax.management.JMRuntimeException: ADMN0022E

humble anouncement

The new Google's privacy policy - take notice!

IBM HTTP Server not starting - http_plugin.log excess size

Installation Manager java.lang.UnsatisfiedLinkError: Could not load SWT library

SECJ0053E and AWXJR0044E: PolicyConfiguration exists = false

Global security > External authorization providers

choosing "Update with application names listed" and typing in your application name (as appears in Enterprise Applications tab), then Apply

Java exception: org.omg.CORBA.BAD_INV_ORDER

WebSphere JSSL0080E: javax.net.ssl.SSLHandshakeException - The client and server could not negotiate the desired level of security.

WebSphere SOAP message trace (XML, JAX-WS)