There are those moments when everything just falls into place.

Before Xmas and before I got the notification from the CAB at cf.Objective(), one of my submitted sessions was accepted for Scotch On The Rocks 2014. I’m still extremely happy about that as the amount of submissions they got was massive (around ~160, iirc) and given that it’s a quite small event, it must have been a tremendously hard choice for them to select sessions. And I’m very, very glad that I made the cut.

It’ll be about 6 years ago that I’ve been to Edinburgh the last time and I’m very keen on going back for both the city and the conference. SOTR used to be very much a CFML conference back then. That’s changed quite a bit over the last few years and the conference is now much more of a (web) development conference open for both topics on the frontend development side of things as well as for backend topics. I’ll also get to see my friend Kay again, yay! Co-incidentally she wrote a very interesting piece about SOTR’s “incentive ticketing”.

However, I was actually a bit surprised when my talk: “Digging in the dirt or digging for Gold? The Internals of the Java Virtual Machine!” was selected.

My initial thoughts were along the lines of:

“Yay, they accepted my talk!!!! Awesome, I’m going to Scotland again!!!”

“Oh, hang on. They want… THAT ONE? Really? WTF!”

Please note that this session is a different talk from what I’m going to present on at cf.Objective() – which is about JVM and memory tuning for CFML developers. My session at Scotch On The Rocks will be quite fundamental in a way that I will endeavour to explain the JVM. Well, maybe not all of it in ~50 minutes, but the Essentials; The parts that people really should understand to then being able to understand what happens right below the language/server of their choice and why JVM tuning might even become necessary. It’s actually kind of geeky – read for yourself, below is my talk’s abstract:

Digging in the dirt or digging for Gold? The Internals of the Java Virtual Machine!

The JVM is a funny odd little thing. Loved and hated by just a few, ignored and misunderstood by many. People talk about Memory Management, Garbage Collection and all sorts of stuff, but what IS the JVM and how does it work?

This talk will discuss the common architecture features of Java Virtual Machines. What does it take to compile one’s Java, CFML, Clojure, Scala (or whatever JVM-based language you might prefer) source code into byte code and execute that? What are life cycle and memory constraints of a Virtual Machine running on actual hardware? And yes – how does it manage memory and how does Garbage Collection work?

The session will – as indicated in the title – focus on the Java Virtual Machine. But a lot of concepts that apply to the JVM are generic problems of Computer Science and this talk might bring back some memories from the past: stacks, types, threads, pointers and much more. Along the way we’re also having a discussion about the (sometimes subtle) differences between the Sun/Oracle JVM, JRockit or the SAP JVM (did you even know SAP was in this business?)

You might ask: Why is this important anyway? The answer is astonishingly simple: Without having at least a fundamental understanding of how the JVM works, you will not be able to write effective and efficient code in any language on top of the JVM.

It already happened an few days ago, but I’m very pleased to announce that I’ll be at cf.Objective() 2014 in Minneapolis in May 2014 and that two of my three session proposals got chosen for the conference’s agenda.

I’ve been to cf.Objective() 2013 as an attendee (Kudos to Gert @ Railo for throwing me a ticket of their sponsor contingent) and it was a really, really good event. To be honest, that really wasn’t a big surprise as cf.Objective() has a really good reputation for being the major CFML conference, very well organised and generally awesome; and it’s true. Also, the venue (Radisson Blu right at Mall of America) is kind of interesting in a very special way.

Anyway – in case you’re interested in what my sessions will be about… (you can also find them on the cf.Objective() Lanyrd site or the actual cf.Objective() website — the session descriptions are not yet 100% up-to-date on the latter):

1. The JVM is your friend

Both Adobe ColdFusion and Railo are Java-based web development platforms. In layman’s terms that means: Your CFML engine runs on top of a JEE application server or a servlet container and the JVM (Java Virtual Machine).

The latter is pretty much the low-level runtime environment of your CFML application. To make sure your CFML engine operates in a well performing and stable way, you need to have some knowledge about Java memory management and how that can impact the behaviour of your CFML server.

The objective of this talk is to change developers’ mindsets when it comes to the JVM below ACF and Railo. There’s an myth that JVM behaviour and JVM tuning is a “dark art” and that has to stop. It will also equip you with a level of fundamental knowledge that you can use to push back when someone tries to tell you to “just use my JVM settings” or to “use the settings this guy had on his blog”.

We’re going to cover:

• Foundations of Java memory management and the important bits for CFML developer
• Java Garbage Collection and various memory cleanup strategies
• How load generation, load testing and measuring the right data plays into JVM tuning
• JVM tuning specifics for CFML developers
• The JVM and the Garbage Collectors in Java 7 and beyond

2. Real-World lessons in jQuery Mobile

“Hey, let’s build a mobile web app!”
“Awww, what an awesome idea – I’ve heard jQuery Mobile is super-easy, bro! Let’s start right away!”

(6 months later)

“Oh my god, this code is a mess – why did I do this thing 4 months ago?”
“How hard can it be to change this jQuery Mobile UI widget just so that it does what I need?”
“Do you know why my icons render distorted on the new {iOS|Android|Windows} device?”
“Why are the page transitions really rough on some of those Android 2.2 and 4.1 devices?”
“The back button navigation is acting weird – I wish we just had built a native app”

Does that sound familiar? jQuery Mobile is a mobile web development framework that’s really easy to get started with. Product Evangelists from big corporates show their tooling around the framework or build their own products using it and it always looks SOOO easy and straight forward in their scripted demos.

There are pitfalls though and a lot of lessons are to be learned along the way – starting from making decisions on how to architect your application (in particular in conjunction with having to incorporate a backend) and ending with bug-fixing discussions like the ones above.

It doesn’t have to be like that – in this session I’m going to talk about a few fundamental architectural concerns when it comes to building a mobile web app (and maybe to wrapping it into a native app later). We’re also going to have a quick look at the pros and cons of using a framework like jQuery Mobile vs. using responsive design and even if and how those two concepts could work together. We’ll discuss some of those common pitfalls and device quirks and things we had to learn the hard way ourselves when we started building mobile apps with a very early version of jQuery Mobile for our clients.

• Defining a good architecture for a jQuery Mobile application
• The relationship between jqM and Responsive Design
• Data handling mistakes to avoid
• Quirks on certain devices and how to deal with them
• The pitfalls of tweaking jqM’s UI widgets
• jqM beyond markup

This is part two of a loosely connected series of blog posts dealing with JVM settings. Make sure you check out the first post titled “JVM memory settings for Railo (and Adobe ColdFusion) on Tomcat” before continuing to get an idea of the overall context of this series.

Today’s post is about why generic recommendations for JVM settings are almost every time going to fail you and why I personally try to avoid having to provide advise along those lines.

Quite often one sees questions like:

Similarly dangerous are scenarios in which people post supposedly awesome JVM settings on their blogs and advertise them as the best thing since the invention of sliced bread (or Jet Planes). This is dangerous because even though those settings might actually work really well for their use case they can never be generic enough to be advertised as such.

Usually either of the above are well intended and that has to be acknowledged and appreciated. Well, that is until the advice is actually not helpful or people can’t really explain what they’re are suggesting and the JVM settings become a dark art. I’d rather see us as a community provide people with the knowledge of being able to help themselves and understand what’s actually happening on their system.

All that being said, what are you supposed to do then? There are some obvious generic recommendations that can be made. Those entail parameters such as overall memory sizing. Or the pros and cons of certain garbage collection strategies or what one can do to mitigate the risks of running into, let’s say… PermGen Out-Of-Memory situations. But that’s about it. From there onwards, proper advise can’t be given in a generic way without having to have a look at the specifics of one’s environment.

I will elaborate on some of the more generic recommendations in further parts of this series. If you want to know more, you can also attend cf.Objective() 2014 and/or Scotch on the Rocks 2014 at which I will talk about those and other subjects related to the JVM.

This is the first post of a loosely connected series about JVM settings (some of them related to memory, some others not). I got kind of inspired by a series of discussion threads on various CFML-related lists sitting in my inbox for a while now (because I felt the urge to comment on them — but never got round to for various reasons…). I’d really like to get to Inbox-Zero with that particular email account, so let’s get some of those topics out of the way via blog posts then instead.

This first post deals with Railo on Tomcat and where to set JVM options in the first place. It might also be useful for Adobe ColdFusion users with a custom configuration running on Tomcat or users of Adobe ColdFusion 10.

When you install Railo with Tomcat, you might wonder what you can do to influence the default memory behaviour of Railo and Tomcat. Reasons for looking into this might be that you have a server with a lot of memory and want to use it (better) for your CFML server. There are plenty of other reasons, but those are beyond the scope of this post.

If you have installed Railo through one of Jordan‘s installers, it essentially gives you a version of Apache Tomcat with Railo deployed (and potentially, depending on your setup, also connect Railo/Tomcat to Apache or IIS).

For Windows users, the installation process normally ends up in a Windows service being installed on the server. The Apache Foundation provides a few tools that help you manage the service(s) and that are really worthwhile looking into. Also have a look at Paul Kukiel’s blog post – it shows and explains the Railo Tomcat Service Control Panel/App. This will help you to set some basic memory parameters and other Java options. Apparently on Windows, you can also tweak settings in the Windows Registry (ugly).

I’m not sure if there are similar UI-based tools for Linux and OS X, I certainly don’t tend to use them to set up daemons/services or for JVM configuration. So, if you’re on one of those platforms and want to make manual adjustments, you’d be looking into using catalina.sh or catalina.bat in Railo’s resp. Tomcat’s bin directory.

There are multiple approaches how you could shove your custom JVM options into the Tomcat configuration this way. Depending on operating system and your preferences you could tweak the catalina.sh or .bat files and set an environment variable in there or you could use your shell’s or system’s (global) way of setting environment variables or use Tomcat’s idea of setenv.sh or .bat (look in section 3.4). I trust you’d find a good way of how to do that for your own circumstances.

Note: Please be aware that if you are running Railo/Tomcat as a Windows service, the aforementioned manual changes to configuration files will not be picked up. It seems you then have to use Tomcat’s own service management tools as mentioned above to configure memory. For more info please refer to Jason Dean’s very comprehensive blog post on that issue.

However, at first glance Tomcat offers two places (environment variables) you could use: JAVA_OPTS and CATALINA_OPTS. In the vast majority of cases the latter is the correct option. There are a few  interesting and very subtle difference between the two (from the docs/config files comments):

CATALINA_OPTS: Java runtime options used when the "start", 
"run" or "debug" command is executed. Include here and not 
in JAVA_OPTS all options, that should only be used by Tomcat 
itself, not by the stop process, the version command etc. 
Examples are heap size, GC logging, JMX ports etc.

JAVA_OPTS: Java runtime options used when any command is 
executed. Include here and not in CATALINA_OPTS all options, 
that should be used by Tomcat and also by the stop process, 
the version command etc. Most options should go into 
CATALINA_OPTS.

So, what does this tell us? CATALINA_OPTS are specific to the Catalina process (Tomcat), JAVA_OPTS might be used for something else going on in your system. Also CATALINA_OPTS are the ones that are supposed to be used to launch the JVM but not the ones to be used for the process that gets spawned off to stop the JVM.

In most cases that differences is negligible, it does make a noticeable difference though when you’re using JMX (for instance to monitor your JVM remotely) as using JAVA_OPTS instead of CATALINA_OPTS for certain JVM settings might then make the shutdown process of Tomcat server actually break (Side note: This is a common occurrence on Adobe ColdFusion 8 and 9 because there’s no clear, default separation of JVM settings for the various start/stop actions in JRun). If you want to know more about this specific issue, read Roni Licht’s excellent post.

A few months ago, one of my clients was testing a possible migration from Adobe ColdFusion 9 to Adobe ColdFusion 10. One of the issues they ran into was a NullPointerException when it came to their cookie use.

Something simple such as <cfcookie name=”cfid” value=”574857485748543″> didn’t quite work and resulted in: “The system has attempted to use an undefined value, which usually indicates a programming error, either in your code or some system code. Null Pointers are another name for undefined values.” with that particular line of code being highlighted as the culprit.

Interestingly enough, this code used to work fine for them in Adobe ColdFusion 9. To provide some additional context, here’s a barebone Application.cfc that shows exactly the behaviour:

<cfcomponent>       
  <cfset this.name = "test_app">
  <cfset this.applicationTimeout = createTimeSpan( 2, 0, 0, 0 ) />
  <cfset this.clientManagement = true />
  <cfset this.sessionManagement = false />
  <cfset this.setClientCookies = false />
 
  <cftry>
    <cfcookie name="cfid" value="123456">
 
    <cfcatch>
      Error:<cfdump var="#cfcatch#">
    </cfcatch>
  </cftry>
               
  Cookie:<cfdump var="#cookie#">
 
  <cfabort>
</cfcomponent>

You might be able to guess now what’s causing the issue here: it’s the fact that the cookie’s value is being set in the pseudo-constructor of Application.cfc. If the respective code is moved into onRequestStart for instance, it will just run fine. The code as shown above will work without any issues in ACF 9.

There’s nothing overly wrong with this – it seems to be a funny little quirk when it comes to “changed behaviours” while upgrading from one version of Adobe ColdFusion to another.

Also when dealing with cookies in ColdFusion 10, make sure to look at the links below – some interesting discussions about slightly changed behaviour and functionality:

http://forums.adobe.com/message/4714631
http://www.shilpikhariwal.com/2012/09/prevention-for-accidental-manipulation.html

Oracle released Java 7 Update 40 back in September. Obviously there are a bunch of improvements in the security department and some new third-party libraries. Apart from this standard stuff, you’ll find a lot of really awesome goodies in this update, too.

Depending on how long you’ve been dealing with Java and JVMs you might remember that there used to be a really awesome JVM from BEA: JRockit (well, it still exists, but that’s a different story). Oracle purchased BEA a few years ago (before they took over Sun). In the JRockit ecosystem there are two really interesting tools available:

• Java Mission Control: monitoring and diagnostics
• Java Flight Recorder: recording events and various metrics at runtime

Check out the Mission Control website and have a play with it, it’s really cool. Be aware though that Java Mission Control is only free for development purposes and that you’ll need a license from Oracle to use it in production.

In January this year, I wrote a blog post to advise people how to make the default installation of a Railo 4 server more secure. One of the elements was to make sure you’re using HttpOnly marked cookies for your session cookies (depending on your setup that might be JSESSIONID or CFID/CFTOKEN).

In the blog post, I’ve described how this can be achieved on a Tomcat context level if you’re using J2EE sessions and JSESSIONID, but there was no solution for using the legacy CFID/CFTOKEN session cookies stemming from the early days of Adobe ColdFusion.

In the meantime, there’s been some activity:

a) Since Railo 4.1, you can create HttpOnly-marked cookies through CFCOOKIE. Even though I haven’t tried it out yet, it’s been clearly documented in the (now closed) ticket RAILO-1339

b) There’s a new ticket (RAILO-2773) to make sure that the client/session cookies of your Railo installation are by default going to be marked HttpOnly. In the comments to Railo-1339, we discussed the potential implications in regards backwards compatibility, but there’s really no reason whatsoever for Javascript to access CFID/CFTOKEN — not even in old legacy apps. Go and vote for RAILO-2773 please! 🙂

In addition to that, Chris Blackwell has suggested a workaround on the Railo mailinglist:

Set the following in Application.cfc:

this.setclientcookies = false;

And then put this in your onRequestStart function:

array('CFID','CFTOKEN','JSESSIONID').each(function(c) {
  if(structKeyExists(session, c)) {
    cookie name=c value="#session[ c ]#" httponly="true";
  }
});

Interesting topic and you’ve got an opinion on it you’d like to share? Just join the discussion thread!

Users of Adobe LiveCycle quite regularly interact with PDF documents. Some examples are:

• Rendering customised documents for print purposes
• Creating PDF forms for on- and offline use to collect data for further processing
• Rendering pre-filled PDF forms to send out to customers/users for completion and physical signature etc.

In a lot of cases those PDF documents are what’s called an XFA-based PDF form/template. XFA is Adobe’s XML Forms Architecture and a proprietary technology to describe form information. I spare you the technical details, but essentially quite often those XFA-based forms get wrapped into a PDF document, this is done by rendering the XFA information over a PDF page background.

Now, in a lot of corporates and large public sector organisations (who tend to be the main users of Adobe LiveCycle) Internet Explorer is commonly used as “the” web browser. If users have Firefox or Chrome installed, they’d quite often be considered either advanced users or would work in the web/communication/IT teams.

Also, organisations as mentioned above are in general Windows-heavy and would generally have Adobe Acrobat Reader or Acrobat in various versions (from shockingly old to recent) deployed to their user base as part of their PC images.

What I’ve seen happening a few times in the recent past is that people would build and deploy XFA-based PDF forms or templates into their systems (or even on their public website for customer/clients to work with) and then realise that those documents create various issues when being rendered in Firefox or Chrome. Examples I’ve seen in the past were buttons randomly not working properly or the form opening up in the browser with weird error messages.

Why is that? Essentially both Chrome and Firefox use their own PDF rendering engine that’s built into the respective browser, IE doesn’t. In general that’s a good thing – Adobe’s Acrobat Reader is kind of overkill and bloated when it comes to rendering static PDF content. For XFA-based documents though, the built-in engines don’t work as well for a variety of reasons.

You essentially have two options here:

a) In some instances you can try to render a static PDF file instead of an XFA-based form/template. That would not work for certain types of dynamic and/or interactive forms though.

b) You can switch off the built-in PDF viewers in Chrome and Firefox and make them use Adobe Acrobat (Reader). That’s usually an option to pursue if your users are internal. If you have external users whose environment you can’t control, that’s obviously not an option and communication with and towards those users is the key.

Symantec has recently discovered a trojan/worm-ish thing that threatens application servers running Apache Tomcat. It seems to follow the typical command & control pattern with control servers having been found in Taiwan and Luxembourg so far.

This threat is using a very specific attack vector by trying to spread via the Apache Tomcat Managers and their (quite often unchanged) weak passwords and the weak users/passwords Apache Tomcat ships with as (commented and not by default active) examples . If it’s successful, it’ll try to deploy itself as a servlet and the cycle restarts.

You can prevent the whole thing from happening if you either have disabled the Manager applications or have setup users with non-default, strong passwords. The file to check is tomcat-users.xml and its content.

Why is this even interesting for Adobe ColdFusion and Railo users? Mainly because a lot of people run their CFML servers on Apache Tomcat. There are various use cases of which you should be aware of:

a) Adobe ColdFusion 9 as a single server install is safe from this attack as it’d be using JRun. In Adobe ColdFusion 9 Tomcat deployment  was never properly supported officially, but certainly doable through custom .war deployments. If you did that, it’s quite likely that you might be running a full version of Apache Tomcat and that you might be vulnerable, depending on your setup (see above).

b) Adobe ColdFusion 10 comes with a preinstalled and embedded Tomcat instance if you do a single server install. That could theoretically expose you. However, I’ve checked an install I’m running on OS X and it seems that there are no users and roles enabled in the configuration either. Again, if you’ve done a custom J2E deployment on your own Tomcat – make sure you check that and know what you’re doing.

c) Current Railo 4 installers and custom installs: The installers are all very safe and secure by default, there haven’t been any modifications to the users/role setup. The current 4.1 installers don’t even install the Tomcat-own webapps. If you used a vanilla Tomcat from Apache’s website and dropped Railo into that as a .war/.jar file you’ll be fine, too, as there are no users enabled for the Tomcat Manager apps.

The essence is: by default you should be fine according to what I’ve seen. If you’ve modified your Apache Tomcat setup in any way, please make sure you’re staying safe as well. Not to forget that the full credit for making me aware of the Tomcat threat in the first place goes to Jordan Michaels.

Updated (29/11/2013): Changed the wording in 2nd paragraph.

So, there we go. Adobe got hacked and according to Krebs on Security and Adobe themselves, among other things, the source code of ColdFusion, ColdFusion Builder and other Adobe products has been stolen and shown up on hacker sites.

This is obviously an issue. I don’t want to comment on how it might or might not have happened and what the implications are for Adobe Acrobat (Reader) users. Let’s have a think about Adobe ColdFusion, a hosted server product.

First of all, according to my knowledge, it’s not clear yet which versions of Adobe ColdFusion’s source code has been stolen/leaked. It might be CF 4 or 5 and therefore less of a worry. More likely however is that it’s a or multiple recent versions of ColdFusion. What does that mean for both Adobe and you, your users and customers? I’ll try to look at the problems this development is most likely going to cause from a few different angles. Your mileage might vary.

1. Hackers have the source code of a server product that’s being hosted on thousands of web and application servers all over the world. A major user for instance is the US Federal Government and currently the European Union (even though there are strong movements towards changing to Railo people within EU organisations looking into moving towards Railo in the latter).
2. ColdFusion 9 and 10 recently had a a reputation for being easily hackable due to some exploits that got not that well patched and even then the patches are notoriously difficult to apply for administrators. This has become much better with ColdFusion 10 though. There has been a whole series of hacks against hosting companies and users of CF servers.
3. Take 1 and 2 and you can safely expect that there will be a bunch of new exploits coming along for CF users. This time about 10x worse because hackers have the actual source code.
4. Some people argued on mailing lists or in blogs that hackers could just decompile the Java bytecode CF is delivered through. That is true. However, if you HAD ever done that, you’d see that it’s much easier for a hacker (or anyone, really) to go through actual, probably commented, source code instead of auto-generated decompiled-from-a-bytecode-file source code.
5. Trust: When one buys a commercial, closed source product, one puts a certain level of trust into the vendor of the product and its support. It’s implicit “security-by-keeping-the-source-code-in-a-vault”. I don’t want to discuss if that’s good or bad in the first place, but now that vault has been broken into. So – the people who know about bugs, security holes and other issues are not solely Adobe anymore, but Adobe and the Hackers. Can I safely trust an CF installation from now on? Can people safely recommend customers to install CF? I struggle with that thought.
6. Trust flow-on effects: I wonder how long it takes until organisations like PCI have issues with issuing CF-driven sites as PCI-compliant in an e-commerce context. A commercial application server of which the hacked source code is out in the wild – is that trustworthy?
7. Some people argued recently that Railo’s and OpenBD’s source code are open and out there as well. Wouldn’t that be the same then? The short answer – no. Both Railo and OpenBD are open source. It’s intentionally opened. There’s a community, there’s transparency. If you don’t trust Railo, inspect the source code – that’s (among other things) why it’s out there.

What’s the way out here for Adobe? I honestly don’t know – but my gut feeling is that their only option would be to in some way open source the product for everyone while keeping commercial distribution rights (I assume they still try to make money from it). I’m at this stage not even sure if and how that’d be feasible. In the meantime – be aware of this situation and that there will be a sting a hacking attacks soon. It’s not a question of IF, but WHEN.