David's Blog Posts

Flux 7.8 Beta 1 is Available!

The first beta release of Flux 7.8 is available to existing Flux users on the Flux Portal.

(Flux Portal users can download it by selecting "7.8" in the drop-down box in the upper-right corner of the Flux Portal where it says, "Select Flux Version". Then click Downloads on the menu bar. That will navigate you to the 7.8 beta download file.)

Flux 7.8 Beta 1 already has a temporary license key built-in, so you can start playing with Flux 7.8 right away.

So, what's new in Flux 7.8? We took some things that were originally planned for Flux 8 and added a couple other things. Here is the rough list of what's new:

Database actions for querying and updating databases
REST action for invoking REST services
Web Service action improvement so that it properly supports modern Web services available today
FlowContext.isExpedited() method to allow an action to determine if it is running due to having been expedited
Persistent variable rules expanded to support POJOs (plain old Java objects -- Java objects having the usual getter and setter methods)
PostgreSQL database support
In-memory database upgraded to the H2 database, which removes the previous limits imposed by HSQL
Documentation and example improvements
Java 5 or Java 6 required.

Our continuous integration server is running positive for all tests across all supported databases. We are continuing to test Flux 7.8. The release date will be 24 March 2009.

Flux 7.8 Release Targeted for 24 March 2009

We are very close to having a beta ready for Flux 7.8. The generally available release for Flux 7.8 is targeted for Tuesday, 24 March 2009.

New features in Flux 7.8 include:

Database actions for querying and updating databases
REST action for invoking REST services
Web Service action improvement so that it properly supports modern Web services available today
FlowContext.isExpedited() method to allow an action to determine if it is running due to having been expedited
Persistent variable rules expanded to support POJOs (plain old Java objects -- Java objects having the usual getter and setter methods)
PostgreSQL database support
In-memory database upgraded to the H2 database, which removes the previous limits imposed by HSQL
JMS 1.1 support
Documentation and example improvements
Java 5 or Java 6 required. As a result of this requirement, an explicit xerces.jar dependency is no longer required.

Flux 7.8 includes elements that were originally implemented in Flux 8.0, which has not been released yet.

Better WebSphere Support in Flux 8

Flux 8, scheduled for release in the first quarter of 2009, will support WebSphere better. Here's how.

Flux 8 will allow indirect JNDI lookups, which use the java:comp/env prefix. In Flux 7.x, a job or workflow can do direct JNDI lookups but indirect JNDI lookups are not supported. Typical JNDI lookups include accessing data sources and EJBs. When you do a direct JNDI lookup from a Flux 7.x job or workflow running inside WebSphere, there are two primary concerns.

Direct JNDI lookups are deprecated beginning in WebSphere 6. You'll receive a warning message when you use them, which will probably be logged repeatedly. Furthermore, since direct JNDI lookups are deprecated, you may wonder if WebSphere will completely drop support for them sometime in the future.
Direct JNDI lookups are undesirable, from a Java developer's point of view, since you are looking up a resource in the global JNDI namespace and you probably cannot depend on that resource being available to you at that exact position in the global namespace. Better that your Java code reference a location in your application's local namespace using a resource reference, which, in turn, can point to a proper data source or EJB. Furthermore, in the resource reference, you can tune the properties of how you use the reference, something not available to you when using direct JNDI lookups.

Flux 8 can use WebSphere-managed threads to perform work instead of directly creating threads. In Flux 7.x, Flux creates its own threads to do its work, even when Flux is instantiated inside WebSphere. When Flux creates its own threads, Flux still runs normally within WebSphere and behaves correctly. However, there are two primary concerns:

WebSphere may log warnings when Flux creates a thread.
WebSphere may drop support for such "unmanaged" threads sometime in the future.

This improved WebSphere support occurred because Flux 8 is adding Work Manager support. In essence, a WebSphere CommonJ WorkManager performs work asynchronously, using its own thread pool. When properly configured, Flux can delegate all its background processing to Work Manager threads. In this case, all of Flux's background, asynchronous processing is performed by threads managed by WebSphere ("managed threads"). When appropriately configured, Flux will not use its own "unmanaged threads", which WebSphere has deprecated.

When it comes right to the heart of the matter, Flux 8 will have better support for WebSphere by making it possible to:

perform indirect JNDI lookups for data sources and EJBs
use WebSphere's Work Manager to make use of managed threads
resolve warning messages
remove concern for use of techniques that WebSphere has deprecated

Thread-safe objects formally documented in Flux

Developers often want to know exactly which Java objects they use are thread-safe. Equally as important, developers want to know which Java objects are not thread-safe. Of course, it's important to know when it's safe to have lots of threads invoking methods on objects simultaneously.

Starting with Flux 8.0, thread-safe objects in Flux are formally documented in Flux's Javadoc API documentation. Here's a snapshot of the Flux 8.0 Javadoc API documentation.

In the image above, you can see the top of the Javadoc for flux.repository.RepositoryAdministrator . In the highlighted section at the bottom, it says, " Thread-safe. Can be invoked safely from multiple threads simultaneously. "

This is the formal notation indicating that the RepositoryAdministrator class is thread-safe.

In the absence of this formal notation, you should assume a Flux class is not thread-safe. For example, we make no guarantees that the flux.FlowChart class is thread-safe. (It doesn't need to be.)

In case you're interested what tools I used to generate this thread-safe documentation, it's nothing fancy. I first searched online to find out what other Java API documentation writers were using. I found Alex Miller's blog posting about documenting thread safety .

Alex and others reference some concurrency annotations from the book Java Concurrency in Practice .

So, I tagged Flux's thread-safe classes with @ThreadSafe in the source code. The @ThreadSafe tags are up in the class declaration, right beneath the @author tag. That was easy enough.

Then I marked up our Maven pom.xml file, which uses Maven's Javadoc plugin, to describe the verbiage to be emitted when the @ThreadSafe tag is encountered.

Running mvn javadoc:javadoc in our build environment produces standard Javadoc along with the customized notices about thread-safety that you see above.

Having Flux's thread-safe classes formally documented in this way will be a productive boost for Java developers using the Flux APIs.

R.I.P. flux.Factory

Eight years ago, I wrote the first version of Flux's Factory class. It was the starting point for Flux users. Flux users would first make an instance of Flux's Factory class and then use it to create other objects involved in Flux programming.

Yesterday, I deleted the Factory class from the upcoming Flux 8.0 code base.

Back in 2000, using the factory design pattern was the natural choice for Flux. These days, it's become clear that Flux needs to use simple classes instead of the factory design pattern.

It felt good to simplify.

Lightweight or heavyweight job scheduler?

On a fairly regular basis, we're asked how Flux is different than one of the large, enterprise-class job schedulers like AutoSys and CONTROL-M. The short answer is: apples and oranges.

Is someone on your team doing Java development?

Answer Yes : an embeddable Java job scheduler is the lightweight solution I would use.

Answer No : a stand-alone job scheduler server like AutoSys, CONTROL-M, etc is the heavyweight solution I would use.

I've written a short technical article that addresses this topic. I hope you find it useful if you're sorting out whether you should use an embeddable, lightweight Java job scheduler like Flux or a stand-alone, heavyweight job scheduler server like CONTROL-M or AutoSys.

What's your opinion? Leave a comment or call me on skype at flux.david

New in-memory database for Flux: H2

Flux has used the HSQL in-memory database since Flux's launch in 2000. It's a nice and easy way to get programming with simple Flux examples in literally five minutes after downloading and installing Flux.

However, as I've talked about before, HSQL has its limitations.

The Derby database is now supported by the forthcoming Flux 8.0. It's a simple, embeddable, well supported database (Apache, Sun) that stores jobs and workflows persistently without using an enterprise class database such as Oracle, DB2, or MySQL.

From a Flux perspective, the only problem with Derby is that it doesn't support an in-memory mode, which can make things fast and simple. Fast, because an in-memory database doesn't hit the disk. Simple, because a fresh in-memory database is created each time a Flux engine is created.

Now enter H2 . H2 is the popular successor to HSQL. It overcomes the glaring limitations in HSQL, supports an in-memory mode, and is still fast!

Yesterday I integrated H2 into Flux 8.0. It was easy.

I just ran a simple speed test of loading and firing a simple Java Action job. H2 took about 4.5 seconds. Derby took between 6.5 and 7.5 seconds.

All in all, H2 provides (a) simplicity over Derby since a fresh database is created whenever a Flux engine is created and (b) speed over Derby.

Of course, Derby is still very handy since it provides a robust persistence option. Since Derby is supported by both Apache and Sun, I have more confidence in it for data persistence than H2's persistence mode, which Flux won't be using.

H2 is now the default database in Flux 8.0. Derby is available as an option. Both H2 and Derby will ship embedded with Flux 8.0 so you'll never even have to include their jar files on your class path. Just include flux.jar.

Towards a Flux POJO programming model

I want to introduce the idea of programming in Flux using POJOs (plain old Java objects). In a Flux release later this year, you'll be able to instantiate Flux objects as POJOs in addition to the traditional factory approach that Flux has employed since its inception.

The historical perspective is simple. Back in 2000 when Flux 1.0 was released, J2EE was the most important enterprise Java technology, and J2EE architectures used factories extensively. That concept influenced Flux 1.0. For that reason, Flux uses the factory design pattern to instantiate objects.

Today, the POJO programming model is the modern way. After all, it's simpler. It also makes it easier to integrate with other frameworks, most notably Spring, but Spring is certainly not the only framework that Flux could plug into easier if Flux supported a POJO programming model.

So, what will the new Flux APIs look like? Picture lines of code like this:

new FlowChart();
new TimerTrigger();
FlowChart.add(timerTrigger);
new Configuration();
new Engine(configuration);

I'm working on this task now and will report back once something more substantial is in place. Comments? Questions?

HSQL to Derby

By default, Flux stores its information in an HSQL in-memory database. Flux is preconfigured with HSQL by default so that people can download Flux and get started with running simple examples within a few minutes. No need to configure your Oracle, DB2, or MySQL instance.

Over the years, a few problems have arisen from this approach:

1) HSQL's database transaction isolation level is READ_UNCOMMITTED (instead of the usual READ_COMMITTED), which places severe limitations on Flux's ability to run multiple jobs and workflows concurrently. In short, when using HSQL as its database, Flux can run at most one job or workflow at a time.

2) Flux requires up to three database connections for its internal use. Due to this requirement, not all of Flux's features are available when using HSQL. For one, Flux messaging doesn't work when using HSQL.

3) It would be useful if HSQL could store its information persistently. It can, actually, but it performs so poorly with sizable databases that Flux explicitly prevents HSQL from running in a persistent mode. For a Flux user to get a persistent database working with Flux, it would be nice if Flux's built-in database supported it. Not everyone wants to go to a full-fledged database such as Oracle, DB2, or MySQL.

Moving from HSQL to Derby would solve these problems.

Derby is well supported, would allow many jobs and workflows to run concurrently from the get-go, and would allow for an option to persistently store Flux data.

The plan for Flux 8.0 is to move from HSQL to Derby. For nearly all those using HSQL with Flux, the transition will be transparent and seamless.

Flux 8.0 development begins imminently. Flux 7.7 is very nearly ready to be released. Flux 8.0 starts afterwards.

I've already coded a proof-of-concept transition from HSQL to Derby in the Flux 7.7 code base. It was straightforward. I ran Derby-based Flux jobs and workflows.

It was cool.

Resources to help you learn and use Flux better

Here at Flux, we have a lot of resources that are new or updated as of late. They could be useful to you or your team to help you flatten your Flux learning curve or to help you use Flux better.

Here are seven helpful resources updated as of late.

New website: We reworked the content on our website and the visual design to provide more information than before. It's organized better. The goal is to help you find what you need to learn Flux and to use Flux better.
Blogs: We blog. We do it to give you insight into what we're thinking and how we're planning to help our customers succeed. There are truly useful things in our blogs.
New customer portal: We updated the Flux Portal to better organize the info that is available. This includes discussion forums with other Flux users. It all helps you learn and use Flux.
New technical specs page: It lists Flux's technical specifications and is updated regularly.
New performance metrics: Flux engine performance data is online. The link to the performance metrics is near the bottom of the technical specs page.
Comparison to Quartz: This comparison document gives a straightforward, unbiased viewpoint of when it's appropriate to use each.
Technical articles: Short, to-the-point knowledge base articles that present solutions to common Flux situations.

How else can we help you? What else would you like to see?