Greetings dear Petals users,
we are very glad to announce the public availability of our latest ESB, Petals ESB 4.0!
This distribution packs sweeties.

• Latest container with new kernel 3.2, which brings strong enhancements under the hood: beyond usual performance/stability profits, many parts have been rewritten to prepare for exciting new features (yeah, we're teasing you :)). Beware though, in spite of our care, there may be a few regressions, which we'll detect and fix as fast as we can.
• A new business monitoring system: no more Petals View+KPI, now all is based on plain texts logs, for extensive configurability and choice in third-party exploitation tool.
• A new Command Line Interface: now brings all essential administration tools in one dedicated shell, usable locally or remotely (including replacement for old "stop" and "shutdown" scripts).
• Pack of updates for most used components, to make them compatible with the new monitoring: Filetransfer, FTP, Mail, SFTP, SOAP, SQL, BPEL, EIP, JSR, Quartz, Validation, XSLT.
• BC-SOAP now supports WS-Security and SSL connexions.
• BC-Filetransfer is simpler to use as a service provider, and is now ready for high availability use-cases as a consumer.
• Petals startup script has been improved, using your feedback as guidelines.

Also, please note that package structure of the container and our versioning policy has slightly evolved. 

You're welcome to download Petals ESB 4.0 and try it at work... Or at home? For more information about an individual novelty, please refer to previous news on the blog.

We hope you'll enjoy Petals ESB 4.0, and are waiting for your feedback on Petals forum.

Permalink | Leave a comment  »

Working for PetalsLink was a quite interesting experience:

On the technical side, I enjoyed moving all the XML-based tooling of PetalsStudio to a more powerful EMF-based approach for Petals JBI editor – for those who don’t know JBI, it is a standard that allows to define SOA artifacts in your ESB. Moving to EMF allowed us to provide better tooling faster, because most of the complexity in manipulating JBI can be removed with very few efforts leveraging EMF ExtendedMetaData. That was the first time I faced this part of EMF, and I got pretty impressed of how well it works (working with EMF always gives this impression of “being well”). I also improved the ability to plug new JBI components into the Studio, which is a critical point when you have to deal with connectors for almost everything – Mail, SFTP, Talend, XSLT…. So that was an interesting challenge in term of conception and development.
Petals Studio was also the pretext to start using Git, GitHub and Sonar. I am pretty happy to have learnt these 3 tools that clearly improved the way I work.

Also, I had the great opportunity to work closely to several Eclipse projects:

• I could contribute the Tycho build of GMF Tooling, put it on Hudson, get source moved to Git/mirrored to GitHub, improve wiki… GMF Tooling is a project I’ve used for 3 years now and I often saw in it some critical organization points to improve to make it more dynamic in term of development. Working at PetalsLink gave me the opportunity to do what I think was necessary to keep the project healthy. With the help of Michael Golubev, I now think this was an real success.
• I could contribute to Nebula the TreeMapper widget, which will probably have some very interesting use-cases soon. As I became a committer, I also helped in improving Tycho build and CI, nad it seems like the project liked it if we look at the new p2 update-sites.
• I contributed some small improvements to Eclipse BPEL designer, tried (unsuccessfully) to make SWTBot use Tycho, and developed a useful extension for Draw2d.

The only thing I wish I would be able to do here is to push ahead the usage of Sonar at Eclipse, at least for GMF Tooling and Nebula.

But I probably learnt even more things from PetalsLink by discovering another company organisation that is very different from what I could experiment before (OpenWide and BonitaSoft): PetalsLink is focused on the Research about SOA and agility of Systems of Information. It is a wide topic! Petals products are quite good compared to other alternatives in the SOA landscape, but they don’t meet the success they deserve, it was a bit frustrating for a developer.

I enjoyed working for PetalsLink, all the expectations are fulfilled, so it is time for me to go ahead, to find a new experience, a new team, new challenges, new issues… I love discovering new things!

That’s why I’ll start working for JBoss|Red Hat tomorrow, as part of the team developing JBoss Tools and JBoss Developer Studio.

I’ll have the opportunity to work with a great team! My main occupation for the next monthes will be to assist Nick Boldt in making JBoss Tools CI and build infrastructure better and better. I’d also like to open the road towards efficient QA for JBoss Tools, including -among other- usage of Jacoco and Sonar. Then I’ll also work on developing nice stuff for some JBoss Tools modules, most probably on the SOA/BPM part.

That’s gonna be a lot of fun! I’m eager to be tomorrow and actually get started for this new team/employer/project/product/users.

Let’s keep in touch via this blog and twitter

As a platform for exchanging messages, Petals ESB needs monitoring capabilities. First, technical monitoring, to ensure the behaviour of the bus with load increase. Then, a way to know whatever happened in the bus (what messages were exchanged, what services were invoked, etc).

We provided dedicated tools to fulfill these needs. Thanks to your feedbacks, we've been able to define what to keep, and what to improve, to give you better tools.

As such, major changes are coming in these fields, from Petals 4.0 onwards.

Technical monitoring

The first mission has been fulfilled by the Petals Webconsole since a few years, with a nice GUI and administration/testing features (start/stop components, run mockup services, monitor performances...).

However, this tool was mainly built to help developers setting up their project, and wasn't always fit for a production environment.

Hence, we're currently improving the built-in SNMP probes existing in Petals. This will allow fine-grained, tailored technical monitoring with the use of tools such as Nagios or Cacti.

Messages monitoring

Being able to monitor the messages flows and their content is the first step towards a high-level business monitoring. So far, this was realized with Petals View, a dedicated GUI combined with a notifications component. While this worked fine ;), several users requested a more flexible solution. So, we're currently building a new system based on text logs. The first step will be Petals 4.0, in which logs will contain all necessary information to retrace a message flow.

Meanwhile, we are considering several options for logs exploitation, from logs aggregators to dedicated reporting tools, or API for everyone to build their own application upon.

We'll be eager to get your opinion and needs on this topic, when you get to see the first results with Petals 4.0!

Permalink | Leave a comment  »

Hi all!

some of you have had a hint on what would be in next release of Petals ESB. Apart from bringing a new business monitoring system, the new vintage of Petals will mirror the great efforts accomplished in 2011 in enhancing our tests batteries and cross-integration evaluation.

Another point that was simple to improve, thanks to the feedback of our users, was the way to get hands on Petals and getting the hand of it.

So far, we waited for the core of Petals ESB (the "container") to come to a new release, and we published it altogether with any and all components that were improved during the same time. We called this container "Petals ESB x.x.x", numbering it after the version number of the container. Then, each user had to download this so-called "Petals ESB" and any other component separately.

This had some advantages, but also some drawbacks:

"tunneling effect": some components with great improvments could be released earlier for better user benefit.

confusion about what actually IS Petals ESB get: what we called the ESB was actually its core, or skeleton, but it wasn't very useful before you take some components to use with it.

difficulty of "assembling" your dream team: thanks to our great variety of components, each one having its own roadmap and lifecycle, it could be difficult at times for newcomers to pick up the best component version for a given container.

Things will change with Petals ESB 4.0:

• container version number and distribution number will be decorrelated.
• each component will be released following its own lifecycle, without waiting for the others.
• once or twice a year, we'll publish a package comprising the container and a set of recommanded components to use with it, and this is what we'll now call Petals ESB.
• for expert users, each component and container will still be downloadable separately in any version.

These changes have been decided to make your life easier. We strongly apologize for the confusion it may cause during the transition period.

Yours sincerely,

Petals Team

Permalink | Leave a comment  »

You are right, I need to smile more, be less tired and have a demo of the BPM editor working on low resolution displays… BTW, the demo of the DSB Monitoring & Management console used to deploy and monitor BPEL process works.

I let some traces in this article (as an example and for me, for later).
Notice that the apply command did not work with EGit, I had to use the command line. I guess it will be fixed in a future version.

• First, we need a patch, given as a diff file, e.g. in a bug entry.
• As a committer, you should apply and check this patch. So, you first need to copy the patch in your (cloned) Git repository.
• Get some stats about the patch :

  git apply --stat myPatchFile

• Make sure the patch can be applied :

  git apply --check myPatchFile

• Apply the patch :

  git apply myPatchFile

• Delete the patch file.

Then, you can check that the patch works and then commit it, before pushing the modification.
One important thing is to keep the author’s name in the commit. The author and the committer are different. And both can be kept in the commit and in the history. Here is an example in the command line:

git commit -a --author="authorUsername <author.mail.address@sth.com>"

After the push, both the author and commiter names will appear in the repository.

I don’t know if everyone in Eclipse projects uses this feature, but from the community perspective, this is very interesting (IMO).

WTF, my server is dead again!?

This app is a simple heartbeat manager looking at remote HTTP services and notifying you by email when something becomes unreachable. It uses the background Job feature of the Play framework and just does HTTP GETs on the specified list. That’s all, that’s simple, but that’s can be useful sometimes…

The code is located at https://github.com/chamerling/heartbeat-manager

So, I discussed with GitHub guys through their support channel (Here I want to say that I am really impressed on how fast and professional is the GitHub support team!). Of course, they also told me that they will never use QuickHub if OAuth is not provided. After discussions with the guys, I started to implement a workaround to provide OAuth support in QuickHub by using the OAuth Web flow and adding some stuff to QuickHub.

Let’s see how we can do that in a generic way so that you can use it in your app if needed since every serious Internet service also provide OAuth support… Note that I will not give many details about OAuth itself but I will use some terms, so have a look to OAuth documentation for more details.

Register a Web application

The first step is to register your application to the developer portal. Here you need to provide some information such as the app name and its callback URL. Even if we are developing a desktop app and not a Web one, we need to be able to provide this HTTP based callback URL. We will see in the next section what is inside this application. By registering our application, the service will generate and give us some keys to be used in the next steps, mostly to recognize us when calling the service.

Create a Web application

We need a Web application to handle callback calls from the service we want to use OAuth. This application will only be used at authorization time and just have to be able to receive the service call, get the code sent by the service, then call the service again with the code and our application keys (there is a secret one to be used here). As a result, the service will send you back your OAuth token to be used in all the future service calls. This token is used to authenticate your service calls, no more password stuff inside! Here is a quick sequence diagram showing all the exchanges between actors…

On the QuickHub side, I chose to create this Web application by using the Play Framework I already mentioned several times in this blog. I used the Heroku paas to provide the Web application publicly.

As showed in this Gist (https://gist.github.com/1466592), the callback method is really simple (as usual with Play!): just get the code and call GitHub with it and your application keys. When all lis done, just display a result page with a specific URL. Here it starts with ‘quickhubapp://oauth?’. Let’s see what it means in the next section…

Add custom URL handlers to your desktop app

Once the desktop application is authorized, our Web application redirects the user to a Web page which embeds a button targeting an URL starting by ‘quickhubapp://oauth?’. Here you already understand that by clicking on such a link must drive you to something special. The cocoa framework allows developers to register specific URL handlers for their applications. So we need to register QuickHub handlers so that OS X knows that every link  with the quickhubapp scheme must me routed to QuickHub. This is as easy as showed in this Gist https://gist.github.com/1466628.

This code snippet just tells QuickHub to invoke the getUrl method when it receives a quickhubapp event. Up to the getUrl method to handle things. In QuickHub, I just extract the OAuth token in order to persist it locally for future use.

Done!

So now we are able to use OAuth Web flows for desktop applications. In the best of the worlds, every service provider may provide a real desktop flow to be used without the need to create this additional web application. In the real world, it is not the case but as you see it can be bypassed quite easily.

I will provide more technical details on the second section ‘Create a Web application’ with real sample and code in the next weeks.

Hi dear Petals community!

Here is a big catch-up, for all who couldn't follow news on www.petalslink.com ;)
These last three months have been quite busy, with the releases of the following:

Petals Studio 1.2: New and fresh, with lots of improvments and features, and compatibility with latest component updates! All info and links here or get straight to the download.

Petals ESB 4 Preview: a milestone of Petals ESB 4 has been released. Get a peek on next major edition of Petals ESB!

Petals Components and Dev Kit: BC-SOAP and SE-POJO have been updated, as well as our Component Development Kit.

Petals BPM: discover our latest asset in SOA infrastructure and management! Petals BPM brings business process design through a light and slick GUI!

Stay tuned! Our new service persistence component, SE-ASE, and release of Petals ESB 4, are coming soon!

Your Petals team

Permalink | Leave a comment  »

Background

Historically, Eclipse has 4 to 5 classical qualifier for binary artifacts:

• Release
• Maintenance
• Stable
• Integration
• Nightly

This wording is specific to Eclipse, only Eclipse people do understand the meaning of it. Even worse, some of these are not used accurately. Although this is more or less official wording, I am not sure it is used by most projects.

This guy has driven a revolution in the way we build and deliver software

Now  Eclipse.org provides continuous integration, to automate and manage builds executions, and most of them happen whenever a change happen on your VCS. Continuous integration has highly changed the way binary are produced and made available to consumers, it is now much easier to get builds, lots of project have a ping of less that 10 minutes between a commit and a build ready to be released.

That was the starting point of my thoughts, with the Nebula build job: Why calling nightly a job that does run on a build any time a commit happen?

Requalifying binaries to make consumption clearer

As a producer of builds, here are my opinion on these qualifiers:

• Release: The heartbeat of the product for consumers
• Maintenance: is a release, but with 3rd qualifier digit different than 0
• Stable: Is nothing but an integration build that was put on a temporary update-site for the release train. And to be honest, I don’t use it, I directly point the latest good continuous integration builds to the release train builder
• Integration: The heartbeat of the product for developers, built on each commit
  
• Nightly: What is in it? What does nighly mean at the same time for a developer in Ottawa and a developper in Beijing? Who cares that it is build nightly and not a 2pm CEST? For most projects, the nightly built artifacts are not at all different from the ones that would have been built with the last commit. What is the added value of artifacts built during a nighly scheduled build over the latest artifact built from last commit? Why building something every night if there was no commit in it for 3 monthes?

So I am in favor of removing Maintenance, Stable and Nightly. That gives

• Release
• Integration

Wow, 2 kinds of builds, reminds me a tool… Oh yes! Maven! Maven has 2 types of binaries: Release and SNAPSHOTS. That’s great, we finally arrive to the same conclusion as Maven, except that what Maven calls “snapshot” is called “integration” in the Eclipse teminology.

But now, let’s consider the pure wording: why having 2 names for the same thing? How to call a binary that is work in progress? An “Integration” or a “snapshot”?

Let’s be pragmatic: This report tells us there are about 9000000 Java developers in the world. This onetells us 53% of this population use Maven. Let’s say that about 60% of Maven users do understand the meaning of SNAPSHOT. That means 9000000 * 53% * 60% = 2,862,000 people do know what SNAPSHOT means.  Wayne confirmed me that the size of the Eclipse community is 132,284 people, who might know the meaning of “integration” in an Eclipse update-site name. That’s the number of people who have an account on eclipse.org sites – wiki, forum, bugzilla, marketplace. Even if we assume that 100% of them do understand the differences between the different qualifiers, and I am pretty sure that less than the 600 committers actually do, that makes that Snapshot is 21 times a more popular and understood word than integration.

So, Eclipse projects and update sites would be easier to understand and consume for the whole Java community by accepting the Maven wording, and using it on the websites and update-sites name.
Following the Maven naming and the release/snapshots dogmas would make consumption easier, but also avoid duplication of built artifacts, and also make things go more “continuously”. Your project is on rails, it’s going ahead with Snapshots, and sometimes make stops at a Release. That’s also a step ahead towards the present (see how GitHub works) of software delivery: continuous improvement, continuous delivery.

Requalifying build job to make production clearer

So now let’s speak about build management, no more about delivery.

You need several jobs to keep both quality and fast enough feedback, then set up several jobs!

Dave Carver reminded me about some basics of Continuous Integration: keep short feedback loops, one builds for each thing. That’s true. If you have a “short” build for acceptance and a “long” build for QA, you need to have separated jobs. Developers need acceptance feedback, they also need QA feedback, but they for sure cannot wait for a long build to have short-term feedback. Otherwise, they’ll drink too much coffee while waiting, it is bad for their health.

But do not create several builds until you have real needs (metrics can also help you to see whether you have need). If you have a 40 minutes full-build, that fails rarely, with slow commit activity, and that nobody is synchronously waiting for this build to go ahead, then multiplying builds and separating reports could be expensive for low Return On Investment. That’s the case for GMF-Tooling: we have a 37 minutes build with compile + javadoc + tests + coverage + signing, and we are currently quite happy with it, no need to spend more effort on it now. Let’s see how we feel when there will be a Sonar instance at Eclipse.org and that we’d have enabled static analysis… Maybe it will be time to split job.

Avoid scheduling build jobs, it makes you less agile

Before scheduling a job that could happen on each commit, just think about how long will be the feedback loop between a commit and the feedback you’ll get: it is the time that will happen between the commit and the clock event starting your job. It can be sooooo long! Why not starting it on commit and get results as soon as possible? Also why schedule and run builds when nothing may change between to schedule triggers?
I can only see one use case where scheduling job executions is relevant: when you have lots of build stuff in the pipe of your build server, and you have limited resources. Then, in that specific but common case, you want to set up priorities: you don’t want a long-running QA job to slow down your dev team who is waiting for feedbacks from a faster acceptance build. For this reason, I would use scheduling, but I would do so because I have no better idea on how to ensure the team will get the acceptance feedback when necessary, it is a matter of priority. Maybe inversting in more hardware would be useful. Then you could stop scheduling, and get all your builds giving feedbacks as soon as they can. Nobody would wait for a schedule event to be able to go ahead the project.
As I said to Zoltán on Twitter: “The best is to have all build reports as soon as possible, as often as possible” (and of course only when necessary, don’t build something that has already been built!). Scheduling goes often against those objectives, but it is sometimes helps to avoid some bottlenecks in the build queue, and then save time projects.

Name it to know what it provides!

Remember Into the Wild or Doctor Zhivago... "By its right name"...

Ok, at Eclipse, there are “Nightly” jobs. I don’t like this word. Once again, “Nightly” is meaningless in a worldwide community. And the most important issue are “What does this build provide?”, “Why is it nightly, can’t I get one on each commit?”.
If this build is run on each commit, then don’t call it “Nightly”, because it feeds the consumer with false information. You can think about having both “acceptance” and “QA” jobs, then put that in their names rather than a schedule info, that’s a far more relevant information.

Conclusion

Continuous integration has changed the way we produce and deliver software, we must benefit of it and adopt the good practices that come with it. Continuous integration is the first step towards what seems to be the present of near future of software delivery: continuous improvement and continuous delivery. We must not miss that step if we want to stay efficient.