From Java to Java EE (through C# .NET)

Creating Mylyn Connector Plugin - part 1

2010-03-08T13:00:00.002+01:00

Quite recently I struggled with my company's internal bug tracker, especially lack of Mylyn integration in Eclipse. The easiest way to make it work was just to develop it :) I started with Mylyn/Integrator Reference but as usual Eclipse documentation sucks (I used to develop a lot of SWT/JFace applications and the best documentation I found was outside of Eclipse). It's too short, too shallow and most of all written by/to people who knows Mylyn APIs and architecture quite well. As far as I understand someone who wants just to integrate Mylyn with some bug tracker doesn't have to have this kind of knowledge, but I also understand that Mylyn guys don't really have time for stuff like this.

Anyway, I also found very short tutorial (http://jvliet.blogspot.com/2007/02/creating-mylar-connector-plugin-for.html) explaining in a bit more details how to create Eclipse PDT project etc. But this tutorial doesn't explain all the tiny details that made me spend many hours wondering HTF it should all work together.

In this short series of posts I'll try to give you much simpler tutorial (for morons like me) how to start new Mylyn Connector Plugin even if you're new to Eclipse Plug-in Development. I'll save your time!

OK - let's start. First video shows how to checkout Eclipse Mylyn and Trac packages (only packages required for connector development reference). In all tutorials authors say that you should take Trac Connector as example but they never point to concrete CVS repository and CVSROOT directory. I do:

Second video shows how to create new Mylyn connector project and make it visible in the new Mylyn Connection wizard (I took icons from Trac for simplicity - you can/should use your own). All steps in this tutorial are MANDATORY - if you miss something I'm 99.9% sure it will not work:

Enjoy and stay tuned for more videos in the upcoming weeks...

PS. I used Eclipse 3.5 (Galileo):

JavaOne is dead?

2010-02-03T08:06:00.002+01:00

I was just looking for the information regarding JavaOne 2010 and found some worrisome blog posts:

Was JavaOne 2009 the last?
JavaOne 2010?
JavaONE 2010? Seems Like It Will Take Place

According to the Moscone Center website JavaOne 2010 is already scheduled between 22nd - 25th of June 2010.

I hope to go there this year! Keep your fingers crossed for my dept's budget :)

Memory Leak Protection in Tomcat 7 - fwd

2010-02-02T10:33:00.002+01:00

Picture (c) DZone

I'm just forwarding you to a good article on memory leaks protection implementation in the upcoming Apache Tomcat 7.

This article is a in a form of short (but quite comprehensive) interview with:

Mark Thomas, a Senior Software Engineer with SpringSource and committer for Apache Tomcat

Mark points directly to SVN repository where you can actually see how the mechanism he's describing works under the hood.

Mark also mentions about other new features that will be available in new Tomcat 7 (which should be released in alpha in Feb 2010).

How to ensure your IE toolbar buttons are always visible?

2010-02-01T16:59:00.004+01:00

Some time ago I encountered a wall while developing add-on for Internet Explorer. I wanted my add-on's button to be always visible after installation process, similarly to delicious stuff.

Problem seems to be quite simple - just check Browser Helper Object's (BHO) options and set some flag (aside of this subject I will tell you that developing pure BHO is a nightmare). In fact there is no such option. Internet Explorer seemed not extensible to this extent. Also finding something in the Internet appeared to be impossible. What the hell? Delicious guys made it!

After quite long searching I was very close to give up and resign from this "feature". But my sixth sense (yes, you should use it a lot while working with Microsoft) was telling me - "check the registry". So I did, and it was a bull's eye shot.

The thing you have to do is to increase the width of Tool Band Width of Internet Explorer (my NSIS script contains this):

WriteRegDWORD HKCU "Software\Microsoft\Internet Explorer\CommandBar" "ToolBandWidth" 800

It's very very primitive but it's the only solution I found to work. You just have to provide the desired width in pixels (decimal value) you'd like IE toolbar to be. Don't worry - you can put there even a value of 3000 - IE is smart enough not to extend the toolbar more than needed i.e. if you put a value of 1000 and your buttons need only 500 pixels IE will automatically adjust toolbar width to 500px.

This solution works but has also a big disadvantage. As you can see I write information to the registry for HKCU (HKEY_CURRENT_USER), so if you are installing the add-on with user Administrator, only this user will have Tool Band enlarged. If you open IE with another user she will probably not see your add-on button(s) (applies to IE7 and IE8).

Until now I couldn't find a better way - did you?

Tomcat 6 Developer's Guide - book review

2010-01-18T08:00:00.004+01:00

Picture (c) Packt Publishing

Few days ago I received brand new "Tomcat 6 Developer's Guide" book which I’ll try to review shortly here. The book is best described in few sentences by the author himself:

"While Tomcat is one of the most popular servlet containers, its inner workings still remain a mystery to many developers. If you only have a superficial familiarity of how this container actually functions, much of its power remains untapped and underutilized. [...]"

Before I start let me just mention that I’m experienced Java and Java EE developer and have worked with Tomcat server (with breaks) since version 4.x (year 2003). This means that I know how to develop, deploy, monitor and tune web applications in this web server. I also worked a little bit with Tomcat internals like custom class loaders. Despite that, this book gave me even more information and knowledge. With this book I was able not only to understand how Tomcat works internally but also where and how I could improve it or basically modify internal components for my needs.

Let’s start the review here - first of all I’d like to say that the author Damodar Chetty meets his goals in this book. He states in the preface that this book is not for web developers who want to learn how to write Java Servlet / JSP applications and how to deploy them in Tomcat container. I totally agree with the author – this book will not teach you that. This book is for people who want to know how Tomcat works internally in order to tune it or modify it for specific needs.

The author promises to go deep inside Tomcat’s architecture and implementation in order to better understand what’s under the hood, how it works and how you can improve or monitor it. I can assure you that this goal is met. This book describes in detail also other aspects of Java EE environment that are not directly connected to the Tomcat container but on which its implementation is based.

In this book Damodar Chetty firstly describes Tomcat’s high-level architecture and then breaks them down up to basic Java classes. This way he makes the whole complex architecture easy to understand. The author describes all Tomcat’s components one by one which is reflected in the Table of Contents. I think the author’s intention here was to make the reader read the book from cover to cover. I think so, because if you just want to learn about one particular component it will be a bit difficult to understand all the references to the previous sections. In my opinion this book should be read entirely to make every aspect clear, otherwise you will need to go back in the book and search for previously explained subjects, which is not easy.
This book is definitely component-oriented rather than how-to- or problem-oriented, so if you have a concrete problem or question regarding Tomcat you may have problems finding answer in this book.

All topics are difficult and in my opinion this book is for experienced Java and Java EE developers and administrators. In this field the book fully satisfied me. More so, many topics like "URLs and protocol handlers", "Java class loading" or "Implementing a custom class loader" may be very useful out of the scope of Tomcat server.

What I like in this book’s style is that all, even most complex subjects are explained in a clear way. Every complex component is broken down into pieces that are more easy to explain and understand. This book is also very concise in the source code examples. This makes it more time-consuming to read but on the other hand it forces the reader to understand the subject rather to copy-paste included source code. This aspect again makes this book accessible for experienced developers who know Java and Java EE environments pretty well and are able to start developing their own examples from the textual description.

This book does not promise to cover all the subjects around Tomcat but few of them are really missing in my opinion. Despite full description of how Tomcat is built and how it works I really miss chapters describing how to configure load-balancer for Tomcat or a cluster of Tomcat web servers. I know there are many books or Internet resources that cover these subjects but it would be just nice to have it here. I also realize that load-balancers and clustering components are not core Tomcat components and maybe this is the reason why they are not covered in this book.

To summarize, I think "Tomcat 6 Developer's Guide" book is a really good one and describes Tomcat web server in a pretty detailed fashion. Please note again that you will not learn from this book how to develop web applications using Java Servlet and JSP technologies. You will learn how to effectively use and also modify tomcat internals for you specific needs. You need to be experienced Java / Java EE developer to fully understand all the topics and successfully apply knowledge from this book. I would basically recommend you reading full description of this book available here before buying it.

Tomcat 6 Developer's Guide is here

2009-12-21T11:32:00.005+01:00

Picture (c) Packt Publishing

Few days ago I was contacted by Amit Sharma (Marketing Research Executive at Packt Publishing) who asked me if I can review their new book Tomcat 6 Developer's Guide. It was published just few days ago and sound interesting, so I agreed.

I expect the book to be rather for advanced users who want to dig in into Tomcat's architecture and more tricky configurations (e.g. SSL and load balancing with Apache front-end, clustering, security realms for Single Sign-In like Kerberos or Windows NTLM).

My short review of the book will follow shortly (in mid Jan 2010). Stay tuned!

Spring 3.0 goes public

2009-12-16T18:51:00.001+01:00

Here it is http://www.springsource.com/download

Spring 3.0 is here and I hope it will serve Java developers as good as the previous releases. Spring Framework used to be de facto Java EE standard and I believe it will remain so for the next couple of years:

For some very recent news, Spring 3.0 GA is compatible with Java EE 6 final in terms of runtime environments now (e.g. on GlassFish v3 as released last week) and supports JPA 2.0 final already (e.g. using EclipseLink 2.0). We also support the newly introduced @ManagedBean (JSR-250 v1.1) annotation for component scanning now, which nicely complements our @Inject (JSR-330) support for annotation-driven dependency injection.

Server timeout for maven release:perform or site:deploy

2009-09-29T13:56:00.003+02:00

I just wanted to release a piece of software I'm working on when I encountered a wall. Big, high and not very talkative wall - read timeout. I thought maybe the configuration of the server (and Web DAV) I described in this post has changed? No way - everything is like it was. The only thing that has changed is Maven version from 2.0.9 to 2.2.1 (where you don't have to define DAV extension in the build part).

Fortunately I also had my old environment where it worked (I meanMmaven 2.0.9). I noticed that the full build takes over 2 minutes while on the new Maven I get the timeout after 1 minute and few seconds. This way I realized that maybe increasing the timeout value will help. IT DID!!!

So how to do this? It's simple - just add:

<servers>
  <server>
    <id>someId</id>
    <username>blahblah</username>
    <password>blahblah</password>
    <configuration>
      <timeout>120000</timeout> <!-- your timeout in milliseconds -->
    </configuration>
  </server>
  ...

in the MAVEN_HOME/conf/settings.xml file and restart your Maven task.

Enjoy!

My daily WTF

2009-09-22T08:46:00.006+02:00

I was just slapped but web application: "Your password must be different from the 12 previous passwords".

Very "secure" (obscure) system forcing me to have my original password (of value "password") plus additional 12 passwords of values "password1", "password2", etc. On the other hand there is no other policy requiring me to have at least one digit or capital letter. The only one is that my new password has to be different than 12 that I had before.

WTF?

Add-in Express™ - a great tool for tiny price

2009-09-20T20:13:00.003+02:00

Picture (c) Add-in Express™

I recently had to develop Internet Explorer plugin that had had to be compatible with versions 6, 7 and 8 of this browser. I searched the net a bit and I found a lot of solutions to do this. The best (if not the only one) solution is to implement Browser Helper Object (BHO). The problem with doing it manually is that it was designed by Microsoft which means that even if implementing it is easy you have to store zillions of information in the Windows registry to make your application work. This, on the other hand, is not that easy as you have to know exactly what and where to store, how to generated COM classes' GUIDs, and if it's not enough this kind of work is not testable or debugable.

Of course, it's easy to find some ready solutions with demo how to register all objects, etc. Another problem with this is that all the documentation from MS or from opensource community is pretty old and irrelevant. For example some example plugins I found worked out-of-the-box, however I had some special needs that were not addressed and that I couldn't find any documentation for.

Let's try the other way
In the previous paragraphs I forgot to mention that the software I was up to develop had to be delivered in two days (I mean the demo showing that it's possible to develop such plugin at all). This way I didn't have time to dig into the COM docs and BHO registration options. I found Add-in Express™ plugins for Visual Studio. My obvious choice was Add-in Express™ 2009 for Internet Explorer® and Microsoft® .net (AE4IE).

Benefits
With this tool I achieved more in 5 minutes than with Microsoft and opensource documentation in two days. It's not an exaggeration if I say that every developer is able to create at least working skeleton of IE plugin in 15 minutes. AE4IE has excellent documentation and is extremely easy to use. ROI is obvious and very quick as this tool gives a great quality for the price it costs.

With AE4IE templates for Visual Studio you not only create plugins in 5 minutes. This tool also creates the whole project structure as well as the Setup project that enables your users installing the plugins in the user-friendly manner. And it takes only few clicks!!! I love it.

It saves a lot of time and hard work of digging into MS documentation or reading (and buying) books on BHO/COM subjects.

Another good point is that developers are given pure Microsoft API such as IWebBrowser2 - this way you gain full power over your plugin and don't have to worry about COM specification and Windows registry configuration.

What about the support
Add-in Express™ team replies and solves problems very quickly and professionally. There is a special forum where you can find answers for many real-life questions and find solutions to common problems.

Extremely useful thing is also having a technical blog of Add-in Express™ team where they present new features and explain how to solve advanced issues (e.g. with sharing data between COM objects).

Conclusions
I'm very happy with this tool and would recommend it to every small or big company that has to develop IE plugin. You will never do what they did for such a small amount of money. The whole purchasing procedure takes 5 minutes and after another 15 minutes you have your plugin ready. The only remaining thing to do is to implement the business logic. How cool is that?

DISCLAIMER: I have no business connections with Add-in Express™ company and my review is not a commercial break

Visual VM - a great Java tool you were waiting for!

2009-07-23T10:00:00.001+02:00

Picture (c) VisualVM

I planned to publish this post some months ago but unfortunately I was too busy to do so. I'm still busy :) but I can write couple of sentences about VisualVM - a VISUAL (no surprise) tool for monitoring Java applications. If you had to monitor state of your Java systems you probably already know JConsole - a JMX console to you Java stuff. Of course, you can just connect to JVM's JMX services and monitor almost every aspect of VM but the way data is not presented in a very friendly manner. Some differences between VisualVM and JConsole are described in this article.

Anyway, VisualVM is an absolutely GREAT tool to monitor Java apps. You are able to see all threads (with possible deadlocks), memory state (heap and stack), profiling information on CPU and memory utilization, etc. VisualVM has pluggable architecture and you can find many useful addins to it including JConsole plugins. This way you will be able to use only on tool and get rid of JConsole (which is a great tool too).

More details and full-blown tutorial can be found here: http://www.taranfx.com/blog/?p=930

More advanced topics are described here: http://blog.grovehillsoftware.com/2009/05/visualvm-and-cutting-method-calls-by.html

Présentation de Scrum pour Riviera JUG

2009-07-16T22:33:00.002+02:00

Picture (c) Riviera JUG

Hier, j'ai présente Scrum exclusivement pour Riviera Java User Group dans le Mistral Auditorium d'Amadeus. Mon public, environs 120 personnes est venu de Sophia Antipolis, Antibes et de région de Nice en général. Selon le retour que j'ai reçu après ma présentation, elle était un grand succès. Maintenant j'attends les réponses de participant au sujet de la création et la participation dans Riviera Agile User Group. J'espère ce groupe sera bientôt crée et nous pourrons réunir régulièrement et présenter nos idées et échanger nos expériences a'propos d'agile.

Si vous êtes intéresses par la projet comme Riviera Agile User Group, vous pouvez me contacter.

Si vous êtes (ou vous travaillez pour) une société qui pourrait financer les réunions mensuels (nous aurons besoin de la salle de conférence, des boissons et snacks environs 200-400 EUR) , je serai ravi de le savoir. Cela serait une bonne promotion pour vous car les méthodes agile gagnent plus en plus de la popularité ici au sud de France - vous serez bien VISIBLE si vous décidez de nous aider.

La résume de la dernière présentation, aussi que les slides et les photos vous pouvez trouve ici.J'espere que la video va apparaitre bientot.

Scrum presentation for Riviera JUG

2009-07-16T21:11:00.004+02:00

Picture (c) Riviera JUG

Yesterday I presented Scrum exclusively for Riviera Java User Group in Amadeus' Mistral Auditorium. My audience was about 120 people from Sophia Antipolis, Antibes and Nice area in general. According to the feedback I received the presentation was a big success and now I'm waiting for the answers from attending people whether they are interested in creating and participating to Riviera Agile User Group. I hope we will create such initiative shortly and will be able to meet regularly presenting ideas, thoughts and experiences on agile subjects.

If you are interested in open initiative such as Riviera Agile User Group please contact me.

If you are (or work for) a company that would like to sponsor such monthly events (we will need conference room, some beverages and snacks i.e. 200 - 400 EUR) I would love to know about it. It will be a great marketing for you as the agile methods gain more and more popularity here in southern France - you will be very VISIBLE if you decide to help us.

Summary of the last meeting, slides and pictures can be found here. I hope the video will follow shortly too.

Half-dead project saved! Thank you - Direct Communication

2009-06-20T22:12:00.001+02:00

Picture (c) Przemyslaw Bielicki

In my previous post I shortly described the half-dead project in an over-waterfall company me and my team had to save three months before production. In this part I will show you how direct (instead of discrete) communication with customer or good Product Owner in general can help saving almost dead projects.

As I described earlier me and my team had a lot of problems with the systems we had to integrate - technical issues were present but they were not dominant, even taking into account the fact that the team was not experienced in .NET and T-SQL stuff.

Customer saved our project
After some time struggling with technical issues we finally were contacted by the person who knew how the product should behave. This person (let me call her Product Owner - PO) had all acceptance tests in her mind and knew exactly how to help us understand the product. PO was in fact our client - to be completely clear, the product we were delivering was either win-win or lose-lose so we had to cooperate and the success was our mutual goal.

PO was remotely available (although in the same time zone) but that was not a problem for us at all - we exchanged tons of emails and some phone calls and that was perfectly sufficient.

We were working like this for over two months and we delivered every single customer's request to the product. After the development phase our software had to pass the certification phase which, in fact, was couple-hour testing process in production. Our PO was present there and was helping other clients with testing (yes, there were other clients eventually using our product).

I could risk a claim that it was a huge success because we just found couple of minor issues that were not even concerning the software but insufficient or incomplete data in our production databases. Everybody is sure that what saved the project and what really caused it to be such success was direct contact and constant help from our PO. As I wrote in one of my previous posts, namely Customer Team Member - a way to winning together having customer as your team member is usually really good and efficient idea.

Fool's gold?
There are also (of course) problems with such attitude - the most "popular" one is the problem with actually having a customer. But even if you have a customer, how would you encourage him/her to be involved in the project (they usually aren't, especially at the beginning of the project)? It may depend on the actual contract i.e. how you define success or whether customer gain something by helping you - to know which contract best suits your current situation please refer to Peter Stevens' article: 10 Contracts for your next Agile Software Project.

It saved us - it can save You
To summarize, from my experience every time I worked with customer that was involved in the project I produced software with the highest business value (at that time). Every time I worked directly with involved customer I delivered the product that everybody treated as theirs, everybody was the owner - that's very cool feeling. This has also real business effect - satisfied customer is more likely to come back to your company. You gain credibility working together with your customers e.g. by showing them that you really work hard to meet their needs and to bring the best business value reducing costs. That usually pays off.

So, my final advice here for you would be to take some effort and time and look for a GOOD Product Owner. When you find her then start bombing her with emails, phone calls, whatever that can help your team deliver the product with highest possible business value.

What are your experiences with working directly with customers? Are they positive or negative? How would you improve thing in your project - could customer's involvement help?

Originally published on AgileSoftwareDevelopment.com

The Vendor Client Relationship (In Real World Situations)

2009-06-10T09:19:00.002+02:00

ROTFL

Copied from {codesqueeze}.

Sysinternals (Process Explorer) saved my a**

2009-05-20T08:00:00.003+02:00

After deploying our .NET system in production functionally speaking everything was OK. However, after couple of days we saw our processes consuming more than 80% of the CPU of some pretty powerful servers - and we were not computing any nuclear or medical killer stuff. Something was wrong - but what?

At this moment I recalled using free Sysinternals tool from Microsoft to see what's deep inside my operating system (in runtime, of course).

What really happened?
I started Process Explorer belonging to the Sysinternals suite, found my process and looked at it's runtime properties:

What I saw was the increasing number of TCP/IP connections (because used connections were not closed/returned to the pool properly). After some time spent on monitoring the process I noticed that this pattern is stable and each request caused the number of connections to database grow - until the process reached max and OS couldn't handle this process any more. Of course I fixed this issue wit one or two lines of code because it was a "little" problem (as usual) that appeared to have a big impact on the operating software.

I think wouldn't be able to discover this issue so quickly without using Sysinternals. Why? Because my colleagues were trying to investigate and fix this issue for some time before me. They didn't know Process Explorer tool :)

Direct communication as a half-dead project saver: When you start in an over-waterfall company

2009-05-18T09:00:00.000+02:00

Picture courtesy of clagnut@flickr

At the end of year 2008 our team was given a project that was critical from the $$ point of view. We had to deliver the project by the end of March 2009 in order to avoid huge fees from our customer. The problem was that the project we got was being developed by different team in different location and the quality of the existing solution was at least questionable. It turned out that the whole old team left the company and our small commando squad had to save the day. I just have to mention that the company we all worked for was a pure-waterfall monster.

You know, agile books are fun to read, but how do you start it if you happen to be a developer in a pure waterfall company on an impossible project? I went through it and going to show what worked well for us. I will tell how we were able to get an impossible project apply the agile principles and survive.

A bit of a history
The company I used to work for sometime didn't know/use Agile methods. Everything there was pure waterfall i.e. there was a marketing that defined what it wanted (previously negotiated the contract i.e. feature list with the customer), then it passed the specification docs to product definition team that was in charge of designing the solution, finding ALL possible problems and answering ALL possible questions. Then after couple of months the technical specifications were passed to the development team that... started everything all over again. I mean that the development team usually found documents received from product definition team useless - it's not my opinion, rather my conclusion after talks with many friend-developers. Some docs like protocol specifications or conversion specs were REALLY useful but they were vast minority.

Anyway, when developers started working on the project it usually took just few classes to find some blocking issues that had not been predicted/designed in the docs. "How it's possible?" "They were supposed to find ALL possible problems and answer ALL possible questions". Well, they were supposed to but it was IMPOSSIBLE to do so. Even perfectly prepared and thoroughly checked documentation will not replace direct communication channels.

The project I'm going to describe was quite different - there was no documentation, no specification but more so there was not a single person understanding the problem. This was a real shocker for the company used to work in an "ordered" and "predictable" way.

The Project
In short we had to develop an unknown communication protocol to connect the two unknown existing systems. And the time we were given was pretty much estimated as we already knew all the guts and gears in this system. Ah - I forgot about some pretty "unimportant" detail: the test database for the test system was located around 1000 km from our desks and the VPN connection was not yet established. So, for couple of weeks we were just having a pretty "black box" environment - it was not so bad though because we had some time to learn how to compile given sources (still, this was the time when we were supposed to be developing software).

What did we have?
We had the documentation for the protocol we had to implement with examples (requests and responses). The documentation was not very useful, though because it was designed for people who knew this protocol before. This documentation was not explaining many concepts - it was assuming the reader's knowledge on quite high level - unfortunately we were GREEN.
We also had some stored procedures (in our unreachable database) that were supposed to contain business logic and they were supposed to work already - we were told that they require just small changes. Basically we had to develop a part that had to receive the requests, parse and process it and invoke already existing stored procedure with transformed parameters – we knew that we would also modify or create completely new stored procedures from scratch.

When we eventually got to know what to do we knew that we had to implement hotel availability (3 different request/responses) and booking (couple of quite complex transaction flows) messages. The availability part was supposed to be 100% ready (this way we could focus only on booking) - we just had to test it and confirm (probably do some small fixes).

To summarize: we had some germ of the C# code, fully functional stored procedures in T-SQL (MS SQL server) that required some significant rework and proprietary ASCII format specification. I also have to mention that neither of the team members was C# or T-SQL expert (I put it as delicate as possible).

Finally @Work
After some time of investigation we knew how the frontend and backend work but we still couldn't test the system because we had no database. When it was ready we established a test environment and we finally was able to integrate the whole system. We were able to start this machinery and inject some requests. We were even receiving some responses - but at this moment we had completely no idea whether these responses were correct or not.

Here is the moment we started communicating directly with our customer who started testing delivered system in test environment and giving us feedback. If you ask why we didn’t communicate with them earlier I would answer that we had nothing to show them as the test environment was unreachable for developers. If we couldn’t show them anything we weren’t able to get any feedback. That sucked…

I will leave the details of our communication process with customer for the next post.

Lessons learned so far
We took over the project in a rush and were just given a source codes from the previous team – which wasn’t the first class anyway (I mean the code). There was no unit tests, no continuous integration environment, no single product backlog but we started creating those artefacts. And you know what – this was the best decision we could make - we took as much as we could from the agile world in this project.

The main problem that was probably the root cause of all other issues was lack of communication. Before starting contacting directly with customer (who became our product owner, in fact) this was the worst part of the project because we not only didn’t know what to do but also we had no one who could verify our assumptions. Lack of communication was very visible and the deadlines were getting closer. We were really thinking that we will be late with such pace and information flow (or rather lack of it). We knew that with such level of uncertainty agile practices - especially open and effective communication - could help us saving our jobs.

In the next part I will explain how we solved our communication problems, how agile practices helped keep the project on track and keep the number of bugs close to zero as well as how the whole project finished. Stay tuned!

Feedback
Do you recall such situation in your own project(s)? What was the solution that worked for your team? I'm really curious your opinions and thoughts.

Originally published on AgileSoftwareDevelopment.com

Niezły kawałek muzyki

2009-05-15T20:36:00.004+02:00

This post will be in Polish but don't worry it won't be about Java or technology at all. Anyway, if you want to listen to some really good music (one of the artist - Piotr Czerski - is my friend from university and some really wasting parties ;) visit this website: http://towary.art.pl.

Mam nadzieję, że Groszek się nie obrazi jeśli skopiuję post z jego bloga - nic lepszego i tak nie wymyślę na temat nowiuteńkiej płyty grupy "Towary Zastępcze" pt. "Dolne Miasto" bo o tym będzie ten wpis. Przesłuchałem tę płytę już dzisiaj dwa razy i jestem bardzo kontent.

A oto co do powiedzenia ma na temat płyty Groszek (ze wszystkim się w stu procentach zgadzam):

Na płytę składa się dwanaście zaśpiewanych wierszy, opowiadających strzępy tragicznej historii dziewczyny z Grand Hotelu. Szczególną uwagę zwraca strona internetowa towary.art.pl promująca album, na której możemy wcielić się w rolę detektywa prowadzącego śledztwo. Oglądając kolejne fragmenty video i podążając właściwym tropem, z pewnością uda nam się zdobyć odpowiedzi na trzy pytania, co pozwoli ściągnąć album w formacie mp3 za darmo z sieci. Wytrwali mogą również odnaleźć prawdziwego mordercę i dotrzeć do kolejnych bonusów.

Album dostępny jest również w bardziej namacalnej postaci. W atrakcyjnej cenie można nabyć akta sprawy, w których jednym z dowodów jest płyta. Doskonały pomysł na nietuzinkowy prezent i wsparcie twórczości artystów. Gorąco polecam!

Ja też polecam - kawał dobrej roboty i muzyki!

Java Concurrency Bugs/Programmer's Errors

2009-05-15T15:13:00.004+02:00

Alex Miller has been posting very interesting series on Java concurrency bugs (or pitfalls programmers could fall into). Especially interesting is the last post: Inconsistent synchronization. This post is most probably inspired by the problems Alex discovered in Hibernate statistics and I am pretty proud I was able to find possible concurrency issues reading the code. Anyway, I was not able to fully and correctly solve those issues.

The conclusion for me is that I thought about "magic" solution instead of the correct one i.e. I suggested that declaring a integer field as volatile would solve the issue. In fact it would but the "correct" solution to this problem is using AtomicInteger from java.util.concurrent.atomic package.

Another conclusion is to remember to keep bombarding your Java code with FindBugs - it can find many many Java issues in your code (not only related to concurrency).

The rest of the Alex's series (so far) is here:
Java Concurrency Bugs #4: ConcurrentModificationException
Java Concurrency Bugs #3 - atomic + atomic != atomic
Java Concurrency Bugs #2 - what to synchronize on
Java Concurrency Bugs #1 - mutable statics

Enjoy you Java Concurrency Issues ;)

Fun soft: regex checker

2009-05-04T11:00:00.002+02:00

I'm not a regular expressions expert but I use them often - as probably most of software developers. Before using them I want to be sure my regexes are written correctly and they will serve their purposes. I wrote very small command-line app for doing the check for me :) As a first input user provides the regex to check, then the user can type all texts to check against previously given regex until she types 'q' or 'exit' (for which program exits). Enjoy!


package com.bielu;

import java.io.Console;

public class RegexTester {
  public static void main(String[] args) {
    Console in = System.console();
    if (in == null) {
      System.err.println("Cannot obtain system console - exiting application");
      return;
    }

    System.out.print("Enter regex to test: ");
    String regex = in.readLine();
 
    while (true) {
      System.out.printf("Type value to match with given regex '%s': ", regex);
      String string = in.readLine();
  
      if (string.equalsIgnoreCase("q") || string.equalsIgnoreCase("exit")) {
        break;
      }
  
      if (string.matches(regex)) {
        System.out.printf("'%s' matches '%s' regex\n", string, regex);
      } else {
        System.out.printf("'%s' does NOT match '%s' regex\n", string, regex);
      }
    }
  }
}

Certified Scrum Master Training in Gdańsk

2009-04-27T21:09:00.008+02:00

I hope you already know that a cool company that is definitely one generation ahead than their competitors (at least as far as the way they work is concerned) called Spartez organizes the first CSM Training in northern Poland, namely in Gdańsk. The training will be led by famous Henrik Kniberg from whose book my adventure with real Scrum began. It is really a PITTY I will not be there :(

But anyway - if you live somewhere in the central, eastern or northern Europe this training is definitely worth taking. I took CSM course in Zurich last week led by Angela Druckman and am very satisfied. This training is really worth it's price (or even more because I think the training is quite cheap).

Don't waste your time and sign up for this CSM session.

The power of the final keyword - updated

2009-04-17T16:00:00.016+02:00

Picture courtesy of Okko Pyykkö@flickr

These two articles of Alex Miller:

A final bit of advice

JMM and final field freeze

clearly explain why using final fields in concurrent programs is extremely powerful and safe (thread-safe).

Take a look a this simple example:


public class SomeClass implements Runnable {  
    private final LinkedHashMap map;
      
    public SomeClass(LinkedHashMap copiedMap) {
        map = copiedMap;
        preProcessMap();
    }  
     
    private void preProcessMap() {
        //some non-trivial and not-so-short computations
    }

    // A read method  
    public Object get(Object key) {  
        return map.get(key);  
    }  
     
    public void run() {
        //assume that in this method only 
        //unsynchronized read operations on map may occur
        //any unsynchronized write operation is not thread-safe
    }
    // etc...  
  }

public class Main {
  public static void main(String[] args) {
      LinkedHashMap map = // map creation
      // map initialization
      new Thread(new SomeClass(map)).start();
      // etc.
  }
}

UPDATE:
I preserved the original version of the code but after some thoughts and thanks to some comments (thank you dear readers) I conclude that this example is not 100% correct. Before Thread.start() could be started the object has to be fully initialized and initialization will be completed only AFTER the SomeClass object is created. This way final keyword doesn't add any value there. On the other hand I meant this piece of code:


public class Main {
    public static void main(String[] args) {
      LinkedHashMap map = // map creation
      // map initialization
      final ExecutorService service = Executors.newFixedThreadPool(1);
      service.submit(new SomeClass(map));
      // etc.
    }
}

In such case I'm pretty sure you cannot assume that e.g. second processor (in a multi-core platform) will wait before executing newly scheduled task until the object of class SomeClass is fully initialized, right? Correct me if I'm wrong because I feel have some mess in my brain now ;)
End of UPDATE

Whatever happens in the preProcessMap() method you can be sure that LinkedHashMap will be copied and preprocessed BEFORE starting the run() method by the new thread. This way any READ (get) operation on this map in all threads will be safe and will operate on completely and correctly initialized map object.

If only you remove final keyword from LinkedHashMap declaration you cannot be sure our previous assumption. It may happen that the thread will be started BEFORE copying the map and if in run() method you process this map in any way you may get NPE.

Let me use a bottle metaphor :) For me final keyword is like the guarantee that you closed the bottle of whisky before passing it to your crazy friend who likes juggling them. If you ensure that the bottle is closed (final) before passing it to your friend you can be sure he will not spill any of the precious whisky on the floor. On the other hand if you start passing the bottle to him while trying to close it you're not sure whether it's fully closed while he starts juggling it or not. It may be closed 100% of time when you meet in your place but when you go to your other friend's party he may spill very expensive whisky and break the bottle in the same time (test and production environments metaphor this time).
Conclusion? Close the bottle before giving it to anybody else.

I will not be original and simply copy Alex Miller's advice:

Final fields are ever so important and useful. Whenever possible, strive to make your fields final. One tip for this is to enable a Save Action in Eclipse (or your IDE of choice) that automatically attempts to make fields final if possible so you can’t forget!

Why Object-Oriented languages rock?

2009-03-24T14:13:00.001+01:00

I loved coming back from using T-SQL to C# or Java like I loved the view from the picture (view from the l'Esterel's mountains towards Cannes). Of course, SQL-like languages are very powerful and when you do some complicated computations that need many requests to the underlying relational database, this choice may be the right one. At least, as far as the performance is concerned.

If we leave performance concerns behind we still have the language itself. So, what did I miss in this language? I was thinking of enumerating things that I didn't like but after some thinking I conclude that I just missed object-orientation in general.

But what it means? Let's see... I was not able to use any design pattern (e.g. Factory, Strategy or Decorator) and I needed them very badly! Instead of using design patterns I had to add many ugly ifs.

Another thing I missed was abstraction i.e. I wasn't able to create abstract stored procedure and then implement specialized subprocedures. In T-SQL everything you can do is to add some more ifs :) This language is strongly data-oriented which is maybe good but it's not sufficient IMHO.

Yes - it's very vague but it doesn't make sense to compare stored procedures with OO constructs. T-SQL is just yet another language I've learnt and used after many years of writing Java/C++ code and I just wanted to express my feelings.

I don't know what are perspectives for such SQL languages like T-SQL or PL/SQL but I think they should evolve in an object-oriented way. It would be much easier to write stored procedures/classes in e.g. Java, test them and then deploy on the server. This would be a win-win situation. We would gain performance by having stored code invoked directly on the SQL server while we will still be able to use OO constructs and profit from unit tests, design patterns, etc.

Maybe there already are such languages but I just don't know them (most of them are procedural languages) - if you know some let me know.

Everybody Needs ... Unit Tests

2009-03-02T10:16:00.001+01:00

Picture courtesy of Lotus72@flickr

... please remember people, no matter who you are, and whatever you do to live, thrive and survive, there are still some things that make us all the same: you, me, him, them - everybody, everybody! - this is a fragment of Blues Brothers' "Everybody Need Somebody" song. Nothing to do with software development but when you change the next line of this song from "Everybody needs somebody" to "Everybody needs Unit Tests" it becomes closer to software development.

I intentionally used lyrics of this song to keep everyone's attention (just a little bit) to this post. If you are familiar with TDD and unit test your code you can disregard it - I'd like to address this post to all software engineers/developers/programmers/etc. that DO NOT unit test their code.

In this post I will try (again) to convince all software developers that still don't unit test their code that it's right thing to do and it's damn simple.

Assume that Agile software development is wrong, really wrong

You don't have to be the fan of Agile movement - you can even think it's stupid, immature and doesn't help software development at all. Maybe you're right, maybe not - it's rather a question of taste. You don't have to use or like Scrum, you may think that XP is crazy and an overkill - you have right to think so. I can understand everything but PLEASE do unit test your code! Forget about Agile, forget about processes and methodologies - just unit test your code.

I just took over another project that is quite complex (of course it is!) but has NO SINGLE unit test (or any test at all). More so, there is no documentation of any kind. Yes I can read the code and see what it does - that's simple but how the hell I could know it does what it should do? How could I know it's not a mistake made by the developer who created this creature?

This code lacks Unit Test, but fortunately day after day I add more and more and I'm more confident this piece of software works.

Unit Testing once again

If you want to know something more about unit testing check out our previous articles. I will just recapitulate the most important, in my opinion, qualities of unit testing:

unit tests document internal and external architecture of the software system

unit tests help you and other developers to immediately see whether code "improvements" broke already existing code

unit tests help making your software bug-free - all issues should be addressed by writing a failing test first

unit tests together with code coverage tools improve quality of your software

unit tests can be used as a presentation platform - you can present hard to visualise issues using unit testing green bar - green means that it works

unit tests significantly improve team confidency in progress - if all the tests pass it means that all features implemented so far work and nobody broke the system

unit tests improve efficiency - you don't have to perform laborious manual tests again and again after code changes, just start unit tests and watch the red/green bar

already there but so important that worth reminding - unit tests work as a DOCUMENTATION (auxiliary or main one)

I'm open for your propositions

I hope I convinced you to start writing unit tests - if you don't like or believe in Agile movement it's OK. Unit Testing is just an obligatory practice of every decent software developer. If you do unit test it doesn't mean you're good one but if you don't unit test you're a poor developer (I mean it).

I wonder when unit testing will be taught obligatory at every decent university. It does so many good and is so simple - people, please - write unit tests!

If you have any ideas how to convince every developer that unit testing is right thing to do I'm open for suggestions.

Originally published on AgileSoftwareDevelopment.com

Synchronized collections in Java 5+

2009-02-16T08:00:00.001+01:00

Picture courtesy of Swamibu@flickr

You probably encountered problem of which collection to use while writing concurrent applications. Which implementation to use when you have N concurrent writers and M concurrent readers? I will try to shortly answer this question on simple example.

The code below presents usage of different collection implementations while having 16 concurrent writer threads and 100 concurrent reader threads. At the same time writer threads modify the collection i.e. add new elements and reader threads iterate through actual collection state and operate on its elements.


public class ConcurrentCollectionsTest {
 public static final int READERS_COUNT = 100;
 public static final int COLLECTION_SIZE = 5000;
 public static final int WRITERS_COUNT = 16;

 static volatile int readExceptionCount = 0;
  static CountDownLatch latch;

 public static void main(String[] args) throws Exception {
  Map<String, Collection<String>> map = 
    new LinkedHashMap<String, Collection<String>>();

  map.put("ArrayList", 
      new ArrayList<String>());

  map.put("Synchronized ArrayList", 
      Collections.synchronizedCollection(
        new ArrayList<String>()));

  map.put("ConcurrentLinkedQueue", 
      new ConcurrentLinkedQueue<String>());

  map.put("CopyOnWriteArrayList", 
      new CopyOnWriteArrayList<String>());

  for (Entry<String, Collection<String>> e: map.entrySet()) {
   Collection<String> collection = e.getValue();
   latch = new CountDownLatch(READERS_COUNT + WRITERS_COUNT);
   readExceptionCount = 0;

   long start = System.currentTimeMillis();
   for (int i = 0; i < WRITERS_COUNT; ++i) {
    Thread t = new Thread(new Writer(collection));
    t.start();
   }

   for (int i = 0; i < READERS_COUNT; ++i) {
    Thread t = new Thread(new Reader(collection));
    t.start();
   }

   latch.await();
   System.out.println("-----------------");
   System.out.println("Stats for [" + e.getKey() + "]:");
   System.out.printf("Collection's size: %d\n", collection.size());
   System.out.printf("ConcurrentModificationException count [read]: %d\n", readExceptionCount);
   System.out.printf("Duration: %d\n\n", System.currentTimeMillis() - start);
  }
 }

static class Reader implements Runnable {
 private final Collection<String> collection;

 Reader(Collection<String> col) {
  this.collection = col;
 }

 public void run() {
  try {
   for (Iterator<String> i = collection.iterator(); i.hasNext();) {
    // additional computing operation just to consume some time
    String t = i.next() + "_added";
    if (t.length() > 10 && i.hasNext()) {
     i.next();
    }
   }
  } catch (ConcurrentModificationException e) {
   readExceptionCount++;
  }
  latch.countDown();
 }
}

static class Writer implements Runnable {
 private final Collection<String> collection;

 Writer(Collection<String> col) {
  this.collection = col;
 }

 public void run() {
  try {
   for (int i = 0; i < COLLECTION_SIZE; ++i) {
    collection.add(Thread.currentThread().getName() 
        + System.currentTimeMillis());
   }
  } catch (Exception e) {
   // ignore
  }
  latch.countDown();
 }
}
}

Results on my machine (numbers can vary on different hardware but problems/conclusions will be the same):
----------------- Stats for [ArrayList]: Collection's size: 77038 ConcurrentModificationException count [read]: 17 Duration: 1047 ms ----------------- Stats for [Synchronized ArrayList]: Collection's size: 80000 ConcurrentModificationException count [read]: 4 Duration: 1141 ms ----------------- Stats for [ConcurrentLinkedQueue]: Collection's size: 80000 ConcurrentModificationException count [read]: 0 Duration: 1046 ms ----------------- Stats for [CopyOnWriteArrayList]: Collection's size: 80000 ConcurrentModificationException count [read]: 0 Duration: 14344 ms
Conclusions
ArrayList (unsurprisingly) doesn't work at all - there is 17 ConcurrentModificationException on read attempt and what is more unacceptable this collection is in unpredictable state i.e. it's size should be 80000 instead of 77038 - it means that many items were not stored by the writer threads.

Synchronized ArrayList works better i.e. its state is correct but still you need additional synchronization in order to get rid of ConcurrentModificationException while iterating through it.

ConcurrentLinkedQueue works best in this case and does not add any performance penalties.

CopyOnWriteArrayList is the worst in this case because traversal operations does not vastly outnumber mutations.

The last conclusion would be to perform such test before deciding which implementation works best in your particular case. Also, read javadoc carefully to see implications and suggested use cases.